Skip to content

feat: compressible proc macro draft #1857

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 44 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
5172b84
wip
SwenSchaeferjohann Jul 3, 2025
e534deb
add compress_pda helper
SwenSchaeferjohann Jul 3, 2025
a81442c
compress_pda compiling
SwenSchaeferjohann Jul 3, 2025
fc30a31
decompress_idempotent.rs
SwenSchaeferjohann Jul 4, 2025
b3363e4
wip
SwenSchaeferjohann Jul 4, 2025
ef558bb
wip
SwenSchaeferjohann Jul 4, 2025
4231fb7
decompress batch idempotent
SwenSchaeferjohann Jul 4, 2025
12694e5
wip
SwenSchaeferjohann Jul 4, 2025
897a2a1
add compress_pda_new and compress_multiple_pdas_new
SwenSchaeferjohann Jul 4, 2025
23073b3
native program with decompress done
SwenSchaeferjohann Jul 6, 2025
a84280b
compress_dynamic, decompress_dynamic
SwenSchaeferjohann Jul 6, 2025
f0a694b
wip
SwenSchaeferjohann Jul 6, 2025
c0fbfcc
adding anchor
SwenSchaeferjohann Jul 6, 2025
2c3ce19
testprogram uses sdk
SwenSchaeferjohann Jul 6, 2025
fef023d
fix compilation
SwenSchaeferjohann Jul 6, 2025
6773f81
wip
SwenSchaeferjohann Jul 6, 2025
bbd77d2
experiment with procmacro
SwenSchaeferjohann Jul 6, 2025
41498da
skip SLOT check at compress_pda_new
SwenSchaeferjohann Jul 6, 2025
e9b9a71
wip
SwenSchaeferjohann Jul 7, 2025
70bbba4
add_compressible_instructions() works for compress, idl gen works
SwenSchaeferjohann Jul 7, 2025
13d5418
add proc macro
SwenSchaeferjohann Jul 7, 2025
7e5689b
decompress_multiple_pdas is working as it should
SwenSchaeferjohann Jul 7, 2025
4122e3c
fix decompress_idempotent impl
SwenSchaeferjohann Jul 8, 2025
800792d
rm expanded
SwenSchaeferjohann Jul 8, 2025
2a4c73c
add remove_data
SwenSchaeferjohann Jul 8, 2025
27749d3
force apps to pass the whole signer_seeds directly
SwenSchaeferjohann Jul 9, 2025
5c4e16a
add compressible_config draft
SwenSchaeferjohann Jul 9, 2025
a71aadf
add create_config_unchecked and checked
SwenSchaeferjohann Jul 9, 2025
e7dcc59
use config, add unified header struct (just last_written_slot) for now
SwenSchaeferjohann Jul 9, 2025
a194292
use hascompressioninfo and compressioninfo
SwenSchaeferjohann Jul 10, 2025
9783cdf
add config support to compressible macro
SwenSchaeferjohann Jul 10, 2025
3f4f1d6
add expanded.rs
SwenSchaeferjohann Jul 10, 2025
698cadc
cleanup anchor-derived example
SwenSchaeferjohann Jul 10, 2025
cc17cef
add support for multiple address_trees per address space
SwenSchaeferjohann Jul 10, 2025
4161d3d
add support for multiple address_trees per address space
SwenSchaeferjohann Jul 10, 2025
cfd0202
update macro to multiple address_trees
SwenSchaeferjohann Jul 10, 2025
0e3e5f4
add test-sdk-derived program
SwenSchaeferjohann Jul 11, 2025
2c0078d
wip
SwenSchaeferjohann Jul 11, 2025
0c6be6a
cleanup native macro-derive example
SwenSchaeferjohann Jul 11, 2025
054bd7c
wip
SwenSchaeferjohann Jul 14, 2025
b321d87
fix compilation
SwenSchaeferjohann Jul 15, 2025
dc36e51
config tests working
SwenSchaeferjohann Jul 16, 2025
388279f
clean up test_config.rs
SwenSchaeferjohann Jul 16, 2025
2c34cfb
testing
SwenSchaeferjohann Jul 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
698 changes: 302 additions & 396 deletions Cargo.lock

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ members = [
"forester-utils",
"forester",
"sparse-merkle-tree",
"program-tests/anchor-compressible-user",
"program-tests/anchor-compressible-user-derived",
"program-tests/sdk-test-derived",
]

resolver = "2"
Expand Down Expand Up @@ -90,6 +93,7 @@ solana-transaction = { version = "2.2" }
solana-transaction-error = { version = "2.2" }
solana-hash = { version = "2.2" }
solana-clock = { version = "2.2" }
solana-rent = { version = "2.2" }
solana-signature = { version = "2.2" }
solana-commitment-config = { version = "2.2" }
solana-account = { version = "2.2" }
Expand Down
13 changes: 3 additions & 10 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

45 changes: 45 additions & 0 deletions program-tests/anchor-compressible-user-derived/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
[package]
name = "anchor-compressible-user-derived"
version = "0.1.0"
description = "Anchor program template with user records and derived accounts"
edition = "2021"

[lib]
crate-type = ["cdylib", "lib"]
name = "anchor_compressible_user_derived"

[features]
no-entrypoint = []
no-idl = []
no-log-ix-name = []
cpi = ["no-entrypoint"]
default = ["idl-build"]
idl-build = ["anchor-lang/idl-build", "light-sdk/idl-build"]

test-sbf = []


[dependencies]
light-sdk = { workspace = true, features = ["anchor", "idl-build"] }
light-sdk-types = { workspace = true }
light-sdk-macros = { workspace = true }
light-hasher = { workspace = true, features = ["solana"] }
solana-program = { workspace = true }
light-macros = { workspace = true, features = ["solana"] }
borsh = { workspace = true }
light-compressed-account = { workspace = true, features = ["solana"] }
anchor-lang = { workspace = true, features = ["idl-build"] }

[dev-dependencies]
light-program-test = { workspace = true, features = ["devenv"] }
light-client = { workspace = true, features = ["devenv"] }
light-test-utils = { workspace = true, features = ["devenv"] }
tokio = { workspace = true }
solana-sdk = { workspace = true }

[lints.rust.unexpected_cfgs]
level = "allow"
check-cfg = [
'cfg(target_os, values("solana"))',
'cfg(feature, values("frozen-abi", "no-entrypoint"))',
]
278 changes: 278 additions & 0 deletions program-tests/anchor-compressible-user-derived/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,278 @@
# Example: Using the add_compressible_instructions Macro

This example shows how to use the `add_compressible_instructions` macro to automatically generate compression-related instructions for your Anchor program.

## Basic Setup

```rust
use anchor_lang::prelude::*;
use light_sdk::{
compressible::{CompressionInfo, HasCompressionInfo},
derive_light_cpi_signer, LightDiscriminator, LightHasher,
};
use light_sdk_macros::add_compressible_instructions;

declare_id!("YourProgramId11111111111111111111111111111");

// Define your CPI signer
pub const LIGHT_CPI_SIGNER: CpiSigner =
derive_light_cpi_signer!("YourCpiSignerPubkey11111111111111111111111");

// Apply the macro to your program module
#[add_compressible_instructions(UserRecord, GameSession)]
#[program]
pub mod my_program {
use super::*;

// The macro automatically generates these instructions:
// - create_compression_config (config management)
// - update_compression_config (config management)
// - compress_user_record (compress existing PDA)
// - compress_game_session (compress existing PDA)
// - decompress_multiple_pdas (decompress compressed accounts)
//
// NOTE: create_user_record and create_game_session are NOT generated
// because they typically need custom initialization logic

// You can still add your own custom instructions here
}
```

## Define Your Account Structures

```rust
#[derive(Debug, LightHasher, LightDiscriminator, Default)]
#[account]
pub struct UserRecord {
#[skip] // Skip compression_info from hashing
pub compression_info: CompressionInfo,
#[hash] // Include in hash
pub owner: Pubkey,
#[hash]
pub name: String,
pub score: u64,
}

// Implement the required trait
impl HasCompressionInfo for UserRecord {
fn compression_info(&self) -> &CompressionInfo {
&self.compression_info
}

fn compression_info_mut(&mut self) -> &mut CompressionInfo {
&mut self.compression_info
}
}
```

## Generated Instructions

### 1. Config Management

```typescript
// Create config (only program upgrade authority can call)
await program.methods
.createCompressibleConfig(
100, // compression_delay
rentRecipient,
[addressSpace] // Now accepts an array of address trees (1-4 allowed)
)
.accounts({
payer: wallet.publicKey,
config: configPda,
programData: programDataPda,
authority: upgradeAuthority,
systemProgram: SystemProgram.programId,
})
.signers([upgradeAuthority])
.rpc();

// Update config
await program.methods
.updateCompressibleConfig(
200, // new_compression_delay (optional)
newRentRecipient, // (optional)
[newAddressSpace1, newAddressSpace2], // (optional) - array of 1-4 address trees
newUpdateAuthority // (optional)
)
.accounts({
config: configPda,
authority: configUpdateAuthority,
})
.signers([configUpdateAuthority])
.rpc();
```

### 2. Compress Existing PDA

```typescript
await program.methods
.compressUserRecord(proof, compressedAccountMeta)
.accounts({
user: user.publicKey,
pdaAccount: userRecordPda,
systemProgram: SystemProgram.programId,
config: configPda,
rentRecipient: rentRecipient,
})
.remainingAccounts(lightSystemAccounts)
.signers([user])
.rpc();
```

### 3. Decompress Multiple PDAs

```typescript
const compressedAccounts = [
{
meta: compressedAccountMeta1,
data: { userRecord: userData },
seeds: [Buffer.from("user_record"), user.publicKey.toBuffer()],
},
{
meta: compressedAccountMeta2,
data: { gameSession: gameData },
seeds: [
Buffer.from("game_session"),
sessionId.toArrayLike(Buffer, "le", 8),
],
},
];

await program.methods
.decompressMultiplePdas(
proof,
compressedAccounts,
[userBump, gameBump], // PDA bumps
systemAccountsOffset
)
.accounts({
feePayer: payer.publicKey,
rentPayer: payer.publicKey,
systemProgram: SystemProgram.programId,
})
.remainingAccounts([
...pdaAccounts, // PDAs to decompress into
...lightSystemAccounts, // Light Protocol system accounts
])
.signers([payer])
.rpc();
```

## Address Space Configuration

The config now supports multiple address trees per address space (1-4 allowed):

```typescript
// Single address tree (backward compatible)
const addressSpace = [addressTree1];

// Multiple address trees for better scalability
const addressSpace = [addressTree1, addressTree2, addressTree3];

// When creating config
await program.methods
.createCompressibleConfig(
100,
rentRecipient,
addressSpace // Array of 1-4 unique address tree pubkeys
)
// ... accounts
.rpc();
```

### Address Space Validation Rules

**Create Config:**

- Must contain 1-4 unique address tree pubkeys
- No duplicate pubkeys allowed
- All pubkeys must be valid address trees

**Update Config:**

- Can only **add** new address trees, never remove existing ones
- No duplicate pubkeys allowed in the new configuration
- Must maintain all existing address trees

```typescript
// Valid update: adding new trees
const currentAddressSpace = [tree1, tree2];
const newAddressSpace = [tree1, tree2, tree3]; // ✅ Valid: adds tree3

// Invalid update: removing existing trees
const invalidAddressSpace = [tree2, tree3]; // ❌ Invalid: removes tree1
```

The system validates that compressed accounts use address trees from the configured address space, providing flexibility while maintaining security and preventing accidental removal of active trees.

## What You Need to Implement

Since the macro only generates compression-related instructions, you need to implement:

### 1. Create Instructions

Implement your own create instructions for each account type:

```rust
#[derive(Accounts)]
pub struct CreateUserRecord<'info> {
#[account(mut)]
pub user: Signer<'info>,
#[account(
init,
payer = user,
space = 8 + UserRecord::INIT_SPACE,
seeds = [b"user_record", user.key().as_ref()],
bump,
)]
pub user_record: Account<'info, UserRecord>,
pub system_program: Program<'info, System>,
}

pub fn create_user_record(
ctx: Context<CreateUserRecord>,
name: String,
) -> Result<()> {
let user_record = &mut ctx.accounts.user_record;

// Your custom initialization logic here
user_record.compression_info = CompressionInfo::new()?;
user_record.owner = ctx.accounts.user.key();
user_record.name = name;
user_record.score = 0;

Ok(())
}
```

### 2. Update Instructions

Implement update instructions for your account types with your custom business logic.

## Customization

### Custom Seeds

Use custom seeds in your PDA derivation and pass them in the `seeds` parameter when decompressing:

```rust
seeds = [b"custom_prefix", user.key().as_ref(), &session_id.to_le_bytes()]
```

## Best Practices

1. **Create Config Early**: Create the config immediately after program deployment
2. **Use Config Values**: Always use config values instead of hardcoded constants
3. **Validate Rent Recipient**: The macro automatically validates rent recipient matches config
4. **Handle Compression Timing**: Respect the compression delay from config
5. **Batch Operations**: Use decompress_multiple_pdas for efficiency

## Migration from Manual Implementation

If migrating from a manual implementation:

1. Update your account structs to use `CompressionInfo` instead of separate fields
2. Implement the `HasCompressionInfo` trait
3. Replace your manual instructions with the macro
4. Update client code to use the new instruction names
2 changes: 2 additions & 0 deletions program-tests/anchor-compressible-user-derived/Xargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[target.bpfel-unknown-unknown.dependencies.std]
features = []
Loading