Atlatl is a performance-conscious wrapper for redb with ergonomic access to typed records, multiple serializers, index support, and range queries. Built for precision, clarity, and speed.

Looking For Ecosystem Contributions

• Compression dictionary trainer.
• MySQL Workbench or Microsoft SQL Server Studio type administration software using the debug databases.

Special Thanks

A special thanks to:

1. Christopher Berner and team for the redb crate.
2. David Koloski and team for the rkyv crate.

The Layer Pipeline

1. Serialization

Atlatl hosts a smörgåsbord of serialization options.

Serialization Features

• serialize-bincode-native · Ty Overby, Zoey Riordan and Victor Koenders's bincode crate's native format
• serialize-bincode-serde · Ty Overby, Zoey Riordan and Victor Koenders's bincode crate's serde implementation.
• serialize-bitcode-native · Finn Bear and Cai Bear's bitcode crate's native format.
• serialize-bitcode-serde · Finn Bear and Cai Bear's bitcode crate's serde implementation.
• serialize-borsh · NEAR's borsh crate.
• serialize-musli-descriptive · John-John Tedro's musli crate's descriptive format.
• serialize-musli-storage · John-John Tedro's musli crate's storage format.
• serialize-musli-wire · John-John Tedro's musli crate's wire format.
• serialize-musli-zerocopy · John-John Tedro's musli-zerocopy crate.
• serialize-postcard-serde · James Munns' postcard crate's serde implementation.
• serialize-rkyv · David Koloski's rkyv crate.
• serialize-messagepack · MessagePack serialization using Kornel Lesiński and Evgeny Safronov's rmp-serde crate.
• serialize-zerocopy · Jack Wrenn and Joshua Liebow-Feeser's zerocopy crate.

Zero-copy Deserialization

Zero-copy is supported out of the box. Zero-copy “from disk to wire” is possible using the rkyv, musli-zerocopy, and zerocopy serializers as long as the rest of layer pipeline isn't used. This means that compression, encryption, and error correction must be disabled for your setup to be truly zero-copy.

Warnings

• If you change your Cargo.toml serialization features, you will lose access to any existing databases that used the previous serialization method.

2. Compression

Atlatl has an optional compression system. All data is compressed at a per-value. Compression can be enabled or disabled, or configured, on a per-table basis. Compression & decompression directions can be controlled - this means that data can be compressed on store, and served from the database in compressed form.

Compression Features

• compress-brotli · Brotli compression using Servo, Simon Sapin, and Daniel Horn's brotli crate.
• compress-bzip2 · Bzip2 compression using Alex Crichton, bjorn3, and Folkert de Vries's bzip2 crate.
• compress-deflate · DEFLATE compression using Sebastian Thiel and Josh Triplett's flate2 crate's deflate implementation.
• compress-gzip · Gzip compression using Sebastian Thiel and Josh Triplett's flate2 crate's gzip implementation.
• compress-lz4 · LZ4 compression using Pascal Seitz's lz4_flex crate.
• compress-zlib · Zlib compression using Sebastian Thiel and Josh Triplett's flate2 crate's zlib implementation.
• compress-zstd · Zstandard compression using Alexandre Bury's zstd crate.

Dictionaries

When the compress-dictionaries feature is enabled, the lz4, zlib, zstd compression algorithms will support dictionary-based compression. Dictionaries can significantly improve compression ratios and speed. This feature is particularly useful when data and data structures are similar and repetitive.

Warnings

Treat your compression dictionaries like encryption keys:

• If a compression dictionary becomes lost or corrupted, all data will be permanently lost.
• Dictionaries may contain sensitive information or sufficient information for decryption your data to be possible.
• If you change your Cargo.toml compression features, you will lose access to any existing databases that used the previous compression method.

3. Encryption

Atlatl hosts a zero-trust capable encryption system.

Zero Trust Capable

All data is encrypted at a per-value level. This allows for very high-grain access. Keys can be provided on a per-value, per-table, or per-database level.

Key Rings

Keys are provided using key-rings which can be chained together and used to decrypt data and can be used to define database security policy.

Cipher Features

• encrypt-aes-gcm · AES-GCM encryption using Tony Arcieri's aes-gcm crate.
• encrypt-chacha20 · ChaCha20-Poly1305 encryption using Artyom Pavlov's chacha20poly1305 crate.

Key Derivation Function Features

• kdf-blake3 · BLAKE3 Key Derivation Function using Jack O'Connor's blake3 crate.
• kdf-sha256 · SHA-256 Key Derivation Function (KDF) using Brian Smith's ring crate.

Warnings

• If your keys become lost or corrupted, all data will be permanently lost.
• If your keys are leaked, or become public knowledge, your data may be compromised.
• If you change your Cargo.toml encryption features, you will lose access to any existing databases that used the previous encryption method.

4. Error Correction

Atlatl includes optional per-value error correction using Reed-Solomon codes. This allows you to safeguard individual records (such as documents, media assets, or user data) against silent corruption and disk-level bit rot.

Features

• ecc-reed-solomon · Reed-Solomon encoding using Darren Li, Michael Vines, and Nazar Mokrynskyi's reed-solomon-erasure crate.

Benefits

• Granular protection: ECC is applied at the value level, not at the file, table, or disk level.
• Seamless recovery: Errors are automatically corrected during reads with zero downtime. Your application never knows corruption occurred.
• Configurable: Enable it globally, per table, or disable it entirely. Tune it to fit your performance and integrity needs.
• Transparent: Works on any filesystem or storage backend. No dependency on ZFS, RAID, or special hardware.

What It Protects Against

• Bit rot: Silent corruption from aging storage media.
• Partial disk failures: Readable sectors with corrupted data.
• Hardware glitches: Transient errors in disk controllers, cables, or RAM.
• Cosmic rays: Random bit flips in memory or storage.

Limitations

• Metadata vulnerability: Only the value payload is protected by ECC. If redb's structural metadata, indexes, or the ECC metadata itself gets corrupted, Atlatl cannot restore those parts.
• Capacity limits: Reed-Solomon can only correct a limited number of errors per record. Massive corruption beyond the correction threshold will still result in data loss.
• Performance cost: ECC adds computational overhead and storage overhead from parity data.
• Less efficient on small values: For small records, ECC introduces measurable space and compute overhead with less protection benefit. It performs best with larger blobs (PDFs, images, documents).
• Not a silver bullet: ECC complements but does not replace traditional data protection strategies like offsite backups, snapshots, RAID, or ZFS.

When To Use

Enable ECC when:

• Data integrity is more critical than raw performance or disk efficiency.
• You're storing valuable data or assets (documents, media, configurations).
• You want protection against silent corruption.

Bottom line: ECC won't make your data immortal, but it can save the day in many real-world corruption scenarios that would otherwise require restoring from backups.

What's an Atlatl?

The atlatl is an ancient spear-thrower used by humans for tens of thousands of years before the bow.

It didn't launch projectiles itself; it amplified the human hand, unlocking reach, speed, and power far beyond what was possible alone.

It was a force multiplier: deceptively simple, brutally effective, and optimized through ages of refinement.

Why the Name?

Just like its namesake, the Atlatl crate doesn't store data by itself.

• It sits between you and redb, acting as a powerful extension of your intent.
• It transforms your inserts and reads with optional serialization, compression, encryption, and error correction.
• It gives you layered, per-value control over your data without cluttering your business logic.

It operates with precision, performance, and zero-overhead in the compiled binary.

Atlatl is the lever that makes your database do more faster, safer, and smarter.

Credits

Developed by Dylan Bowker and the Atlatl Team, with strategic insight from Ariadne.