Cassandra

Apache Cassandra 6.0 Part 5: Zstd Dictionary Compression

Hayato Shimizu · July 7, 2026 · Updated September 7, 2026 · 7 min read

Zstd dictionary compression in Cassandra 6.0

Zstd dictionary compression is one of the Cassandra 6.0 changes I would test with real production-like data before making a storage decision. Cassandra tables often contain repeated field names, identifiers, JSON-like structures, and recurring value patterns, which can make a trained dictionary useful, but the result depends entirely on the data in the SSTables.

Cassandra 6.0 is pre-release software. This post reflects 6.0-alpha3 and must be checked again against the release notes and upgrade documentation for the exact version being deployed.

Benefits and caveats

Potential benefit	Caveat to test before adoption
More compact SSTables for tables with recurring byte patterns.	High-entropy values, already compressed payloads, encrypted fields, and highly variable records may see little improvement.
Less disk used by SSTables, snapshots, backups, streaming, and compaction output when the ratio improves.	The gain must be measured against the CPU required to train and use the dictionary.
Fewer bytes read from storage can help an I/O-bound workload.	Dictionary-aware decompression has its own CPU cost, so read latency needs measurement rather than assumption.
A dictionary is stored with the SSTable compression metadata, so older files remain readable after the active dictionary changes.	Multiple dictionary versions consume cache and native Zstd memory until the SSTables that use them have been rewritten or removed.

The feature is useful where a table has stable, repeated structure and the storage reduction exceeds the additional CPU and memory cost. It is not a cluster-wide setting to enable blindly. Test training, compression ratio, flush and compaction time, read latency, dictionary cache memory, backup, restore, and streaming for every table that uses it.

The work behind this feature is broader than adding another compressor. CASSANDRA-17021 introduced Zstd dictionary compression in the 6.0 line, alongside dictionary training controls, import, export and list commands, dictionary metadata, memory reporting, lifecycle handling, and guardrails around training configuration.

The feature only becomes practical when the dictionary can be inspected, backed up, restored, and understood as part of ordinary SSTable operations.

The primary ticket lists Yifan Cai as author, with Jon Haddad and Stefan Miklosovic as reviewers. The work has continued through the associated CEP and follow-up tickets covering training, lifecycle handling, observability, and tests.

Dictionary compression mechanics

A compression dictionary gives Zstd a prepared set of common sequences to use when compressing data. It can improve the ratio for data with repeated shapes, although it can also add CPU cost and memory use with little benefit when the data is already highly variable or compressed upstream.

For a Cassandra operator, the effect is not limited to disk capacity. A different compression ratio can affect compaction, backup volumes, restore duration, streaming, disk headroom, and capacity planning. A different decompression cost can affect the read path. Those trade-offs need measurement on representative tables rather than a blanket cluster setting.

How Zstd dictionary compression works in an SSTable

An SSTable compressor works on chunks of the Data.db component. With ordinary Zstd, every compressed chunk has to describe recurring byte patterns using only the data within that chunk. That is a poor starting point when chunks are small and the same field names, JSON keys, identifiers, headers, or structured payload fragments occur across many rows.

A trained dictionary gives Zstd a shared history before it starts compressing a chunk. It contains byte sequences learned from representative samples and allows Zstd to encode matching content with shorter references. The dictionary also provides the information from which the native Zstd compression and decompression tables are built.

This is not a change to CQL semantics, the memtable format, commit log compression, or internode compression. It is an SSTable compression feature used by ZstdDictionaryCompressor. A table without an available dictionary falls back to ordinary Zstd, so enabling the compressor does not require a dictionary to exist before the first flush.

Concrete example of Zstd dictionary compression in Cassandra. Repeated JSON event data from existing SSTables is trained into common byte sequences, then a new event reuses those sequences during compression and uses the same dictionary during decompression.

The D1, D2, and D3 labels in the diagram are explanatory shorthand. Zstd does not write those labels into Data.db; it emits an entropy-coded compressed block that uses the trained dictionary as match history.

SSTable compression step	Cassandra 6.0 implementation
Dictionary selection	The table’s dictionary cache tracks the newest dictionary ID as the active dictionary for writes. A new dictionary does not rewrite old SSTables by itself; those files keep the dictionary with which they were written until `nodetool recompresssstables` or another maintenance operation rewrites them.
Chunk compression	`ZstdDictionaryCompressor` passes each direct byte-buffer chunk to Zstd’s dictionary-aware compression API. Cassandra creates and retains a native `ZstdDictCompress` object for each compression level used with that dictionary.
SSTable metadata	The dictionary ID, dictionary bytes, and a checksum are embedded in the SSTable `CompressionInfo` component. The SSTable therefore carries the material needed to read its compressed chunks.
Chunk decompression	Cassandra reads the dictionary identity from `CompressionInfo`, resolves it from the local cache or the SSTable metadata, and lazily creates one native `ZstdDictDecompress` object for that dictionary. Reads then use the matching dictionary rather than assuming the newest dictionary is correct.
Cache lifetime	The per-table dictionary cache is bounded and expires entries after a configured period of inactivity. Dictionary and compressor references are counted so eviction cannot free native Zstd state while a compressor is still using it.

The SSTable-attached design is important during backup, restore, streaming, repair, and node replacement. A restored SSTable can be decompressed with the dictionary it was written with, even if the active dictionary for the table has since changed.

Training and publishing a dictionary

Cassandra trains dictionaries from existing canonical SSTables rather than from a random application-side export. When no SSTables exist, the training workflow forces a memtable flush first so there is an SSTable to sample. The trainer feeds the selected sample bytes into Zstd’s dictionary trainer and requires at least 11 samples before it will produce a dictionary.

The CQL compression parameters control the maximum dictionary size, the total training sample budget, and the minimum interval before the table can be trained or imported again. Cassandra 6.0 defaults to a maximum dictionary size of 64KiB, a maximum sample size of 10MiB, and training_min_frequency of 0, which allows retraining without a minimum interval. The nodetool command describes a sample budget around 100 times the target dictionary size as the recommended starting point.

ALTER TABLE commerce.events
  WITH compression = {
    'class': 'ZstdDictionaryCompressor',
    'chunk_length_in_kb': '64',
    'compression_level': '3',
    'training_max_dictionary_size': '64KiB',
    'training_max_total_sample_size': '10MiB',
    'training_min_frequency': '24h'
  };

The values above are an example configuration, not a universal recommendation. In particular, 24h is a deliberate replacement for the default unlimited retraining. The sample budget must represent the rows that will be written into future SSTables, while the minimum frequency should prevent a table from accumulating a new dictionary for every small payload change.

Training can then be triggered for a specific table with this command.

nodetool compressiondictionary train commerce events \
  --max-dict-size 64KiB \
  --max-total-sample-size 10MiB

After training, Cassandra obtains Zstd’s dictionary ID from the generated bytes and combines it with a timestamp-based version to create a monotonically increasing Cassandra dictionary ID. It calculates a checksum over the dictionary kind, ID, and raw bytes. The dictionary is persisted in system_distributed.compression_dictionaries, added to the local cache, and only then announced to other nodes. The ordering avoids another node being told about a dictionary that has not yet been stored.

Dictionary versions and memory

Dictionary compression introduces versioning at the SSTable level. A table can have more than one dictionary because older SSTables may still reference a previous version while new flushes and rewrites use the active one. Cassandra needs to retain the old version until no live SSTable needs it, which is why frequent retraining is not free.

The raw dictionary bytes are only part of the memory cost. Cassandra derives one decompression object on first use and can derive compression objects for each compression level in use. nodetool tablestats reports cached dictionary memory, which should be reviewed alongside table count, compaction activity, and the number of dictionary versions present.

The operational aim is not to create the largest possible dictionary or to retrain continuously. It is to keep a small number of dictionaries that reflect stable data shapes, while retaining enough history for existing SSTables to remain readable through the normal lifecycle.

Dictionary lifecycle

Stage	Cassandra 6.0 behaviour	Operator checks
Training	A dictionary is trained from representative table or SSTable samples through Cassandra’s dictionary workflow, subject to configured size and retraining controls.	Use data that reflects the current application payload and retain the training configuration.
Metadata	Cassandra records dictionary information in `system_distributed.compression_dictionaries`, including the table relationship and creation time.	Confirm the expected dictionary is present and that old or orphaned entries are understood.
SSTable writes	New SSTables written with a compatible dictionary-capable compressor can use the active dictionary.	Compare SSTable size, flush time, CPU use, and compaction behaviour with the existing compressor.
Reads and compaction	Cassandra needs the relevant dictionary to decompress the SSTable data during normal reads and storage maintenance.	Measure read latency, compaction throughput, memory use, and behaviour during streaming or replacement.
Import, export, and retirement	`nodetool compressiondictionary` supports managed dictionary workflows, including `cleanup [--dry]` for orphaned dictionaries.	Rehearse export, import, cleanup, restore, and rollback before treating the feature as a production default.

The 6.0 tooling gives administrators more commands to work with.

nodetool compressiondictionary subcommands for dictionary management;
metadata in system_distributed.compression_dictionaries;
dictionary memory visibility in nodetool tablestats;
CQL controls for training parameters;
import and export support for controlled dictionary workflows;
nodetool compressiondictionary cleanup [--dry] for orphaned dictionaries;
nodetool recompresssstables to rewrite existing SSTables with the active dictionary;
safeguards around dictionary size, samples, and retraining intervals.

I would treat a dictionary as a table-specific storage decision, with an owner, a rollback path, and enough dashboard coverage to see its effect.

Testing a candidate table

Start with a table whose data shape is stable enough to train against and large enough that a compression gain would justify the work. Compare the candidate configuration against the current compressor using production-like partitions, TTL patterns, tombstones, and write rates.

The useful measurements are compression ratio, SSTable size, compaction throughput, disk utilisation, read latency, CPU time during reads and compaction, dictionary memory, flush behaviour, and restore or bulk-load time. Re-run the test when the application changes the payload shape. A dictionary trained on yesterday’s event format can be the wrong choice after an application rollout.

The safety checks are equally practical. Verify that imported dictionaries can be restored in a fresh environment, that every relevant tool understands the files, and that a node replacement or streaming operation behaves as expected. Cassandra storage improvements are only useful when the normal failure and recovery paths remain straightforward.

Where AxonOps fits

Compression decisions need more than an aggregate disk-used graph. AxonOps can bring disk growth, compaction backlog, I/O saturation, table size, read latency, and configuration history into the same operational view. That makes it possible to compare a dictionary rollout with the storage and latency signals it changes, without treating the test as an isolated benchmark.

Demo Sandbox

Thank You from AxonOps: Current London 2026 Recap