Cassandra

Apache Cassandra 6.0 Part 2: Accord Transactions

Hayato Shimizu · · · 6 min read

Accord in Cassandra 6.0

Accord is the largest architectural change in the Cassandra 6.0 line. The changelog lists General Purpose Transactions under CEP-15, while related work includes support for BEGIN TRANSACTION mutations that touch multiple partitions.

Cassandra 6.0 is pre-release software. This post reflects 6.0-alpha3 and must be checked again against the release notes and upgrade documentation for the exact version being deployed.

Accord is beta-designated and disabled by default through accord.enabled: false. It should be enabled only in an environment where the transaction workload, driver behaviour, failure handling, and migration procedure have been tested.

I am grateful to the Cassandra committers and contributors working on this area because adding transaction coordination to a distributed database is a substantial engineering effort that extends well beyond new syntax. The system also has to behave correctly through restarts, topology changes, recovery, and latency pressure.

I have been using Cassandra since 2008, so I am cautious about transaction features. Cassandra has always rewarded good data modelling and exposed the cost of designs that assume coordination is free.

Cassandra application design has usually avoided cross-partition transactional assumptions. Lightweight transactions through Paxos helped with conditional updates, but they did not make Cassandra a general multi-partition transaction system. The accounting-transfer example in Phil Eaton’s Cassandra 6 exploration makes the distinction clear. Moving value between two accounts in different partitions needs one decision that covers both rows, not two independent conditional writes.

How Accord runs a transaction

CEP-15 describes Accord as a leaderless timestamp protocol for transactions that span any set of keys. It is designed to provide strict-serializable isolation without introducing a permanent cluster-wide leader. Cassandra supplies the CQL, schema, messaging, and storage integrations; Accord supplies the agreement protocol.

The transaction’s reads, conditions, and writes are declared before it runs. Cassandra uses that information to identify the partitions and ranges involved, then gives the transaction an ID, t0, which is also its initial timestamp. Accord can then reason about conflicts before the transaction is executed.

StageWhat happens
PreAcceptThe coordinator sends the proposed transaction and its initial timestamp to replicas for every participating shard. Each replica records that it has seen the transaction, returns that timestamp when it can accept it, and includes lower-timestamp conflicting transactions as dependencies.
Fast pathWhen a fast-path quorum in every participating shard returns the initial timestamp, the coordinator commits the transaction with the combined dependency set. This is the normal one-round-trip agreement path described by CEP-15.
Slow pathA replica that has seen a newer conflicting timestamp responds with a higher timestamp. If the coordinator cannot take the fast path, it selects the highest timestamp it received and sends Accept to a simple quorum in each shard so that choice is durable before Commit.
ExecutionAfter Commit, the coordinator reads from the participating shards with the relevant dependencies attached. Replicas wait for earlier dependencies to commit and apply, return their read results, and then receive Apply for the final transaction result.
RecoveryA replica that has witnessed an incomplete transaction can coordinate recovery. It asks replicas for their local state, work it must wait for, and superseding transactions, gathers a simple quorum from every shard, then resumes the highest durable stage or selects the appropriate slow-path timestamp.

Only the replicas responsible for the declared data participate; the coordinator is a Cassandra node, not a new global transaction leader. Each participant keeps local transaction state, and a replica that witnessed the transaction can coordinate recovery when the original coordinator stops making progress.

Fast path

Mermaid sequence diagram for Accord's fast path, showing Node 2, Node 4, and Node 6 receiving PreAccept, confirming the initial timestamp, then reading and applying the transaction result.

The fast path does not depend on every replica replying. CEP-15 introduces fast-path electorates so that a minority of unavailable replicas does not automatically force a slower agreement path.

Slow path

Mermaid sequence diagram for Accord's slow path, showing Node 6 reporting a higher timestamp, a simple quorum accepting the selected timestamp, then Commit, reads, and Apply.

When the fast-path responses cannot support the initial timestamp, the coordinator selects the highest timestamp it received and records that decision through Accept with a simple quorum. It then proceeds through Commit, execution reads, and Apply.

Recovery after a replica failure

Mermaid sequence diagram showing Accord recovery after the original coordinator stops before a final decision and Node 4 is unavailable. Node 2, which witnessed the transaction, gathers state from Node 6 and resumes the strongest durable stage or proceeds through the slow path.

A replica becoming unavailable does not by itself trigger recovery; Accord can still use a fast electorate or the slow path when the required quorum is available. The sequence above shows recovery for an incomplete transaction after the original coordinator stops before a decision. A participant that witnessed the transaction asks replicas for the state they have recorded, including whether they saw a PreAccept, Accept, Commit, or Apply, as well as work that must be waited on. When it finds an accepted, committed, or applied transaction, it continues from that strongest recorded stage. If no stage has been decided, it uses the reports to determine whether the initial timestamp remains safe or whether it must take the slow path with a higher timestamp. Where earlier transactions must finish first, recovery waits and retries.

Preparing tables for Accord

Accord must be enabled in cassandra.yaml, and the participating tables need a transactional mode. For a new table that is intended to run fully through Accord, the CQL setting is transactional_mode = 'full'.

CREATE TABLE commerce.orders (
  order_id uuid PRIMARY KEY,
  status text,
  sku text
) WITH transactional_mode = 'full';

CREATE TABLE commerce.allocations (
  sku text,
  order_id uuid,
  quantity int,
  PRIMARY KEY (sku, order_id)
) WITH transactional_mode = 'full';

The branch also documents mixed_reads, where writes and serial operations use Accord while ordinary non-serial reads remain on Cassandra’s existing eventually consistent read path. Moving an existing table to a transactional mode has its own range-migration and repair procedure, so it should be treated as a planned migration rather than a schema toggle during an application release. Completing that migration requires a full repair and a Paxos repair. During phase two, the first access to each key also pays for a Paxos repair round trip.

Accord does not support counter tables. Transactional reads accept ONE, QUORUM, SERIAL, and ALL; transactional writes accept ANY, ONE, QUORUM, SERIAL, and ALL. LOCAL_QUORUM, TWO, and THREE are rejected. Once a table is fully migrated to transactional_mode = 'full', Cassandra ignores a supplied consistency level and commits through the Accord path with ANY semantics.

A cross-partition transaction

The example below uses a transaction-level condition. It checks that an order is still pending and that an allocation row does not already exist, then changes the order state and creates the allocation together. LET names the reads used by the transaction, while IF ... THEN controls the mutations for the whole transaction rather than adding separate IF clauses to each write.

BEGIN TRANSACTION
  LET requested_order = (
    SELECT status FROM commerce.orders
    WHERE order_id = 7a3e2a9e-4d51-4c72-a0ee-000000000001
  );
  LET existing_allocation = (
    SELECT quantity FROM commerce.allocations
    WHERE sku = 'widget-42'
      AND order_id = 7a3e2a9e-4d51-4c72-a0ee-000000000001
  );
  SELECT requested_order.status;
  IF requested_order.status = 'PENDING' AND existing_allocation IS NULL THEN
    UPDATE commerce.orders
      SET status = 'ALLOCATED'
      WHERE order_id = 7a3e2a9e-4d51-4c72-a0ee-000000000001;
    INSERT INTO commerce.allocations (sku, order_id, quantity)
      VALUES ('widget-42', 7a3e2a9e-4d51-4c72-a0ee-000000000001, 3);
  END IF
COMMIT TRANSACTION;

This is the type of workflow Accord makes possible without moving the coordination into an application service. It is not an inventory-reservation design by itself. A real system still needs a model for available stock, idempotency, expiry, cancellation, and the operational handling of failed or retried requests. The example only shows the transaction boundary and CQL shape.

The condition belongs to the transaction block. Cassandra’s transaction tests reject ordinary per-statement IF conditions and custom USING TIMESTAMP clauses inside that block, because Accord coordinates the condition and the transactional timestamp for the complete operation.

Strict serializable isolation applies to each transaction’s declared work. Paged reads and partition-range reads have a narrower scope because Cassandra evaluates one transaction per page or sub-range. They need their own application-level correctness tests rather than being treated as equivalent to a single declared transaction.

If an application needed correctness across multiple partitions, the usual options were the following.

  • remodel the data so the operation fit a single partition;
  • coordinate in the application;
  • use another system for that workflow.

Those options remain valid and are often the right answer. Accord adds another option for the smaller set of cases where the coordination cost is justified.

Choosing transactional workloads

Use transactions selectively rather than redesigning normal Cassandra write paths merely because the feature exists.

Because Cassandra remains a distributed system, a transaction feature does not remove the cost of coordination or the need to understand partitions, replicas, consistency levels, failure handling, and latency behaviour.

It gives teams a way to evaluate selected workflows without first moving the coordination logic outside Cassandra.

Examples include the following.

  • updating related records that must move together;
  • applying conditional state transitions across more than one partition;
  • reducing application-side reconciliation for specific workflows;
  • making some consistency-sensitive operations easier to reason about.

That only becomes useful when the workload that will use it has been tested properly.

Production considerations

The Cassandra 6.0 changelog includes Accord hardening and performance work. It covers tail latency improvements, clean shutdown and restart behaviour, batch atomicity fixes, topology serializer improvements, and transaction timestamp handling.

The hardening work in the branch is as significant as the CQL capability itself, because production transaction support depends on solid behaviour through restarts, topology changes, recovery, and tail latency.

If I were testing Accord-backed transactions, I would treat them as a distinct workload and measure the following.

  • p95 and p99 transaction latency;
  • timeout rates;
  • invalid request rates;
  • retry behavior;
  • contention patterns;
  • coordinator and replica latency separately;
  • behavior during restart, node replacement, and topology changes.

I would give transactional writes their own view rather than hiding them inside a generic write-latency dashboard and assuming that was enough visibility.

Where AxonOps fits

Accord transactions add another operational surface that deserves dedicated monitoring, alerts, and a runbook. The dashboard needs to show whether the transaction workload is healthy rather than only whether the cluster is generally healthy.

For Cassandra 6.0 testing, I would track transaction latency alongside normal Cassandra signals. These include coordinator latency, replica latency, timeouts, unavailable exceptions, heap pressure, disk pressure, and any workload-specific contention.

Series

This post is part of the Cassandra 6.0 series.

Sources