Scaling Voyager to 21M TPS

A technical deep dive on Voyager, Celestia’s hyperscale blockchain architecture

Scaling Voyager to 21M TPS

As we recently announced, Voyager sustains more than 21 million account-to-account transfers per second in our benchmarks. That number is real onchain throughput where payments are sent at random across a set of 100 million accounts.

That’s a big claim and many folks have rightfully asked us how such astronomical throughput is possible. This post unpacks the design philosophy and architecture of Voyager that makes it so scalable.

Design Philosophy and Core Components

The key design principle behind Voyager is that dependencies between components are declared explicitly and minimized as much as possible. Components never call each other or share a mutex. They coordinate only through the shared ring buffer. As a result, there’s almost no coordination overhead. The throughput of the whole pipeline is simply the throughput of the slowest component.

Because components are decoupled, each one can be optimized for the kind of work it does.

  • I/O-heavy components batch their work, spreading the cost of each system call or network write across many slots.
  • CPU-bound components use data structures with a carefully designed memory layout to get the most out of a single core and its cache.

Voyager’s core components are:

  • a ring buffer adapted from the LMAX Disruptor
  • a lightning fast state transition function (STF)
  • batched disk and network I/O
  • backpressure that keeps memory bounded when a stage falls behind

Of these components, Voyager’s STF is especially unique. While many other blockchain scaling designs parallelize execution, Voyager runs the STF on a single thread. By caching the relevant data and leveraging out-of-order execution, this single thread can handle significant throughput.

Let’s dive in to see how these components are architected in more detail.

From a User’s Wallet to a Voyager Node

A payment starts in the user’s wallet, which signs the transaction and sends it to an ingress service. Ingress verifies the transaction signatures, groups valid transactions into batches, and forwards them to Voyager. Ingress nodes are stateless, so we can add more as traffic grows. This lets verification capacity scale independently of transaction execution.

Each request is verified by one ingress node. Ingress acknowledgment confirms queuing, whilst verification and forwarding happen asynchronously.

The Challenges of I/O Batching and Backpressure

Inside Voyager, transactions are fed into the mempool for deduplication. From there, they enter the “disruptor” - a ring buffer which coordinates execution, storage, and publication.

The producer writes batches from the mempool into the ring. The journal, STF, DB writer, DA publisher and pruner each read straight from the ring, without waiting on each other.

In the disruptor, each slot starts out empty and is processed in stages. First a “producer” task selects the next batch of transactions from the mempool and writes it into the next available slot. Then, the journal and STF get to work; the journal durably records the transactions to enable crash recovery, and the STF executes them.

During execution, the disruptor slot is augmented with transaction results and lists of state changes. After execution finishes, the database writer jumps in to save the state changes to a remote database. At the same time, a publisher writes the transaction data to Celestia Fibre. Finally, a “pruner” task deletes transactions from the local DB after successful publication.

Each one of these tasks processes data sequentially, but the entire process is pipelined. So, for example, the journal might be recording batch 103 while the STF executes batch 102 and the database writer and publisher are each handling batch 101.

Batching and Backpressure

Voyager batches work at two levels. Each ring slot contains multiple transactions, and consumers can group up to 128 slots before flushing their work. This spreads the cost of journal synchronization (fsync), database submissions, and DA publication across many transactions. Consumers process smaller groups when less work is available.

The ring holds 4,096 slots by default, giving the pipeline room to absorb temporary slowdowns. If storage falls behind execution, pending batches occupy more slots. Once no reusable slots remain, upstream processing waits until consumers make progress. That is backpressure: slower stages limit incoming work before memory grows without bound.

Ring capacity and group size are configurable. A larger ring can absorb a longer slowdown, at the cost of more memory.

The Work Behind Each Transaction

The disruptor keeps the stages coordinated with minimal copying or synchronization overhead - but it still runs at the speed of the slowest stage. To sustain 21 million TPS, each stage has to be carefully designed for maximum performance.

Journal: preserve the inputs for recovery

The journal records transaction batches so that Voyager can reconstruct work that has not reached the database after a crash.

Writing every transaction requires substantial disk bandwidth, and waiting for durable writes adds overhead. We keep the write path simple: append records sequentially, then synchronize a group with fsync. The journal consumer advances only after synchronization succeeds, spreading that cost across many transactions.

STF: turn transactions into state changes

The state transition function checks balances, applies transfers and fees, and produces state changes. Ensuring good performance in the face of unpredictable transaction workloads is the core challenge of blockchain performance engineering.

For the last few years, the default method for speeding up blockchains has been to introduce more parallelism. With Voyager, we do the opposite. Voyager’s execution is completely single-threaded… and that’s a big part of what makes it fast.

The core observation behind Voyager’s unique STF is that CPU isn’t exactly the bottleneck in the STF. The real bottleneck is memory access. And when you’re loading data from memory, adding more cores can actually hurt rather than helping. When two different threads request access to the same memory at the same time, they can both experience pipeline stalls.

But, thanks to out-of-order execution, a single CPU can dispatch many memory accesses concurrently, all while processing the data that’s already in cache. To maximize CPU efficiency, what counts is making sure that the data you need is always in cache. That requires:

  • Keeping accesses predictable: All data dependencies are known well in advance of execution.
  • Minimizing contention: All state data is owned by the single STF thread. That completely eliminates locking overhead and cache coherency traffic.
  • Minimizing your working set: Voyager’s data structures are designed to fit neatly into cache lines, maximizing the amount of useful data we can keep near the CPU.
  • Asking for data long before you need it: Voyager prefetches data as far as 32 transactions ahead of what the CPU is currently executing.

Under sustained load, Voyager’s STF runs at 96–100% utilization of a single CPU core. Reducing unnecessary lookups and memory stalls helps us get more useful work out of that core.

DA publisher: let others verify the work

The DA publisher makes transaction inputs available so other participants can independently replay and verify execution. It uploads data to Celestia Fibre, which distributes erasure-coded blob shards across validators.

Publishing groups of batches spreads network-request overhead across many transactions. Publication waits for journaling, then proceeds alongside execution and database writes.

Database writer: persist the resulting balances

The database writer sends the STF’s state changes to Stargazer (our custom state storage engine) for persistence. Those stored balances support queries and provide a durable starting point for the planned recovery path.

Like the DA publisher, the database writer groups submissions to reduce network overhead and waits for both journaling and execution to finish before sending a batch’s changes.

Journal pruner: reclaim space safely

The journal pruner keeps recovery records from filling the disk. Removing them too early would discard inputs that downstream stages may still need.

Before allowing records to be removed, it checks downstream consumer progress, DA confirmation, and the storage persistence watermark. Those checks determine how far pruning can safely advance.

Stargazer Distributes Storage Work

Stargazer is a custom state storage engine we built that runs on a separate machine and receives state-diff batches over the network. Like Voyager, Stargazer is structured around a central disruptor for maximum performance. where the journal stage is followed by parallel database shard writers.

Each shard writer reads the ring directly and waits for the journal. The pruner waits for every writer. Three shards are illustrated, but Stargazer supports an arbitrary number.

We use sharding to distribute state across multiple RocksDB databases, each on a separate disk volume. Every shard writer reads the shared batches but applies only the changes assigned to its shard.

Stargazer reports a committed height only once every shard has reached it. Its pruner waits for all shard writers before advancing. If storage falls behind, the bounded ring and ingress queue propagate backpressure to Voyager.

Benchmarks

We benchmarked Voyager on a state of about 100 million accounts, running 400 billion transfers across several runs. The charts below come from our Grafana dashboards for one of those runs.

These benchmarks measure the Voyager node itself: execution, journaling, data publication, and storage. The load generator sends transactions straight to the node, bypassing ingress, so signature verification isn’t part of these numbers. Because ingress scales horizontally, verification capacity can be added independently. The transactions still carry full-size signatures, so the journal and DA publisher handle realistic data volumes.

Throughput held steady at 21 million transactions per second for the whole run.
The node’s memory stayed flat at about 48 GiB from start to finish. That’s backpressure at work: the ring and the journal cap how much work can be in flight, so memory doesn’t creep up over time.
The journal writes about 4.8 GB/s to disk, and the DA publisher handles the same volume. Both carry the full transactions, about 230 bytes each including the signature.

What Comes Next

Voyager is still a work in progress. We’re encouraged by the hot-path throughput, but several important pieces remain unfinished.

First, we do not Merkleize the state in the current benchmark. Much like Solana, we plan to compute state roots slightly less frequently than every block. With that design, we can ensure that state root computation never blocks transaction processing - but we’ll incur some extra latency for light clients.

Second, we currently preserve the data needed for replay and replication, but the logic to recover after a restart and replicate to other nodes remains unimplemented. Building and testing those paths is a priority.

Finally, we benchmarked on a working set which was still small enough to fit in RAM on a single machine. We have designed the logic required to scale far beyond the limits of RAM and are confident that this will not become a significant bottleneck - but the implementation is not yet finished.

Because all state lives on one executor, the engine underneath Voyager isn’t limited to payments. Next we’re making execution pluggable, so the same pipeline can run exchanges (the workload the LMAX Disruptor was originally built for) and many other applications at scale. Voyager is built for payments, but the designs that follow will be able to host all kinds of markets.

Beyond 21 Million TPS

Storage throughput can grow by sharding journal and database writes across more disks. Fibre’s benchmark of roughly 3 Tb/s also suggests substantial headroom for data publication.

There is room to explore further improvements in the STF on a single CPU core. High utilization still leaves opportunities to reduce memory stalls, avoid unnecessary lookups, and do less work per transaction. Finally, we have designs for both sharding state across several cores without contention, and for sharding work seamlessly across multiple Voyager instances.

Stay tuned as we continue to bring Voyager to production and demonstrate its capabilities.