Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Architecture

How zingod turns a Zcash chain into the lightwalletd gRPC service, and why it is built this way. Crate APIs live in each crate’s usage.md: what answers each method in zingod-grpc, storage in zingod-persistence.

1. Who owns what

  • The validator (zebrad, zakura) owns consensus: valid blocks, valid transactions, the mempool.
  • The wallet owns keys.
  • Zingod owns every answer derived from the chain.

Data a validator needs for consensus stays there and is not copied; everything derived is a Zingod index and never falls back to a validator. So:

  • Block, tree-state and transparent-address methods come from an index or not at all. A disabled index answers UNIMPLEMENTED naming it.
  • GetTransaction goes to a validator: raw transaction bytes are consensus data it already stores (why). With no validator answering, the txid-location index names the transaction’s block, fetched from p2p peers and checked against the hash our indexes hold (which does not cover v5 proofs and signatures). GetTaddressTransactions works the same way: the index names the txids, GetTransaction’s path supplies the bytes.
  • Zingod does not validate transactions. That would be a second mempool verifier (proofs, nullifiers, scripts, conflicts) tracking every network upgrade. Validators decide; Zingod checks only what the bytes, the next block and its own UTXO set decide (§5).

The principle behind the rest: verify what Zingod can, trust validators for as little as possible, take everything else from as many peers as possible.

2. The map

  trusted validators (1–N, configured)             peers ([p2p] on; default off)
  JSON-RPC or gRPC, + zebrad push streams          zebra-network peer set
                    │                                          │
                    └──────────────┬───────────────────────────┘
                                   ▼
                    zingod-traffic  TrafficBalancer      every request: lanes, hedge, blame
                                   │
       ┌───────────────────────────┼─────────────────────────────────────┐
       ▼                           ▼                                     ▼
  HeaderSync (chainview)      ObservationMerge + PeerMempool       ChainView::validate_transaction
  → HeaderChain               → mempool view                       → broadcast_transaction
                                                                   (SendTransaction)
       │ watch<HeaderChainSnapshot> │
       ├──────────────────┐         │
       ▼                  ▼         │
  FinalizedFollower        Nfs (RAM)    │
  (zingod-sync)         tip overlay  │
       │ FinalizedStream     │ NfsPublication
       ▼                     ▼      ▼
  index writers   ──▶ zingod-grpc-snapshot Publisher ──▶ one Arc<GrpcSnapshot> per request
  (disk)  durable views                                  → zingod-grpc, /status, /readyz, /metrics

Two paths fold blocks, and their windows overlap:

  • Finalized path (disk): blocks at or below the finalized tip, folded and committed by each index writer. Writers never see a reorg.
  • Tip path (RAM, the NFS): every block above the indexes’ durable tips, finalized blocks not yet durable included, folded in memory and served at once. A block leaves once every index has committed it. Reorgs live here alone.

Both run the same fold per index. While the NFS folds, a block above the lowest durable tip is folded on both paths: in RAM by the NFS, and by its writer once finalized. When the NFS stands down (first sync, an index enabled late: §7) the finalized path folds alone.

3. Sources and what each is trusted for

A trusted validator is one the operator configured ([[trusted_validators]]). A peer is any node on the p2p network: discovered, unauthenticated, free to multiply. Nothing a peer says counts until Zingod has checked it.

NeedTrusted validatorsPeersCheck
Headers✓ (getblockheader runs)✓ (FindHeaders), after the anchortrusted: link + work; peer: every rule
Anchor (first sync)✓ only sourceheaders below our durable tip (restart)each hash = the one our indexes hold
Blocks with a header✓✓ (by hash)hash + merkle root + coinbase height
Blocks below the headers✓ only source (by height)—coinbase height (body trusted)
Mempool✓ (listing + fees)✓ (quorum of listings)txid recomputed; §5
Mined tx by id✓ only source—txid recomputed
Finality✓ (vouch)✓ (> 5 netgroups, no difficulty drop)§4
Submission✓ check (checkadmission)✓ relays (zebra’s, ~⅓ of peers)§6

Why peers never decide the tip. A peer count proves nothing an attacker cannot buy: anyone can run ninety. Proof of work costs real mining however many peers present it. So peers are safe sources of headers and of anything checkable against them, and weak sources of claims. Two claims are taken from peers anyway, so Zingod keeps serving with every trusted validator down: finality and mempool admission. Both count only outbound peers (Zingod chose them) spread over more than 5 netgroups (/16 IPv4, /32 IPv6; one operator ≈ one netgroup).

4. The best chain

The Validated Header tree (zingod-header-chain, memory only) holds every valid branch above the finalized tip; the best tip is the leaf with the most cumulative work. HeaderSync (zingod-chainview) is its only writer and publishes an immutable HeaderChainSnapshot on a watch that the NFS, the finalized path and the GrpcSnapshot publisher read.

  finalized tip                          best tip (most work)
  ───●──●──●──●──●──●──●──●──●──●──●  A  ◀─
                 └──●──●──●            B  kept: may still win; overtaking A = a reorg
  • Anchor. No history from genesis. With no chain, or a trusted validator so far ahead its anchor lies above our fetch ceiling, the tree anchors at that validator’s claim − max(depth, 113) (113 = the difficulty rule’s context). With no trusted validator answering (a restart mid-outage, [p2p] on), the indexes’ own chain anchors instead: the 113 blocks below their durable tip, each peer header matched to the hash the indexes hold. A first sync has no such chain: only a trusted validator anchors it.
  • Trusted headers arrive in runs and enter on their parent link and nBits work alone: the validator already ran the rules.
  • Peer headers ([p2p] on, once anchored) run every consensus rule first (zingod_header_chain::validate, the rules). A header from the future is deferred, not blamed; one failing a rule gets its sender blamed.
  • Bounds. Side branches are capped in tips and nodes (past either, the lowest-work side leaf goes), and only a window of finalized headers is kept (limits). depth = [sync] finalised_depth (default and floor).

Finality

A header proves work, not that its block’s transactions are valid. A finalized block is never reorged (the indexes hold it on disk), so finality is where Zingod stops being able to change its mind.

Every header node records its witnesses: who once had that block on its best chain, and so every ancestor too, for good. Each block carries all of its own accounting; nothing about it is kept anywhere else.

  • Vouched. A trusted validator witnessed it (zebra commits only valid blocks). Vouches come from the last header of each trusted run, each validator’s claimed tip, and its getblockhash answer at our best. Peer headers never vouch.
  • Peer-quorate ([p2p] on). Live outbound peers in > 5 netgroups witnessed it by announcing it or a descendant.
  • finalized = min(highest settled, best − depth), where settled = vouched, or peer-quorate while no unvouched best-chain block has under half the work of the last vouched block. An attacker who eclipses us can feed a private chain, but its difficulty retargets to their hashrate, which shows up as that collapse. Both peer checks fail safe: a thin peer set or a real hashrate drop only delays finality.
  • Neither holding: finality waits, the finality paused alarm rises (§11), serving continues from RAM. A trusted validator’s return vouches and finality resumes.

A reorg, step by step

Peers are first to every reorg: their headers enter on the rules alone, while a trusted validator downloads and fully verifies the new branch before it switches, and Zingod learns of that on its next poll (§9). Meanwhile the served tip drops to the fork point (§7) and synced closes (§8).

 network          ─── N ──┬── N+1              branch A, Zingod + the validator on it
                          └── N+1' ── N+2      branch B, more work, wins

 ──────────────────────────────────────────────────────────────────────────────────▶ time
                   t0     │ L2: validator verifies B │ L3 │   L4: fetch N+1', N+2     │
                    │     │                          │    │                           │
 peers ─────────────┤ announce B headers                                              │
                    │                                                                 │
 best ─── N+1 ──────┤ N+2  (every header rule: PoW, difficulty, MTP, Equihash)        │
                    │                                                                 │
 served ── N+1 ─────┤ N ◀── fork point, vouched: what wallets get ───────────────────▶ N+2
                    │                                                                 │
 validator ── N+1 ──┼────────────────────────────▶ N+2                                │
                    │                              │    │                             │
 vouched ─── N+1 ───┤ N ◀── last shared block ─────┼────▶ N+2  (poll, or push hint)   │
latencywhatbound
L1peers announce, headers validated, best switchesmilliseconds
L2the validator downloads and verifies the branch, switchesits verification time
L3Zingod sees the validator’s new tip (the vouch)one poll; ~0 with the push stream (§9)
L4Zingod fetches the branch’s blocks by hash and folds themblock fetch

Through the window Zingod serves the fork point, a vouched block behind an unvouched best: a currency gap, not a trust gap, and the state rule reads it so (states). The tip unvouched alarm (§11) carries the unvouched best. L2 is the validator’s job and no change on Zingod’s side removes it.

5. The mempool view

Every transaction the view knows is a sighting: its bytes (fetched once, txid recomputed), its fee, a bitset of trusted validators listing it, a quorum flag and an ours flag.

Servable = listed by a trusted validator (not Down or catching up), admitted by a peer quorum, or ours (accepted from our own submission). One trusted listing is enough: each trusted validator is trusted.

Peer quorum ([p2p] on). Each live outbound peer is polled for its whole mempool, and each listing counts while fresh (intervals). A transaction is admitted when:

  • more than 2/3 of the counted listings hold its exact id (txid + auth digest), from > 5 netgroups;
  • its bytes decode, its expiry and (v5+) branch id fit the block above the verified best, it is not coinbase;
  • every transparent prevout resolves in the served UTXO set or a held transaction’s outputs, and the resulting fee is not negative.

It stays while more than 1/3 still list it. No fee floor (zebrad’s and zakura’s differ), no proofs, signatures or scripts: each lister is a full node that ran those. A consensus refusal is remembered (REFUSED_TTL); an unknown prevout is retried next round. With honest peers two conflicting transactions are never both admitted (one node’s mempool holds at most one, and > 2/3 + > 1/3 must overlap); a peer listing both can get both admitted and kept (mempool_model_tests.rs). Cost to an attacker: > 2/3 of our outbound peers across 6 netgroups, for a served transaction that never mines.

Transaction inv announcements are not used: nodes announce each transaction to a third of their peers, once, so silence means nothing.

  • Leaving. A transaction goes when nothing lists it and no quorum keeps it. A new tip clears nothing; an unlisted ours goes at the next move of the best block.
  • Gate. GetMempoolTx / GetMempoolStream answer only while a Live trusted validator holds the verified best, or the polled peers span > 5 netgroups. Otherwise UNAVAILABLE, never an unverified mempool.

6. Submission

SendTransaction asks one trusted validator for a verdict without a broadcast, answers the wallet from it, then relays through the peers and watches the transaction spread. Relaying every wallet transaction through the same validators would tell any observer which transactions came through Zingod, and let one validator censor every wallet behind it. The relay follows Dandelion++’s originator with its fail-safe timer.

precheck (nothing contacted): decodes (-22); expiry ≥ next block, v5+ branch = next block's (-25)
check: checkadmission on a random Live trusted validator holding the verified best
       refused → wallet gets that refusal at once, nothing relayed
       admissible → wallet answered Ok(txid) at once; transaction becomes `ours` (servable)
       unsupported / unanswered → next holder; none → unchecked (below)
attempt: peers — zebra's own relay, PushTransaction to ⌈ready/3⌉ random ready peers
         (no peers: a random untried trusted validator, sendrawtransaction)
unseen past propagation_threshold → next attempt, up to max_attempts
then, still unspread: one trusted sendrawtransaction (the fail-safe)
unchecked: Ok at the first acceptance (trusted Ok | any trusted listing | peer quorum);
           never accepted → that trusted push is the verdict; else UNAVAILABLE
  • checkadmission (Zingod’s zebra fork) runs sendrawtransaction’s checks, the fee policy and mempool conflicts included, without admitting or relaying. It goes to a holder of the verified best only (a validator behind it refuses what spends a fresh output). A validator without it (zcashd, stock zebrad, zakurad) answers -32601: never a failure, the next holder is asked. A refusal’s wire shape: SendTransaction.
  • A peer relay gets no reply and zebra names no recipient. Spread = a trusted validator never pushed to lists it, or a peer quorum admits it (past the ~⅓ one relay reaches). The job ends at spread, when nothing outside its trusted entries can be watched, or when attempts run out.
  • Relays ride the peer set’s existing connections: a zebrad peer drops a second connection from our IP, so a fresh connection per push would never deliver. With [p2p] off every attempt is a trusted sendrawtransaction.
  • [submission] sets propagation_threshold_secs and max_attempts (SubmitPolicy).

7. Indexing

Finalized path (zingod-sync)

FinalizedFollower streams lowest durable tip + 1 ..= finalized tip into one FinalizedStream<Block>:

  • Heights the header chain has a header for: fetched by hash from any member (peers included), checked against the header (hash, merkle root, coinbase height).
  • Below the header chain (bulk sync, a restart from an old index): by height, trusted only, batched, checked by coinbase height only. A trusted validator’s body is trusted.
  • Each block must link to every durable tip it extends and to the block sent before it; otherwise the follower stops and the process exits. Nothing is skipped.
  • Queues are byte-bounded and a send reserves room in every queue first, so the slowest writer paces the fetch and memory stays bounded. A dropped queue or failed commit panics.
  • Utxo-set resolves each transaction’s fee and feeds compact-block through a FeeSink.
  • A writer commits when its buffer fills, after a run ending at the finalized tip, and at shutdown.

Tip path (zingod-nfs)

  • Root = the lowest durable tip across indexes.
  • The NFS folds only while the header just above the root is held and best is within 2 · depth of the root. Otherwise (first sync, an index enabled late) the finalized path does the work and the root is served.
  • Each best-chain block above the root is fetched by hash, checked against its header, and folded over the indexes’ durable views (utxo-set first: its fees feed compact-block).
  • Served tip = the deepest folded best-chain node, else the root. A reorg moves it to the fork point at once, then forward as the new branch folds. Side branches stay until the header chain drops them, so switching back costs nothing.
  • A node goes once every index has committed its height.

8. Serving: one GrpcSnapshot

zingod-grpc-snapshot’s publisher combines the NFS’s latest NfsPublication and the chain view’s latest publish into one Arc<GrpcSnapshot>. Every gRPC request or stream, probe, scrape and state log loads one and reads only it, so no answer mixes two moments.

  • No height named → the served tip, which every enabled index holds. A height named → up to that index’s own durable tip.
  • synced opens when the served tip is the best block and no answering trusted validator claims a higher tip (ChainViewSnapshot::claimed_past_best: one already shown off our chain does not count, one not polled since best moved does), and stays while it is on the best chain within depth of it. At boot, header sync anchors depth below a validator’s claim, so it stays shut until the header chain reaches that claim and the NFS has fetched the window.
  • The daemon’s state (serving, degraded, down, …) is judged from the same load: states.

GetMempoolStream

The protocol says “close the stream when a new block is mined”; Zingod makes that exact. A stream is bound to the served tip’s hash: it opens with the servable mempool, sends each new transaction once (every message height: 0, never one mined in a block the served tip extends), and ends OK once the served tip moves, never on an empty mempool. It ends only after the GrpcSnapshot already serves the new tip, so the GetLatestBlock a wallet sends next sees it. One append-only log per served tip is shared by every subscriber, so per-transaction work never scales with subscribers.

9. Talking to validators and peers

Every request goes through one TrafficBalancer (zingod-traffic); lanes, intervals, hedge delays and bench times are in its guide.

  • Members. Each trusted validator, in priority tiers (0 first); then the whole peer set as one member below every tier (zebra picks the peer). Peers answer headers and blocks only.
  • Failure. A round asks each member once, best tier first; slow answers are hedged to another member. Blocks are retried until answered; everything else ends unanswered after one round.
  • Blame. The balancer cannot judge a value; the caller checks it and reports a bad one, which benches that member. Absent is never blame. The peer set is never benched: zebra scores the lying peer instead.
  • Polling. Each trusted validator is polled on its own permit, so no request delays tip following, and sooner on a zebrad push event. Consecutive failures make it Down, probed until it answers. A failing validator never stops the process.
  • Load on a zebrad = max_connections. There is no request-rate or byte-rate limit.

10. When things fail

DownChain + indexesMempool, SendTransactionFinality
one trusted validator✓✓✓
every trusted, [p2p] on✓ near the tip (peers); bulk sync below the headers stalls✓ quorum; GetTransaction from the peers’ blockon peers’ word, else paused
every trusted, [p2p] offtip stopsUNAVAILABLEpaused
every trusted, restart, [p2p] on✓ anchored on the indexes’ own chain, then as aboveas aboveas above
every trusted, first syncnothing (only a trusted validator anchors)UNAVAILABLE—
every peer✓ (trusted)✓✓

11. Telemetry

Observation only: none of it changes a tip or what is served. State is read from one GrpcSnapshot at scrape (zingod_* and zingod_chainview_* gauges, /status); events are emitted where they happen. Alarms log once WARN when raised, once INFO when cleared (the alarms; stale = STALE_TIP_BLOCKS).

12. Seams

A trait exists only at a seam: a boundary with a second implementation (a real one or a test double), or one that keeps a dependency out of the crates above it.

  • zingod_source::ChainDataSource: JSON-RPC or indexer gRPC; tests drive the real client against a MockValidator.
  • zingod_chainview::PeerSource, zingod_traffic::PeerTransport: zingod’s ZebraPeers, which keeps zebra-network out of both crates.
  • zingod_chainview::UnspentOutputs: the chain view sits below the indexes.
  • zingod_persistence::PersistenceEngine and fs::Fs: indexes see the port, never the engine; SimFs tests crashes.

Not seams: the gRPC service, each index, and the validator set (TrafficBalancer is not a ChainDataSource, so a pool is never mistaken for one validator).