Architecture
How zingod turns a Zcash chain into the lightwalletd gRPC service, and why it is built this way.
Crate APIs live in each crate’s usage.md: what answers each method in
zingod-grpc,
storage in
zingod-persistence.
1. Who owns what
- The validator (zebrad, zakura) owns consensus: valid blocks, valid transactions, the mempool.
- The wallet owns keys.
- Zingod owns every answer derived from the chain.
Data a validator needs for consensus stays there and is not copied; everything derived is a Zingod index and never falls back to a validator. So:
- Block, tree-state and transparent-address methods come from an index or not at all. A disabled
index answers
UNIMPLEMENTEDnaming it. GetTransactiongoes to a validator: raw transaction bytes are consensus data it already stores (why). With no validator answering, the txid-location index names the transaction’s block, fetched from p2p peers and checked against the hash our indexes hold (which does not cover v5 proofs and signatures).GetTaddressTransactionsworks the same way: the index names the txids,GetTransaction’s path supplies the bytes.- Zingod does not validate transactions. That would be a second mempool verifier (proofs, nullifiers, scripts, conflicts) tracking every network upgrade. Validators decide; Zingod checks only what the bytes, the next block and its own UTXO set decide (§5).
The principle behind the rest: verify what Zingod can, trust validators for as little as possible, take everything else from as many peers as possible.
2. The map
trusted validators (1–N, configured) peers ([p2p] on; default off)
JSON-RPC or gRPC, + zebrad push streams zebra-network peer set
│ │
└──────────────┬───────────────────────────┘
▼
zingod-traffic TrafficBalancer every request: lanes, hedge, blame
│
┌───────────────────────────┼─────────────────────────────────────┐
▼ ▼ ▼
HeaderSync (chainview) ObservationMerge + PeerMempool ChainView::validate_transaction
→ HeaderChain → mempool view → broadcast_transaction
(SendTransaction)
│ watch<HeaderChainSnapshot> │
├──────────────────┐ │
▼ ▼ │
FinalizedFollower Nfs (RAM) │
(zingod-sync) tip overlay │
│ FinalizedStream │ NfsPublication
▼ ▼ ▼
index writers ──▶ zingod-grpc-snapshot Publisher ──▶ one Arc<GrpcSnapshot> per request
(disk) durable views → zingod-grpc, /status, /readyz, /metrics
Two paths fold blocks, and their windows overlap:
- Finalized path (disk): blocks at or below the finalized tip, folded and committed by each index writer. Writers never see a reorg.
- Tip path (RAM, the NFS): every block above the indexes’ durable tips, finalized blocks not yet durable included, folded in memory and served at once. A block leaves once every index has committed it. Reorgs live here alone.
Both run the same fold per index. While the NFS folds, a block above the lowest durable tip is
folded on both paths: in RAM by the NFS, and by its writer once finalized. When the NFS stands down
(first sync, an index enabled late: §7) the finalized path folds alone.
3. Sources and what each is trusted for
A trusted validator is one the operator configured ([[trusted_validators]]). A peer is
any node on the p2p network: discovered, unauthenticated, free to multiply. Nothing a peer says
counts until Zingod has checked it.
| Need | Trusted validators | Peers | Check |
|---|---|---|---|
| Headers | ✓ (getblockheader runs) | ✓ (FindHeaders), after the anchor | trusted: link + work; peer: every rule |
| Anchor (first sync) | ✓ only source | headers below our durable tip (restart) | each hash = the one our indexes hold |
| Blocks with a header | ✓ | ✓ (by hash) | hash + merkle root + coinbase height |
| Blocks below the headers | ✓ only source (by height) | — | coinbase height (body trusted) |
| Mempool | ✓ (listing + fees) | ✓ (quorum of listings) | txid recomputed; §5 |
| Mined tx by id | ✓ only source | — | txid recomputed |
| Finality | ✓ (vouch) | ✓ (> 5 netgroups, no difficulty drop) | §4 |
| Submission | ✓ check (checkadmission) | ✓ relays (zebra’s, ~⅓ of peers) | §6 |
Why peers never decide the tip. A peer count proves nothing an attacker cannot buy: anyone can run ninety. Proof of work costs real mining however many peers present it. So peers are safe sources of headers and of anything checkable against them, and weak sources of claims. Two claims are taken from peers anyway, so Zingod keeps serving with every trusted validator down: finality and mempool admission. Both count only outbound peers (Zingod chose them) spread over more than 5 netgroups (/16 IPv4, /32 IPv6; one operator ≈ one netgroup).
4. The best chain
The Validated Header tree (zingod-header-chain, memory only) holds every valid branch above the
finalized tip; the best tip is the leaf with the most cumulative work. HeaderSync
(zingod-chainview) is its only writer and publishes an immutable HeaderChainSnapshot on a watch that
the NFS, the finalized path and the GrpcSnapshot publisher read.
finalized tip best tip (most work)
───●──●──●──●──●──●──●──●──●──●──● A ◀─
└──●──●──● B kept: may still win; overtaking A = a reorg
- Anchor. No history from genesis. With no chain, or a trusted validator so far ahead its anchor
lies above our fetch ceiling, the tree anchors at that validator’s claim − max(depth, 113) (113 =
the difficulty rule’s context). With no trusted validator answering (a restart mid-outage,
[p2p]on), the indexes’ own chain anchors instead: the 113 blocks below their durable tip, each peer header matched to the hash the indexes hold. A first sync has no such chain: only a trusted validator anchors it. - Trusted headers arrive in runs and enter on their parent link and nBits work alone: the validator already ran the rules.
- Peer headers (
[p2p]on, once anchored) run every consensus rule first (zingod_header_chain::validate, the rules). A header from the future is deferred, not blamed; one failing a rule gets its sender blamed. - Bounds. Side branches are capped in tips and nodes (past either, the lowest-work side leaf
goes), and only a window of finalized headers is kept
(limits).
depth=[sync] finalised_depth(default and floor).
Finality
A header proves work, not that its block’s transactions are valid. A finalized block is never reorged (the indexes hold it on disk), so finality is where Zingod stops being able to change its mind.
Every header node records its witnesses: who once had that block on its best chain, and so every ancestor too, for good. Each block carries all of its own accounting; nothing about it is kept anywhere else.
- Vouched. A trusted validator witnessed it (zebra commits only valid blocks). Vouches come
from the last header of each trusted run, each validator’s claimed tip, and its
getblockhashanswer at our best. Peer headers never vouch. - Peer-quorate (
[p2p]on). Live outbound peers in > 5 netgroups witnessed it by announcing it or a descendant. - finalized = min(highest settled, best − depth), where settled = vouched, or peer-quorate while no unvouched best-chain block has under half the work of the last vouched block. An attacker who eclipses us can feed a private chain, but its difficulty retargets to their hashrate, which shows up as that collapse. Both peer checks fail safe: a thin peer set or a real hashrate drop only delays finality.
- Neither holding: finality waits, the
finality pausedalarm rises (§11), serving continues from RAM. A trusted validator’s return vouches and finality resumes.
A reorg, step by step
Peers are first to every reorg: their headers enter on the rules alone, while a trusted validator
downloads and fully verifies the new branch before it switches, and Zingod learns of that on its
next poll (§9). Meanwhile the served tip drops to the fork point (§7) and synced closes (§8).
network ─── N ──┬── N+1 branch A, Zingod + the validator on it
└── N+1' ── N+2 branch B, more work, wins
──────────────────────────────────────────────────────────────────────────────────▶ time
t0 │ L2: validator verifies B │ L3 │ L4: fetch N+1', N+2 │
│ │ │ │ │
peers ─────────────┤ announce B headers │
│ │
best ─── N+1 ──────┤ N+2 (every header rule: PoW, difficulty, MTP, Equihash) │
│ │
served ── N+1 ─────┤ N ◀── fork point, vouched: what wallets get ───────────────────▶ N+2
│ │
validator ── N+1 ──┼────────────────────────────▶ N+2 │
│ │ │ │
vouched ─── N+1 ───┤ N ◀── last shared block ─────┼────▶ N+2 (poll, or push hint) │
| latency | what | bound |
|---|---|---|
| L1 | peers announce, headers validated, best switches | milliseconds |
| L2 | the validator downloads and verifies the branch, switches | its verification time |
| L3 | Zingod sees the validator’s new tip (the vouch) | one poll; ~0 with the push stream (§9) |
| L4 | Zingod fetches the branch’s blocks by hash and folds them | block fetch |
Through the window Zingod serves the fork point, a vouched block behind an unvouched best: a
currency gap, not a trust gap, and the state rule reads it so
(states). The tip unvouched alarm (§11) carries the
unvouched best. L2 is the validator’s job and no change on Zingod’s side removes it.
5. The mempool view
Every transaction the view knows is a sighting: its bytes (fetched once, txid recomputed), its
fee, a bitset of trusted validators listing it, a quorum flag and an ours flag.
Servable = listed by a trusted validator (not Down or catching up), admitted by a peer
quorum, or ours (accepted from our own submission). One trusted listing is enough: each trusted
validator is trusted.
Peer quorum ([p2p] on). Each live outbound peer is polled for its whole mempool, and each
listing counts while fresh (intervals). A
transaction is admitted when:
- more than 2/3 of the counted listings hold its exact id (txid + auth digest), from > 5 netgroups;
- its bytes decode, its expiry and (v5+) branch id fit the block above the verified best, it is not coinbase;
- every transparent prevout resolves in the served UTXO set or a held transaction’s outputs, and the resulting fee is not negative.
It stays while more than 1/3 still list it. No fee floor (zebrad’s and zakura’s differ), no proofs,
signatures or scripts: each lister is a full node that ran those. A consensus refusal is remembered
(REFUSED_TTL); an unknown prevout is retried next round. With honest peers two conflicting
transactions are never both admitted (one node’s mempool holds at most one, and > 2/3 + > 1/3
must overlap); a peer listing both can get both admitted and kept (mempool_model_tests.rs).
Cost to an attacker: > 2/3 of our outbound peers across 6 netgroups, for a served transaction that
never mines.
Transaction inv announcements are not used: nodes announce each transaction to a third of their
peers, once, so silence means nothing.
- Leaving. A transaction goes when nothing lists it and no quorum keeps it. A new tip clears
nothing; an unlisted
oursgoes at the next move of the best block. - Gate.
GetMempoolTx/GetMempoolStreamanswer only while aLivetrusted validator holds the verified best, or the polled peers span > 5 netgroups. OtherwiseUNAVAILABLE, never an unverified mempool.
6. Submission
SendTransaction asks one trusted validator for a verdict without a broadcast, answers the
wallet from it, then relays through the peers and watches the transaction spread. Relaying
every wallet transaction through the same validators would tell any observer which transactions
came through Zingod, and let one validator censor every wallet behind it. The relay follows
Dandelion++’s originator with its fail-safe timer.
precheck (nothing contacted): decodes (-22); expiry ≥ next block, v5+ branch = next block's (-25)
check: checkadmission on a random Live trusted validator holding the verified best
refused → wallet gets that refusal at once, nothing relayed
admissible → wallet answered Ok(txid) at once; transaction becomes `ours` (servable)
unsupported / unanswered → next holder; none → unchecked (below)
attempt: peers — zebra's own relay, PushTransaction to ⌈ready/3⌉ random ready peers
(no peers: a random untried trusted validator, sendrawtransaction)
unseen past propagation_threshold → next attempt, up to max_attempts
then, still unspread: one trusted sendrawtransaction (the fail-safe)
unchecked: Ok at the first acceptance (trusted Ok | any trusted listing | peer quorum);
never accepted → that trusted push is the verdict; else UNAVAILABLE
checkadmission(Zingod’s zebra fork) runssendrawtransaction’s checks, the fee policy and mempool conflicts included, without admitting or relaying. It goes to a holder of the verified best only (a validator behind it refuses what spends a fresh output). A validator without it (zcashd, stock zebrad, zakurad) answers-32601: never a failure, the next holder is asked. A refusal’s wire shape:SendTransaction.- A peer relay gets no reply and zebra names no recipient. Spread = a trusted validator never pushed to lists it, or a peer quorum admits it (past the ~⅓ one relay reaches). The job ends at spread, when nothing outside its trusted entries can be watched, or when attempts run out.
- Relays ride the peer set’s existing connections: a zebrad peer drops a second connection from
our IP, so a fresh connection per push would never deliver. With
[p2p]off every attempt is a trustedsendrawtransaction. [submission]setspropagation_threshold_secsandmax_attempts(SubmitPolicy).
7. Indexing
Finalized path (zingod-sync)
FinalizedFollower streams lowest durable tip + 1 ..= finalized tip into one FinalizedStream<Block>:
- Heights the header chain has a header for: fetched by hash from any member (peers included), checked against the header (hash, merkle root, coinbase height).
- Below the header chain (bulk sync, a restart from an old index): by height, trusted only, batched, checked by coinbase height only. A trusted validator’s body is trusted.
- Each block must link to every durable tip it extends and to the block sent before it; otherwise the follower stops and the process exits. Nothing is skipped.
- Queues are byte-bounded and a send reserves room in every queue first, so the slowest writer paces the fetch and memory stays bounded. A dropped queue or failed commit panics.
- Utxo-set resolves each transaction’s fee and feeds compact-block through a
FeeSink. - A writer commits when its buffer fills, after a run ending at the finalized tip, and at shutdown.
Tip path (zingod-nfs)
- Root = the lowest durable tip across indexes.
- The NFS folds only while the header just above the root is held and best is within 2 · depth of the root. Otherwise (first sync, an index enabled late) the finalized path does the work and the root is served.
- Each best-chain block above the root is fetched by hash, checked against its header, and folded over the indexes’ durable views (utxo-set first: its fees feed compact-block).
- Served tip = the deepest folded best-chain node, else the root. A reorg moves it to the fork point at once, then forward as the new branch folds. Side branches stay until the header chain drops them, so switching back costs nothing.
- A node goes once every index has committed its height.
8. Serving: one GrpcSnapshot
zingod-grpc-snapshot’s publisher combines the NFS’s latest NfsPublication and the chain view’s latest
publish into one Arc<GrpcSnapshot>. Every gRPC request or stream, probe, scrape and state log loads
one and reads only it, so no answer mixes two moments.
- No height named → the served tip, which every enabled index holds. A height named → up to that index’s own durable tip.
syncedopens when the served tip is the best block and no answering trusted validator claims a higher tip (ChainViewSnapshot::claimed_past_best: one already shown off our chain does not count, one not polled since best moved does), and stays while it is on the best chain withindepthof it. At boot, header sync anchorsdepthbelow a validator’s claim, so it stays shut until the header chain reaches that claim and the NFS has fetched the window.- The daemon’s state (
serving,degraded,down, …) is judged from the same load: states.
GetMempoolStream
The protocol says “close the stream when a new block is mined”; Zingod makes that exact. A stream
is bound to the served tip’s hash: it opens with the servable mempool, sends each new transaction
once (every message height: 0, never one mined in a block the served tip extends), and ends OK
once the served tip moves, never on an empty mempool. It ends only after the GrpcSnapshot
already serves the new tip, so the GetLatestBlock a wallet sends next sees it. One append-only
log per served tip is shared by every subscriber, so per-transaction work never scales with
subscribers.
9. Talking to validators and peers
Every request goes through one TrafficBalancer (zingod-traffic); lanes, intervals, hedge
delays and bench times are in its guide.
- Members. Each trusted validator, in
prioritytiers (0 first); then the whole peer set as one member below every tier (zebra picks the peer). Peers answer headers and blocks only. - Failure. A round asks each member once, best tier first; slow answers are hedged to another member. Blocks are retried until answered; everything else ends unanswered after one round.
- Blame. The balancer cannot judge a value; the caller checks it and
reports a bad one, which benches that member. Absent is never blame. The peer set is never benched: zebra scores the lying peer instead. - Polling. Each trusted validator is polled on its own permit, so no request delays tip
following, and sooner on a zebrad push event. Consecutive failures make it
Down, probed until it answers. A failing validator never stops the process. - Load on a zebrad =
max_connections. There is no request-rate or byte-rate limit.
10. When things fail
| Down | Chain + indexes | Mempool, SendTransaction | Finality |
|---|---|---|---|
| one trusted validator | ✓ | ✓ | ✓ |
every trusted, [p2p] on | ✓ near the tip (peers); bulk sync below the headers stalls | ✓ quorum; GetTransaction from the peers’ block | on peers’ word, else paused |
every trusted, [p2p] off | tip stops | UNAVAILABLE | paused |
every trusted, restart, [p2p] on | ✓ anchored on the indexes’ own chain, then as above | as above | as above |
| every trusted, first sync | nothing (only a trusted validator anchors) | UNAVAILABLE | — |
| every peer | ✓ (trusted) | ✓ | ✓ |
11. Telemetry
Observation only: none of it changes a tip or what is served. State is read from one GrpcSnapshot at
scrape (zingod_* and zingod_chainview_* gauges, /status); events are emitted where they
happen. Alarms log once WARN when raised, once INFO when cleared
(the alarms; stale = STALE_TIP_BLOCKS).
12. Seams
A trait exists only at a seam: a boundary with a second implementation (a real one or a test double), or one that keeps a dependency out of the crates above it.
zingod_source::ChainDataSource: JSON-RPC or indexer gRPC; tests drive the real client against aMockValidator.zingod_chainview::PeerSource,zingod_traffic::PeerTransport: zingod’sZebraPeers, which keeps zebra-network out of both crates.zingod_chainview::UnspentOutputs: the chain view sits below the indexes.zingod_persistence::PersistenceEngineandfs::Fs: indexes see the port, never the engine;SimFstests crashes.
Not seams: the gRPC service, each index, and the validator set (TrafficBalancer is not a
ChainDataSource, so a pool is never mistaken for one validator).