Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

KERI in Rust

libkeri

libkeri is a Rust implementation of KERI (Key Event Receipt Infrastructure), split into small crates that each do one job. It tracks keripy, the reference implementation: events are checked byte for byte against keripy, and the command line and witness interoperate with keripy's kli and witnesses.

All the code lives in the libkeri group on GitLab. This site is the place to start instead of browsing the repos.

You want to

The crates

CrateStatusWhat it does
libkeriworkingCore: CESR primitives, key events and key state, escrows, the stream parser, exn construction, and Habery/Hab for managing local identifiers
libkeri-httpworkingCESR-over-HTTP codec and an async client for witnesses and OOBIs, shared by the witness and the CLI
libkeri-witnessworkingA KERI witness. keripy can’t tell it apart from its own demo witnesses. No query or mailbox support yet
libkeri-cliworkingA kli-style command line, binary krust: init, incept, rotate, interact, multisig, delegation
libkeri-watcherworkingFollows another identifier’s KEL through its witnesses; the delegate uses it to wait for its delegator’s approval
libkeri-interopworkingScenarios against keripy 1.1, 1.2 and main, and the generator for the golden vectors
libkeri-mailboxplaceholderStore-and-forward mailbox, kept apart from the witness
libkeri-telplaceholderTransaction event logs: credential registries and status
libkeri-acdcplaceholderACDC credentials: building, schema validation, verification
libkeri-ipexplaceholderIPEX: exchanging credentials over exn
libkeri-suiteworkingThis site and the bootstrap script

How they depend on each other

libkeri                      core; depends on no other libkeri crate
├── libkeri-http             transport
├── libkeri-watcher          (libkeri-http only for its optional driver)
├── libkeri-mailbox
├── libkeri-tel
│   └── libkeri-acdc
│       └── libkeri-ipex
├── libkeri-witness          uses libkeri-http
└── libkeri-cli              uses libkeri-http, libkeri-watcher

Each crate depends on the others through git dependencies on GitLab main, so any one repo builds from a fresh clone. For working across several at once, see Getting started.

Why separate crates

keripy is one package that also runs its own event loop. Here each role is its own crate, so a consumer takes only what it needs and none of them forces a runtime on it. A few examples:

  • keripy’s witness also runs a mailbox; here those are libkeri-witness and libkeri-mailbox.
  • keripy bundles HTTP into its apps; here it is libkeri-http, and the core does no I/O at all.
  • keripy’s vdr and vc packages become libkeri-tel, libkeri-acdc and libkeri-ipex.

Getting started

Use one crate

Add it as a git dependency:

[dependencies]
libkeri = { git = "https://gitlab.vroblok.io/libkeri/libkeri.git", branch = "main" }

Work across the crates

Clone the suite and run its bootstrap script. It clones every repo next to the suite and writes a .cargo/config.toml there, so each crate builds against your local checkouts instead of what’s on GitLab:

git clone https://gitlab.vroblok.io/libkeri/libkeri-suite.git
libkeri-suite/scripts/bootstrap.sh          # or: scripts/bootstrap.sh ~/src/libkeri

Use LIBKERI_GIT=ssh to clone over SSH if you’ll push. The script leaves existing checkouts alone and never overwrites an existing .cargo/config.toml; if yours differs it writes the suggested one beside it.

A change in libkeri is then picked up by libkeri-cli, libkeri-witness and the rest on their next build, without pushing.

Build and try it

cd libkeri && cargo test                    # core tests and the keripy golden vectors
cd ../libkeri-cli && cargo install --path . # installs `krust`
krust init --name demo --nopasscode
krust incept --name demo --alias me --nopasscode
krust status --name demo --alias me

Run the interop scenarios

The scenarios need keripy’s kli on your PATH (keripy 1.2.x, for example through pyenv) and ports 5642–5644 free:

cd libkeri-interop
make           # every scenario
make 08        # one scenario
make matrix    # the delegation scenarios against keripy 1.1, 1.2 and main

make matrix also needs a keripy checkout at ../keripy; it exports each version from it without touching the checkout. See the libkeri-interop README for details.

Design

No runtime in the libraries

keripy runs everything in hio “Doers”: generators that a scheduler resumes on every tick. libkeri has no threads, locks or async in its core, and neither do the crates around it. The owner (a CLI, a server, an embedded app) drives the work and picks its own concurrency model.

  • Habery owns the key manager, the database and the event processors. Hab is a borrowing view of one local identifier.
  • Every network call lives in libkeri-http or in an optional driver feature.
  • A library that has to wait on someone else is sans-IO: it says what to fetch and decides from what it is given. libkeri-watcher’s next_request and feed work this way; its http feature adds an async polling loop.

Escrows are retried when data arrives

An event that can’t be accepted yet (short of signatures or witness signatures, out of order, or waiting on a delegator) is stored, not rejected:

  1. the event, its signatures, its source and the time it was first escrowed go into the database, with an index row per escrow;
  2. Habery::ingest routes a batch of messages, then makes one pass over the escrows;
  3. each pass accepts an entry, leaves it waiting, or drops it as invalid or older than its timeout.

With no new input nothing runs. An owner that wants timeouts enforced on a clock calls Habery::process_escrows() on its own schedule. Crates added later (libkeri-tel, libkeri-acdc) follow the same rule: their own escrow tables, and a process_escrows call the owner makes after ingest.

Approvals are explicit calls

Where keripy waits in a loop for something to change, libkeri exposes a call. For delegation:

  • the delegator lists pending_delegations() and calls approve_delegation(...);
  • the delegate’s watcher calls complete_delegation(...) once the delegator’s KEL holds a fully witnessed anchoring event.

keripy is the reference

Behaviour follows keripy 1.2.14. Code ported from keripy cites the keripy symbol (for example keripy: core/eventing.py Kever.escrowPWEvent), and a lint checks that every cited symbol exists. Every known difference is listed in Divergences; any difference not listed there is a bug.

Conformance with keripy

Three kinds of evidence show how closely libkeri tracks keripy:

EvidenceWhat it checksWhere
Golden vectorskeripy generates events, receipts and messages from fixed salts; libkeri rebuilds each one and must match byte for bytelibkeri/tests/keripy_vectors.rs, generated by libkeri-interop/vectors/gen.py
Trace lintevery keripy symbol cited in libkeri’s code exists in the pinned keripylibkeri/scripts/trace_check.py
Interop scenarioskrust and the Rust witness against kli and keripy witnesses, across keripy 1.1, 1.2 and mainInterop scenarios

Everything libkeri knowingly does differently, and the keripy behaviour the scenarios run into, is in Divergences.

Interop scenarios

Generated from the libkeri-interop README when the site is built.

#ControllerWitnessChecks
00keripy (kli)keripy (kli witness demo, wan)baseline: harness and environment work
01krustlibkeri-witness (wan)krust records the witness receipt
02keripy (kli)libkeri-witness (wan)kli resolves the Rust witness OOBI and records its receipt
03krustkeripy (kli witness demo, wan)krust records the receipt; a separate kli resolves the krust AID through the keripy witness and validates it
042-of-2 group: one keripy member, one krust membermembers on keripy wan; group unwitnessedkeripy proposes the group inception, krust checks and co-signs it, keripy accepts it from its partial-signature escrow; kli status and krust status show the same group
052-of-2 group, reversed: krust proposes, keripy joinsas 04krust proposes (multisig incept, escrowed), keripy joins and returns only its own signature, and libkeri completes the event from its partial-signature escrow (multisig ingest)
06krustkeripy wan, wil, wes; toad 2krust receipts with all three and sends each witness the other two receipts; each witness then serves a fully witnessed KEL and kli resolves the AID through each one alone
07kli, then krustlibkeri-witness as wan, wil, wes; toad 2the Rust witnesses take receipts propagated by kli and by krust, and serve /oobi/{aid}/witness/{wit} for the AIDs they witness; kli resolves both AIDs through each Rust witness alone
08keripy delegator (kli delegate confirm), krust delegatekeripy wankrust witnesses its dip, forwards it (/fwd from a proxy) to the delegator’s mailbox at wan, then its watcher follows the delegator’s KEL at wan until a fully witnessed event anchors the seal, completes the dip and publishes it; a separate kli validates the delegate. Then a delegated rotation
09krust delegator (krust delegate confirm), keripy delegate (kli incept --proxy)keripy wankrust watches the delegate’s witness OOBI for the dip, approves it (anchor + wan receipt), and kli finds the anchor; a separate kli validates. Then a rotation
10krust delegator, krust delegatelibkeri-witness wan (delegator), wil (delegate)the whole delegation on Rust components; a kli validator accepts the delegate through wil
11keripy delegator and delegatekeripy demo witnessesbaseline: keripy’s own scripts/demo/basic/delegate.sh, unchanged, for each keripy line (delegated inception, rotation, third-party validation)

keripy version matrix

Result of make matrix (scenarios 08–11 on each line):

1.11.2main
08 kli delegator, krust delegatepass, except A8 (XFAIL): a 1.1 validator can’t place a delegate that witnessed firstpasspass
09 krust delegator, kli delegateB1 (XFAIL): a 1.1 delegate asks before witnessing and krust has no mailboxpasspass
10 krust delegator and delegate, Rust witnessespass, except A8 (XFAIL) for the kli validatorpasspass
11 upstream delegate.shpasspasspass

Divergences from keripy

libkeri ports keripy (the reference KERI implementation) to Rust. This file lists where libkeri knowingly behaves differently. Any difference from keripy that isn’t listed here is a bug.

The evidence behind the rest:

  • tests/keripy_vectors.rs: events and receipts that keripy generated, which libkeri must reproduce byte for byte;
  • scripts/trace_check.py: every keripy symbol cited in the code must exist in the pinned keripy;
  • libkeri-interop: scenarios against kli and keripy witnesses.

Reference version: keripy 1.2.14 (bab95c16e), the version the vectors record.

Entries fall into three kinds:

  • Deliberate: a design choice. On the wire it matches keripy unless the entry says otherwise.
  • To fix: a known mismatch that should go.
  • Not implemented: keripy has it, libkeri doesn’t yet. Each entry says what libkeri does instead (panics, ignores, rejects).

Deliberate

AreakeripylibkeriWhy
ConcurrencyDoers (hio) run parsing and the escrows continuouslyNo threads, locks or async in the core. The owner drives work: Habery::ingest parses, routes and then runs Kevery::process_escrows once per batchlibkeri lets each consumer choose its own model (CLI, server, embedding). Escrow timeouts are only checked when the owner calls in
State ownershipHabery, Hab and Kevery share state freelyHabery owns Manager, Baser, Kevery and Revery; Hab<'a> is a borrowing viewOne copy of key state; Rust ownership
Unverified receipt escrow values (ures, uwes)qb64 values concatenatedDot-separated: said.prefix.cigar and said.wigqb64 never contains ., so values split without parsing sizes. Internal to the database
Delegation approval, delegate side1.1 Sealer: ask the delegator, then witness. 1.2/main Anchorer: witness, ask, then poll the delegator’s KEL (witq.query) and write the source seal couple (setAes) when foundWitness, ask (a /fwd to the delegator’s mailbox, as keripy), then a local watcher (libkeri-watcher) follows the delegator’s KEL from the delegator’s witness OOBIs. Habery::complete_delegation stores the couple only for a fully witnessed anchoring event that is the delegator’s last event at its sn, then the delegate publishes the event with -G and the delegator’s KEL to its witnesses and sends it to the delegatorNeither side writes approval state for the other; on the wire it matches 1.2/main
Delegation approval, delegator sidekli delegate confirm reads the delegable escrow (1.2+) or pses (1.1) that the mailbox fills; 1.2 writes the authorizer seal into its own db (_addAuthorizerSeal), main reprocesses the event directlykrust delegate confirm reads the delegable and misfit escrows; with no mailbox yet it fills them by watching the delegate’s witness OOBIs (--oobi). Habery::approve_delegation anchors the seal and reprocesses the event as local with that couple, as main doesNo mailbox in libkeri; works with 1.2/main delegates, whose witnesses hold the event before it asks
Witnessed group inceptionSupportedHabery::join_group_inception rejects a group inception with backers (“witnessed group inceptions are not supported yet”)Refuses rather than half-supports. The partially witnessed escrow now exists; the group flow is next

The escrow tables above are internal storage. A libkeri database isn’t meant to be opened by keripy, or the other way round.

To fix

Areakeripylibkeri
dt in replies and queries (reply.rs, query.rs)nowIso8601(): always 6 fractional digits and +00:00chrono::Utc::now().to_rfc3339(): up to 9 digits. keripy accepts it (it truncates to microseconds), but the bytes keripy would produce differ
help::helping::nowiso8601+00:00 offsetMicroseconds, but Z instead of +00:00, although its doc comment says it matches Python
Escrow timestamps (dtss)nowIso8601()to_rfc3339() (nanoseconds). Internal only
Partial-delegation escrow (escrowPDEvent, pdes/udes, processEscrowPartialDels)A delegated event whose delegating event is missing is escrowed and retriedRejected with MissingDelegationError (treated as escrowed by Habery::ingest but not stored): it must be sent again once the delegator’s KEL is known
Superseding rules for delegated recovery rotations (validateDelegation rules A–C)CheckedNot checked: a drt is accepted when its anchoring event is the delegator’s last event at that sn
Delegation source seal in the partial-signature, partial-witness and out-of-order escrowsStored with the event in udes (escrowPSEvent, escrowPWEvent, escrowOOEvent), reattached on retry and removed once acceptedNot stored, so a delegated event retried from escrow loses its seal
/loc/scheme and /end/role repliesBADA rule: a later dt replaces an earlier one, older ones are ignoredLast reply wins (process_reply_nontrans)

Not implemented

keripylibkeri behaviour when hit
Likely-duplicitous escrow (escrowLDEvent)Panics (todo!, Kevery::escrow_ld_event)
Receipt couples or transferable receipt quadruples attached to an event (not in a separate rct)Silently ignored: process_attached_receipt_couples and _quadruples return Ok, and route_message drops them
Replies signed by transferable AIDs (rpy with -F)Ignored, with a debug log (route_message)
Other message types (exn, ksn, TEL events, …)Ignored, with a debug log (route_message)
Manager.sign from a prefix and path, without keysPanics (unimplemented!, keeping/manager.rs)
Pather.path, Pather.rootPanics (unimplemented!, cesr/pather.rs)
Group rotation and interaction, signator hidden Hab, passcode/aeid encryptionNot available

keripy behaviour found by the delegation matrix

These are differences between keripy releases, or keripy bugs, that the libkeri-interop delegation scenarios (08–11) run into. They are recorded here so a failure against one keripy line isn’t mistaken for a libkeri bug.

keripyBehaviourEffect
1.1The delegate sends its event to the delegator before its witnesses receipt itA delegator without a mailbox (krust) never sees the request; scenario 09 marks it XFAIL (B1)
1.1A validator needs the -G couple; a 1.2+-style witness (keripy’s or libkeri’s) doesn’t store it for an event it witnessed before approvalA 1.1 kli can’t validate a delegate that witnessed first; scenario 08 marks it XFAIL (A8)
1.1Its witness rejects a /fwd from a delegate identifier that is mid-rotationkrust sends the drt through the proxy, as upstream delegate.sh does
1.2, mainA third party resolving a delegate’s witness OOBI stops at the dip; later drts stay escrowedSame with keripy’s own delegate (scenario 11); scenario 08 checks the rotation through the delegator instead
1.2delegate confirm waits for a drt with while eserder.sn < kever.sn (inverted; fixed in main cb512927)The wait returns at once; harmless here
mainkli writes KERI 2.0 events unless given --version 1.0The harness passes --version 1.0; libkeri reads 1.0 only
mainHTTP content type application/cesr (1.x: application/cesr+json); each answers 406 to the otherlibkeri-http retries a 406 with the other name

Components keripy bundles that live in separate crates here:

  • HTTP transport is in libkeri-http;
  • the witness server is in libkeri-witness;
  • the mailbox will be libkeri-mailbox;
  • the local KEL watcher for delegation approval is libkeri-watcher.

Adding an entry

When a vector, an interop scenario or a review finds a difference from keripy:

  1. Fix it, or add it here under the right heading.
  2. If you add it here, cite the keripy symbol in the code (for example keripy: core/eventing.py Kever.escrowPWEvent) so scripts/trace_check.py checks the reference.
  3. If a vector can show it, add one to libkeri-interop/vectors/gen.py and mark the Rust test #[ignore = "divergence: ..."] until it’s fixed.

Roadmap

Escrows first, then a seam in libkeri that hands non-KEL messages back to the owner, then credentials as three crates. Each phase ends with golden vectors and interop scenarios against keripy.

PhaseWorkCrate
1Finish the KEL escrowslibkeri
2The core seam: ingest hands back exn, TEL and credential messages; exn verification and its escrowlibkeri
3Transaction event logslibkeri-tel
4Credentialslibkeri-acdc
5Mailboxlibkeri-mailbox
6Credential exchange, and the krust commands for itlibkeri-ipex, libkeri-cli

Phase 1: KEL escrows

keripy escrowStatus
Partial signatures, partial witnesses, unverified (witness) receiptsdone
Misfit and delegable (delegation approval)done; approval is an explicit call
Out of order (escrowOOEvent)done
Likely duplicitous (escrowLDEvent)next; panics today
Partial delegation (escrowPDEvent) and the delegation seal kept with escrowed events (udes)to do
Superseding rules for delegated recovery rotationsto do
Unverified transferable receipts (vres)to do
Query not foundto do; needed by the mailbox
Reply escrow retryto do; stored but never retried
Receipts attached to eventsto do; ignored today

Phase 2: the core seam

The parser already reads exn, TEL events and credentials, but ingest drops them. This phase returns them to the caller, with exn messages verified against the sender’s KEL, so the credential crates get their input from the owner rather than from libkeri.

Phases 3 to 6

  • libkeri-tel: registries and status events (vcp, iss, rev, …), with out-of-order, anchorless and partially witnessed escrows. Issuing is a sequence of calls rather than keripy’s waiting loops.
  • libkeri-acdc: building, schema validation and verification of credentials, with escrows for a missing registry, chained credential or schema.
  • libkeri-mailbox: keripy delivers credentials through mailboxes, so this comes before IPEX.
  • libkeri-ipex: the six IPEX messages and their state machine, plus krust commands to issue, grant and admit.

Multisig issuers wait on group interaction and rotation in libkeri, which isn’t scheduled yet.

API documentation

Rustdoc for each crate, built with the site:

The placeholder crates (libkeri-mailbox, -tel, -acdc, -ipex) have no API yet.

Contributing

  • Reference version. Port from keripy 1.2.14 unless a change is specifically about another line. Cite the keripy symbol in the code.
  • Differences. Fix a difference from keripy, or add it to libkeri’s DIVERGENCES.md under the right heading. If a golden vector can show it, add one to libkeri-interop/vectors/gen.py.
  • No runtime in libraries. No threads, locks or async in the library crates; I/O goes in libkeri-http or an optional driver feature.
  • Checks before a commit. cargo test in the crates you touched, python3 scripts/trace_check.py in libkeri, and make in libkeri-interop. After delegation changes, also make matrix.
  • Branches. Work lands on main in each repo.