Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

Corium is a database system in the style of Datomic. It is immutable, time-aware, and fact-oriented. Queries run in the application process against an immutable database value. A single transactor process owns all writes to a database.

This manual is for the person who runs Corium. It covers configuration, initialization, keys, authorization, schema, backup, restore, availability, and recovery. It does not teach Datalog query authoring. For the query language, read the query engine design document.

How to read this manual

The manual has six parts.

  • Theory of operation explains datoms, the log, the indexes, and the process roles. Read this part first. Every operational rule in the manual comes from one of these ideas.
  • Running Corium covers installation, the transactor, storage backends, the database catalog, schema, and index publication.
  • Client surfaces covers the console, the dashboard, the SQL shell, the PostgreSQL wire server, and the peer server.
  • Security covers authentication, authorization, encryption at rest, and attribute protection.
  • Availability and data care covers high availability, backup, restore, forks, and garbage collection.
  • Operations collects the metrics and the runbooks.

The Reference part holds the command list, the environment variables, the default values, and the glossary.

Conventions

Commands appear as the corium binary. A source build runs the same command as cargo run -p corium-cli -- <command>. The manual writes corium for both.

Angle brackets mark a value that you supply, for example <database>.

This manual marks incomplete work in an aside. Two labels are used.

Not implemented. The feature is specified in a design document, but no code implements it. Do not plan a deployment around it.

Partly implemented. Some of the feature works. The aside states what works now and what does not.

Asides that carry neither label give background information.

Terminology

This manual uses one word for one idea.

WordMeaning
transactorThe process that owns writes for a database.
peerA library, or a process, that holds a database value and queries it locally.
storage serviceThe blob store and the root store together.
database valueAn immutable snapshot of a database at one basis.
basis, tThe transaction number that a database value covers.
datomOne fact: entity, attribute, value, transaction, and assert or retract.

The glossary holds the full list.

Getting started

This chapter builds Corium, starts a local system, and runs a query. The whole system is in memory. Nothing is written to disk, so there is nothing to clean up.

The steps take about ten minutes. Use four terminals.

Step 1 — Build the binary

Corium needs Rust 1.85 or newer. From the repository root, run:

cargo build -p corium-cli --release

The binary is target/release/corium. This manual writes corium for that path.

To build the optional storage backends, add their features. The installation chapter lists every feature.

Step 2 — Start a transactor

In terminal 1, run:

corium transactor --store mem --data-dir ./corium-data --listen 127.0.0.1:4334

The transactor prints the databases it serves. --store mem keeps everything in the process. The process loses the whole database when it stops.

--data-dir is required for every store. The mem store does not write to it.

Step 3 — Write a schema file

Create schema.toml in terminal 2:

schema-version = 1

[[entity]]
name = "person"

[entity.attributes]
name = { type = "string", unique = "identity", index = true }
age  = "long"

This file declares two attributes, :person/name and :person/age. The [[entity]] block is an authoring group. It does not create an entity type. The schema chapter explains the full format.

Step 4 — Create the database

corium db create people --schema schema.toml
corium db list
corium db stats people

db create sends the schema to the transactor as an ordinary transaction. db stats prints the basis, the datom count, and the transactor counters.

Step 5 — Write some data

The CLI has no transact command. Writes come from a client library, or from the PostgreSQL wire server. This step uses the wire server, because it needs no code.

In terminal 3, start the server with writes enabled:

corium postgres-server --listen 127.0.0.1:5432 --allow-writes

In terminal 4, insert two rows with psql:

psql 'host=127.0.0.1 port=5432 dbname=people' \
  -c "INSERT INTO corium.person (name, age) VALUES ('Ada', 36), ('Grace', 45)"

Each statement is one transaction. The SQL chapter states which statements the write path accepts.

The Rust, Clojure, Python, and Java clients all transact directly against the transactor. Use them for real data loading. See clients/python and clients/java.

Step 6 — Query the data

Open the Datalog console:

corium console people

Enter a query:

[:find ?name ?age
 :where [?e :person/name ?name]
        [?e :person/age ?age]]

The console also runs pull forms and time-view commands:

(pull [:person/name :person/age] 1000)

Type :basis to see the current transaction number. Type :quit to leave. The console chapter lists every command.

Run the same question in SQL:

corium sql people -c "SELECT e, name, age FROM corium.person ORDER BY name"

Open the dashboard to watch the system live:

corium tui people

Step 7 — Change the schema

Add an attribute to schema.toml:

[entity.attributes]
name  = { type = "string", unique = "identity", index = true }
age   = "long"
email = { type = "string", doc = "primary contact" }

Ask for the plan:

corium schema update people --schema schema.toml

The command writes nothing. It prints the plan and the digest of that plan. Apply exactly the plan you read:

corium schema update people --schema schema.toml --apply --plan <digest>

The last line of the plan is the invocation to run. The schema chapter explains the execution classes and the acknowledgement codes.

Step 8 — Stop the system

Press Ctrl-C in each terminal. The mem store discards the database.

Next steps

How Corium works

An operator who knows the data model can predict what Corium does under load, after a crash, and during recovery. This part of the manual explains the model. Every later chapter refers back to it.

Five ideas carry the whole system.

  1. A fact is a datom. Nothing is updated in place. A change asserts a new fact, and it retracts the old one. See datoms and the fact model.
  2. The log is the truth. A transaction is durable when its record is durable. Everything else is derived. See the transaction log.
  3. Indexes are a fold of the log. They are immutable, content-addressed blobs. They are an optimization, never a durability requirement. See indexes and storage.
  4. A database value is a snapshot. Time views name a basis. They do not copy facts. See time and database values.
  5. Writes and reads are separate roles. One transactor writes. Many peers read locally. See processes and roles.

The consequences an operator sees

The five ideas above produce the operational rules in this manual.

  • Index publication can lag without risk to durability. Lag costs cold-start time and backup freshness.
  • Garbage collection can strand garbage, but it cannot lose data, because it deletes only unreachable blobs after a retention window.
  • Crash recovery equals startup. The transactor replays the log tail. There is no repair step.
  • A peer never blocks on the transactor to get a database value.
  • A deposed transactor cannot publish, because every root write is a compare-and-set on the record that holds the lease.

Where the design documents are

This manual states what an operator needs. The design documents state why. They are in docs/design, and the decisions are recorded as ADRs in docs/adr.

Datoms and the fact model

The datom

The unit of information is the datom. A datom is a five-part fact.

PartNameContent
eentityA 64-bit entity id.
aattributeThe entity id of an attribute.
vvalueA typed value.
txtransactionThe entity id of the transaction that recorded the fact.
addedassert or retracttrue for an assertion, false for a retraction.

A datom is never modified. A new value for a cardinality-one attribute is one transaction that retracts the old datom and asserts the new one. The old datom stays in the history indexes.

Entity ids and partitions

An entity id is a 64-bit number. The high 22 bits hold the partition. The low 42 bits hold a sequence number.

Entities in one partition sort together in the EAVT index. The partition is therefore the locality control. Three partitions are built in.

PartitionHolds
:db.part/dbSchema entities, such as attributes.
:db.part/txTransaction entities.
:db.part/userApplication entities, by default.

Not implemented. User-defined partitions are described in the design documents. The engine has three partitions and no way to add a fourth. All application entities land in :db.part/user.

Transaction numbers

A transaction id is an entity id in :db.part/tx. The basis, written t, is the sequence part of that id. Conversion between t and tx is a bit operation.

t increases by one for each committed transaction. Every value of t up to the current basis names a real transaction.

Transaction time is data

Every commit asserts :db/txInstant on its own transaction entity. The commit time is a datom, not log metadata. Three consequences matter to an operator.

  • The commit time joins like any other fact. The clause [?tx :db/txInstant ?inst] binds the time of a transaction.
  • The attribute is AVET-indexed, so a wall-clock time resolves to a basis with an index seek.
  • Transaction entities are ordinary entities. Datom counts and entity counts include them.

The transactor stamps max(now, last + 1). This rule keeps commit times monotone. A transaction can supply its own :db/txInstant, which is how an import keeps original timestamps. The transactor rejects a supplied instant that does not advance the clock.

Value types

The engine has nine value types.

Schema typeHolds
:db.type/booleantrue or false.
:db.type/longA signed 64-bit integer.
:db.type/doubleA double, totally ordered.
:db.type/instantMilliseconds since the Unix epoch, UTC.
:db.type/uuidA 128-bit UUID.
:db.type/keywordAn interned keyword.
:db.type/stringUTF-8 text.
:db.type/bytesA byte array.
:db.type/refAn entity reference.

Keywords are interned per database. A keyword comparison is therefore an integer comparison.

One binary encoding serves the indexes, the log, and the wire. The encoding is sortable: the byte order of two encoded values equals the semantic order of the values. Index segments therefore compare without decoding.

Not implemented. Arbitrary-precision integers and decimals appear in the data-model design document, but the engine has no such value type. Store a big number as a string, or as a scaled long.

Not implemented. :db.type/fulltext behavior, tuple types, :db.type/uri, and :db.type/symbol are out of scope for version 1. See ADR-0009.

A stored value has one more shape than the nine above. Sealed holds a value encrypted under a protection class key. No schema declares it. It appears when the writing peer seals a value on a protected attribute, and it sorts after every plaintext type. See attribute protection.

Schema is data

An attribute is an entity in :db.part/db, described by datoms. The schema chapter covers the attribute properties.

Because schema is data, a schema change is a transaction. corium db create installs the first schema. corium schema update compares a file with the installed schema, and applies the plan you reviewed.

A database also carries a schema generation. It is a counter, separate from the basis. It advances once for each committed transaction that contains a schema change. The basis says when a change happened. The generation says whether two database values use the same schema.

Excision

Not implemented. Excision, which removes historical facts, is out of scope for version 1. It is the one operation that breaks immutability. The design reserves space for a filter set in the database root, applied at read time.

The transaction log

What the log is

The log is a totally ordered sequence of transactions. Each record holds t, the transaction instant, and the datoms of that transaction.

The log is the source of truth. The indexes are a deterministic fold of the log. Anything except the log can be rebuilt from the log.

The commit point

A transaction is durable when its log record is durable. The transactor acknowledges the caller after that point, and never before it.

Each record uses this frame:

payload-length | (1 << 63): u64 big-endian
payload: [u8; payload-length]
crc32c: u32 big-endian

The CRC32C covers the length word and the payload. Replay rejects a corrupted record even when the payload still decodes. A crash before the checksum is durable leaves a torn tail, and recovery truncates the whole incomplete frame.

The transaction pipeline

One logical thread of control serves one database. That thread is the write serialization point.

  1. Receive the transaction data.
  2. Resolve database functions. Built-in functions are native Rust. User :db/fn code runs in a sandboxed Clojure interpreter.
  3. Expand map forms and nested entities into list form.
  4. Resolve lookup references and tempids. A :db.unique/identity collision becomes an upsert.
  5. Validate against the schema: types, cardinality, and uniqueness.
  6. Retract the prior value of each cardinality-one attribute.
  7. Assign the transaction entity id and :db/txInstant.
  8. Append to the log and flush. This is the durability point.
  9. Apply the datoms to the in-memory live index.
  10. Acknowledge the caller.
  11. Broadcast the transaction report to subscribed peers.

Steps 1 to 5 are pure functions of the database value and the input.

Group commit

Concurrent transactions to one database commit as a batch under one durability boundary. Each transaction keeps its own t, its own report, and its own acknowledgement.

A caller enqueues its work and then contends to lead a flush. The leader validates each transaction against a staging value that already includes its predecessors. Uniqueness, cardinality-one retraction, and compare-and-set therefore see the same state as a sequence of single transactions.

The batch is one atomic log object. A takeover keeps all of the batch or none of it. A rejected transaction fails alone, and the rest of the batch still commits.

Batch size is capped by count and by encoded bytes. Under no contention a batch holds one transaction, so light-load latency is unchanged.

Partly implemented. Group commit works. Three write-path items remain: optimistic-apply overlap across separate batches, more than one flush in flight, and an explicit bounded queue with fast-fail backpressure. See write-path-scaling.md.

Where the log is stored

The log layout follows the store.

StoreLog layout
memAn in-process registry. The log dies with the process.
fsVersioned files under the data directory, named <db>.v<N>.log.
postgres, tursoOne row per transaction, keyed by database, lease version, and t.
s3One create-only object per transaction, with the same key.

On the native backends each commit is one create-only write. Success of that write is the durability point. An append is therefore O(1), and it does not read and rewrite a growing object.

The create-only condition is the fence of the log. A given lease version and t are written at most once.

Partly implemented. Log sealing is future work on the native backends. The design concatenates the per-transaction tail into content-addressed chunks and reclaims the small objects. Without that step, replay cost and list cost grow with the tail since the last index publication. Frequent index publication keeps that tail short.

Lease-versioned files and takeover

Each transactor appends only to the file, or the key prefix, of its own lease version. A pre-HA log reads as version 0.

Readers merge the versions in order. A reader discards any record in an older version whose t is at or after the first record of a later version. Those records are exactly the never-acknowledged appends of a deposed writer.

CAUTION: Never edit or delete log files by hand. Old lease-version files are inert history that readers need to merge correctly.

Reading the log

Two surfaces read the log directly.

  • tx-range(from-t, to-t) streams (t, instant, datoms) to any peer. It does not touch the covering indexes.
  • corium log --data-dir <dir> --db <name> prints committed transactions from a filesystem data directory. The transactor does not need to be running.

An encrypted log needs the key. Pass --storage-key to corium log. See encryption at rest.

Indexes and storage

The four covering indexes

Corium keeps four covering indexes. Each one is a total sort of all datoms, or of a subset of them. Every index holds whole datoms, so an answer comes from one index without a second lookup.

IndexSort orderContainsServes
EAVTe, a, v, txAll current datomsEntity access, pull
AEVTa, e, v, txAll current datomsColumn scans, clauses with a known attribute
AVETa, v, e, txDatoms of :db/index and :db/unique attributesValue lookups, ranges, uniqueness, lookup refs
VAETv, a, e, txReference-typed datomsReverse references, graph walks, component traversal

Each index has a history variant. The history variant keeps retractions and superseded assertions. Attributes marked :db/noHistory are not retained in the history indexes.

The operational rule follows from the table. An attribute needs :db/index or :db/unique before a query can seek it by value. Without one of them, a value lookup is a bounded scan of AEVT.

Immutable segments

Each index is a persistent tree. A leaf segment holds a sorted run of encoded datoms, about 50 KB to 100 KB compressed. An inner segment holds separator keys and child hashes.

A segment is addressed by the BLAKE3 hash of its bytes. Segments are write-once. A new tree reuses every unchanged subtree by hash.

Content addressing has three operational effects.

  • A segment is cacheable anywhere with no invalidation protocol.
  • A publication uploads only the leaves that changed, not the whole database.
  • A stale read is impossible. A reader either has the bytes or does not.

The database root

A database is named by its root record in the root store. The root holds the database name and id, the basis t, and the index basis t. It also holds the eight index roots, the log root, and the keyword table root. Last, it holds the schema revision, the garbage collection epoch, the format version, and the write lease.

The lease lives inside this record. Every lease acquisition, every renewal, and every index publication is a compare-and-set on the same bytes. One atomic operation therefore fences the writer, which is why no consensus protocol is needed.

The current database value equals the index trees at the index basis, merged with the log tail after that basis.

What a peer holds today

Partly implemented. The published format names content-defined leaf chunks under a manifest, so consecutive roots share every untouched chunk. The inner tree levels are future work. A reader therefore cannot seek into a published index, and it materializes the whole index in memory.

This limit has three consequences that an operator must plan for.

  • A peer keeps every datom it has seen, including retractions. Its memory tracks total history, not the size of the live database.
  • A cold time view costs a fold of the whole history, not of the view. First touch of a distinct as-of, since, or history view is slow.
  • Facts are allocated once and shared by handle, so the four indexes cost keys and pointers rather than copies.

Size a peer against total history. The time chapter states the cost of each view.

Incremental publication

One rule decides where a sorted key stream is cut. The published format and the in-memory segment obey the same rule, so a segment leaf is exactly one published chunk.

The indexing job therefore folds the log tail into the segments of the last publication. It rebuilds only the leaves that the tail touched, and it carries every other leaf across by handle. Work per pass tracks the tail, not the size of the database.

A pass that cannot reuse the last publication rebuilds each segment from the database value. A rebuild is a full re-encode, but not a full re-upload, because content-defined boundaries reproduce the chunks of the previous process.

The pass can carry chunks over only while it can prove that the root that names them stays live. It pins the index state through the root compare-and-set. If the pin fails, nothing is installed, and the pass retries from a rebuild.

Storage traits

The storage service has two parts.

  • The blob store holds immutable, content-addressed objects. Its operations are idempotent and need no ordering guarantee.
  • The root store holds a small mutable map of named pointers. It is updated by compare-and-set, and it is the only strongly consistent state in the system.

A writer uploads every segment before it publishes a root that names the segment. Any root that a reader obtains is therefore fully dereferenceable.

Garbage collection

Old roots keep old segments alive until no reader needs them. Collection is epoch-based and never urgent.

  1. The transactor bumps the collection epoch and records the live roots.
  2. Mark: walk the live roots and collect every reachable hash.
  3. Sweep: delete unreachable segments older than the retention window.

Deletion is the only mutation, and it touches only unreachable data. A bug in collection can therefore strand garbage, but a generous window makes data loss a non-risk. See garbage collection.

Segment cache

A read-through, size-bounded cache wraps blob store reads. A peer server can add an SSD tier with --segment-cache-dir and --segment-cache-capacity. The cache never covers mutable roots, and it is not part of durability.

Why this is safe

  • Segments are immutable, so no read sees a torn or stale segment.
  • A root is published only after every referenced segment is durable.
  • A root update is a compare-and-set fenced by the lease version, so a late publish by a deposed transactor fails cleanly.

Time and database values

The database value

A peer holds a database value. The value is immutable. A query runs against one value, so the answer cannot change while the query runs.

A time view wraps the same datoms with a different fold policy. A view never copies a fact.

The views

Given a connection whose latest known basis is t-now:

ViewMeaning
db()Current facts at t-now.
as-of(t)Facts as they stood at basis t.
since(t)Only facts added after t.
history()Every assertion and every retraction ever recorded.
sync(t)Completes when the basis of the peer reaches t.

as-of and since also accept a wall-clock instant. Corium resolves the instant to the last transaction committed at or before it. An instant older than the database resolves to basis 0.

The as-of and history views disable the uniqueness shortcuts of the planner. Uniqueness holds only in the current view.

Naming a view by wall clock

The t to instant correspondence is part of the database value, because every commit asserts :db/txInstant. Resolution in both directions is O(log n).

A derived view keeps the whole correspondence. An instant therefore means the same thing whatever value it starts from.

Five surfaces accept an instant.

  • Db::as_of_instant and Db::since_instant in Rust.
  • d/as-of and d/since in the Clojure API.
  • as_of_instant and since_instant in DbViewSpec on the wire.
  • :as-of <timestamp> and :since <timestamp> in the console.
  • \as-of <timestamp> and \since <timestamp> in the SQL shell.

The cost of a view today

Partly implemented. The design opens a view by descent through the segment tree. The implementation folds the view in memory from the recorded log. The costs below are what a peer pays now.

  • as-of folds the log up to its basis.
  • history folds the whole log.
  • since folds the whole log and then filters. It narrows before it projects, so the floor is applied while the index is built.

The result is cached in the database value, and it is shared by the clones of that value. A view that selects exactly the datoms of an already-folded view reuses that fold. A genuinely distinct view pays a full fold on first read.

The operational rule: a report that opens many distinct historical views is expensive on a peer. Reuse one view where possible.

Transaction reports

A peer receives a stream of transaction reports. Each report holds the basis before, the basis after, the datoms, and the tempid map for the submitting peer.

The peer applies the datoms to its live index and then offers the report to registered listeners. One stream serves three needs: keeping the peer basis current, sync, and application change feeds.

Reports arrive in t order with no gaps for a connected peer. After a reconnect the peer declares its basis, and the transactor backfills the gap.

Reports are not durable per consumer. A consumer that needs exactly-once delivery must track its own high-water t. tx-range replays the gap.

Reading the log directly

tx-range(from-t, to-t) streams transactions from the log tree. It is available on any peer, and it does not touch the covering indexes. Use it for audit, for replay, and for change-data capture.

Processes and roles

Corium separates three roles. Early versions can run them in one process, but no part of the core assumes that they share a process.

                    ┌────────────────────┐
   transact ───────►│     Transactor     │──── append ────► ┌─────────────┐
                    │  (single writer)   │──── segments ──► │   Storage   │
                    │  tx pipeline       │                  │   service   │
                    │  indexing job      │                  │ (blob store │
                    └─────────┬──────────┘                  │  + roots)   │
                              │ tx-report stream            └──────┬──────┘
              ┌───────────────┼───────────────┐                    │
              ▼               ▼               ▼            read segments
        ┌──────────┐    ┌──────────┐    ┌───────────┐              │
        │   Peer   │    │   Peer   │    │Peer server│ ◄────────────┘
        │ (in-proc │    │          │    │ (hosts db │
        │  query)  │    │          │    │ for thin  │
        └──────────┘    └──────────┘    │  clients) │
                                        └─────┬─────┘
                                              │ gRPC query/transact
                                        ┌─────┴─────┐
                                        │Thin client│ (any language)
                                        └───────────┘

Storage service

The storage service is passive. It has a blob store for immutable segments and a root store for named pointers.

Five backends exist: mem, fs, postgres, turso, and s3. See storage backends.

Transactor

The transactor is the single writer for a database. It serializes transactions, validates them, appends to the log, acknowledges the caller, and streams reports to peers.

A background job publishes fresh index trees. Exactly one transactor holds the write lease for a database at a time.

One transactor process serves many databases. It can be active for some databases and standby for others.

Peer

A peer is a library in the application process. It keeps a live connection for transaction reports, reads segments from storage through a cache, and merges them into an immutable database value.

All query execution happens on the peer: Datalog, pull, the entity API, index scans, and time views. Getting a database value never blocks on the transactor.

Peer state is either immutable or disposable. A peer crash loses nothing.

Peer server and thin clients

The peer server is a peer hosted as a standalone process. It exposes query, pull, and transact over gRPC for languages without the peer library.

One peer server hosts one database. See peer server and thin clients.

Operator service

Not implemented. An operator peer service is specified. The design runs backup, restore, fork, garbage collection, index publication, and the encryption migrations as resumable, auditable jobs, behind an API and a web interface. The CLI runs all of those duties in-process today. See operator-service.md.

Transactor fleet

Not implemented. A fleet topology is specified. The design places many databases across many machines behind one client address, with the same lease and failover guarantees. The implemented topology is an active and standby pair. See transactor-fleet.md.

Which process needs what

ProcessNeeds transactor addressNeeds storage credentialsNeeds storage key
corium transactorNoYesYes, for encrypted databases
corium peer-serverYesOnly with --peer-bootstrapYes, for encrypted databases
corium console, tui, sqlYesOnly with --peer-bootstrapYes, for encrypted databases
corium postgres-serverYesNoNo
Thin clientNo, it uses the peer serverNoNo
corium backupYesYesNot supported yet
corium restore, offline gc, logNoLocal data directoryYes, for encrypted databases

A thin client receives plaintext over TLS. It never holds a storage key.

Installation

Toolchain

Corium builds with a stable Rust toolchain, version 1.85 or newer. It uses edition 2024.

Build the CLI:

cargo build -p corium-cli --release

The binary is target/release/corium. Copy it to a directory on the path of the operator, for example /usr/local/bin/corium.

Run the test suite before you promote a build:

cargo test --workspace

Cargo features

Optional backends and authentication methods are Cargo features of corium-cli. A feature that is not compiled in makes its flags fail at startup with a clear error.

FeatureDefaultEnables
cljrsYesThe :db/fn Clojure transaction-function runtime.
postgresNo--store postgres.
tursoNo--store turso.
s3No--store s3.
oidcNoOIDC bearer tokens with a JWKS file.
oidc-discoveryNoOIDC, and JWKS fetch from the issuer.

Build a production binary with the backends that you deploy:

cargo build -p corium-cli --release --features postgres,s3,oidc-discovery

A backend can also be loaded at run time instead of compiled in. Build the driver crate on its own, and give the transactor its library path:

cargo build -p corium-store-turso --release

Do not enable the static-link feature when you build a loadable library. That feature is for a host that links the driver in. See storage plugins.

Workspace build note

corium-cljrs and the MusicBrainz example are excluded from the default workspace members. A --workspace build unifies the Clojure runtime into no-gc mode and degrades their garbage-collection semantics.

Build and test those two crates on their own:

cargo test -p corium-cljrs
cargo test -p corium-mbrainz

What a deployment needs

A minimal deployment has one transactor process and one storage backend.

Add a peer server only when a client language has no peer library. Add a PostgreSQL wire server only when a SQL client must reach the data.

ProcessDefault port
corium transactor4334
corium peer-server4336
corium postgres-server5432
Metrics endpointNone. Set --metrics-listen.

Directory layout of the fs store

The filesystem store keeps two directories under --data-dir.

PathContent
<data-dir>/storeBlobs and root records.
<data-dir>/logsVersioned transaction log files.

Back up the data directory as a unit, or use corium backup. Do not edit files in either directory by hand.

Process supervision

Run the transactor under a supervisor, such as systemd. Two rules apply.

  • Give the transactor a stable --owner value. A restarted member re-acquires its own unexpired lease at once.
  • Stop the transactor with SIGINT, which Ctrl-C sends. The transactor releases its leases on the way out. A standby then takes over without waiting for the lease to expire.

Partly implemented. The transactor and the peer server listen for SIGINT only. SIGTERM kills the process, which leaves the lease held until it expires. A shutdown by SIGTERM is safe, because takeover is ordinary crash recovery, but failover then costs one full lease time-to-live.

For systemd, set the stop signal explicitly:

[Service]
ExecStart=/usr/local/bin/corium transactor --config /etc/corium/transactor.edn
KillSignal=SIGINT
Restart=on-failure

The transactor

The transactor owns writes, logs, indexing, leases, and scheduled garbage collection. One process serves every database in its catalog.

Start a transactor

corium transactor --data-dir /srv/corium --listen 0.0.0.0:4334

The process prints the databases that it serves and the databases for which it stands by. It exits with an error when it cannot acquire a lease and --ha is not set.

Every flag below has an equivalent key in the configuration file. A flag on the command line always wins.

Identity and network

FlagDefaultEffect
--data-dir <path>None. Required.Data directory for the filesystem store and for logs.
--listen <addr>127.0.0.1:4334gRPC listen address.
--owner <id>transactor-$HOSTNAMEStable identity in lease records. Set it.
--advertise <url>NoneClient endpoint that peers use to find the lease holder.
--metrics-listen <addr>NonePrometheus endpoint at /metrics.

Set --owner to a stable value per member, for example the host name. A restarted member then re-acquires its own unexpired lease at once. A service manager usually does not export HOSTNAME, so the default becomes transactor-local on every member.

CAUTION: Keep the metrics listener on a private network. The endpoint has no bearer-token authentication.

Storage selection

--store picks the backend: mem, fs, postgres, turso, or s3. The default is fs. Each backend has its own flags and its own Cargo feature.

--store-plugin <path> loads a storage driver at startup, and --store <kind>:<json> then selects it. See storage backends.

Lease and availability

FlagDefaultEffect
--haOffStand by when another transactor holds the lease, instead of failing at startup.
--lease-ttl-ms <n>5000Failover detection bound. Renewals run at one third of this value.
--lease-wait-ms <n>15000How long startup waits for a held lease before it gives up. Ignored with --ha, which waits without limit.
--heartbeat-ms <n>10000Subscription heartbeat interval.

A lower time-to-live gives faster takeover. It also costs more root-store traffic, and it tolerates shorter pauses on the active member. See high availability.

Index publication pacing

FlagDefaultEffect
--index-interval-ms <n>5000Base interval between publications.
--index-backoff <n>4Minimum wait before the next publication, as a multiple of the duration of the last one. 0 disables it.
--index-tail-threshold <n>0Defer publication while fewer than this many datoms are pending. 0 publishes any pending work.
--index-tail-deadline-ms <n>60000Longest that a small tail defers publication.

These four values can also be changed per database at runtime. See index publication.

Garbage collection

FlagDefaultEffect
--gc-interval <duration>1hInterval of the scheduled sweep. off disables it.
--gc-window <duration>72hRetain unreachable blobs for at least this long.

Collection is serialized with index publication. See garbage collection.

Database functions

FlagDefaultEffect
--db-fn-fuel <n>1000000Execution credits per :db/fn call.
--db-fn-memory-bytes <n>16777216Managed memory per :db/fn call.

User :db/fn code runs on the transactor in a restricted Clojure interpreter. The interpreter has no input or output access. These two budgets bound a runaway function. Both flags need the cljrs feature, which is on by default.

Authentication, authorization, and TLS

FlagEffect
--serve-token <secret>Require this exact bearer token. Strict mode.
--require-authRequire the shared development token. Reject anonymous callers.
--serve-openAccept every request as anonymous.
--oidc-issuer <url>Accept tokens signed by this issuer. Strict mode.
--authz-db <name>Authorize every request against this policy database.
--tls-cert <pem>, --tls-key <pem>Serve TLS. Both are required together.

The default is permissive. The server recognizes the shared development token, and it also admits anonymous callers. Any of --serve-token, --require-auth, or --oidc-issuer switches the server to strict mode.

Read authentication and TLS before you expose a transactor outside a private network.

Encryption keys

--storage-key <uri> names a key that this process can resolve. The flag is repeatable, because one transactor hosts databases under different keys.

The same keyring holds key-encryption keys and protection class keys. A transactor needs no class key: it commits sealed values without opening them.

The process resolves every named key at startup. A misconfigured process therefore fails at startup and names the key. See encryption at rest.

Logging

Tracing is human-readable by default. --log-format json writes structured logs. RUST_LOG filters them.

RUST_LOG=corium_transactor=debug,corium_peer=info \
  corium --log-format json transactor --data-dir /srv/corium

--log-format is a global flag. It comes before the subcommand.

A production example

corium transactor \
  --config /etc/corium/transactor.edn \
  --listen 0.0.0.0:4334 \
  --advertise http://txor-a.internal:4334 \
  --owner txor-a \
  --ha \
  --metrics-listen 127.0.0.1:9464 \
  --serve-token "$CORIUM_SERVE_TOKEN" \
  --authz-db corium_authz \
  --tls-cert /etc/corium/tls/server.pem \
  --tls-key /etc/corium/tls/server.key

The configuration file holds the storage selection and the read-only discovery credentials. The command line holds the identity of the member.

Storage backends

--store selects where the transactor keeps blobs, root records, and the transaction log. Five backends are built in.

StoreCargo featureBlobs and rootsLogShared between hosts
memBuilt inProcess memoryProcess memoryNo
fs (default)Built in<data-dir>/store<data-dir>/logsOnly on a shared filesystem
postgrespostgresPostgreSQL tablesPostgreSQL rowsYes
tursotursoTurso database fileTurso database fileNo
s3s3S3 objectsS3 objectsYes

--data-dir is required for every store. The mem store does not write to it.

A backend is a driver that enters a process-wide registry under a backend kind. mem and fs live in the engine. The other three are separate crates that a Cargo feature links in. A driver can also be loaded at run time. Read storage plugins below.

Read-only discovery credentials

A storage-aware peer, and an online backup, read storage directly. They ask the transactor for connection details with the GetStorageInfo call.

GetStorageInfo never returns the primary write credential of a service backend. For PostgreSQL and S3 you must provision a separate read-only credential. Without it, storage-aware peer bootstrap and online backup fail with an explicit error. They do not fall back to the write credential.

Local filesystem and Turso stores need no separate credential.

mem

corium transactor --store mem --data-dir ./corium-data

Everything lives in one process. The database is lost on exit. Use mem for demonstrations and tests.

An online backup cannot open a process-local memory store. The backup command rejects it clearly.

fs

corium transactor --store fs --data-dir /srv/corium

Blobs are files under <data-dir>/store. Logs are versioned files under <data-dir>/logs.

For a high-availability pair, both members must see the same directory over a shared filesystem. Never run two members against diverged copies of a data directory.

postgres

corium transactor --store postgres \
  --postgres-url 'postgresql://corium@db.example/corium?sslmode=require' \
  --postgres-read-only-url \
    'postgresql://corium_reader@db.example/corium?sslmode=require' \
  --data-dir /srv/corium

The backend creates corium_blobs and corium_roots in the current schema of the connection. It stores log objects as fenced root records with log: names. TLS uses the platform certificate store.

Provision the read-only role with SELECT on both tables. Pass its URL with --postgres-read-only-url, or with CORIUM_POSTGRES_READ_ONLY_URL.

Readers use ordinary MVCC. They do not contend with the root compare-and-set writer.

turso

corium transactor --store turso --data-dir /srv/corium --turso-path /srv/corium/store.db

Turso is an embeddable SQLite database. --turso-path defaults to <data-dir>/store.db.

Turso 0.7 needs its experimental multi-process write-ahead log when independent processes open one file. Corium enables that mode. Every process that touches the file must run the same mode.

s3

AWS_REGION=us-east-1 \
corium transactor --store s3 \
  --s3-bucket corium-prod --s3-prefix corium/ \
  --s3-region us-east-1 \
  --s3-read-only-role-arn arn:aws:iam::123456789012:role/corium-reader \
  --data-dir /srv/corium

Blobs go under {prefix}blobs/. Roots, including versioned log objects with log: names, go under {prefix}roots/.

Root publication is fenced with S3 conditional writes, If-None-Match and If-Match. The bucket, or the S3-compatible substitute, must support them.

Provision the bucket yourself. Corium does not create it, because bucket creation involves region and ownership choices that Corium must not make for you.

Primary credentials come from the standard AWS configuration chain: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_PROFILE, and instance or task roles. Region and endpoint come from that chain, or from --s3-region and --s3-endpoint-url.

Read-only discovery credentials take one of two forms.

  • Static keys. Pass --s3-read-only-access-key-id and --s3-read-only-secret-access-key, and an optional session token. Restrict these keys to reads of the Corium prefix yourself.
  • AWS STS role. Pass --s3-read-only-role-arn. Corium generates short-lived credentials on every GetStorageInfo call, with a session policy limited to s3:GetObject and prefix-scoped s3:ListBucket. The generated token cannot write, even when the role has broader permissions. The AWS identity of the transactor must be allowed to assume the role.

The two forms conflict. Use one of them.

A custom --s3-endpoint-url implies path-style addressing.

Storage plugins

A storage driver can be a dynamic library that the transactor loads at startup. An external author can therefore add a backend without a rebuild of the engine.

corium transactor \
  --store-plugin /opt/corium/plugins/libacme_gcs.so \
  --store 'acme-gcs:{"bucket":"corium-prod"}' \
  --plugin-read-only-config '{"bucket":"corium-prod","role":"reader"}' \
  --data-dir /srv/corium
FlagEnvironmentEffect
--store-plugin <path>CORIUM_STORE_PLUGINSLoad a plugin library. Repeatable.
--store <kind>:<json>Select a registered backend and pass it a JSON configuration object.
--plugin-store <kind>:<json>The same form under an older name. It conflicts with --store.
--plugin-read-only-config <json>CORIUM_PLUGIN_READ_ONLY_CONFIGThe read-only configuration returned by GetStorageInfo.

CORIUM_STORE_PLUGINS takes a path-separator-delimited list of files and directories. Corium searches a directory for platform dynamic libraries only. It never adds the working directory.

The EDN configuration file carries the read-only configuration under :plugin-read-only-config. It does not carry the plugin paths or a kind:{json} store.

--plugin-read-only-config is what GetStorageInfo returns to a storage-aware peer or an online backup. Corium never returns the primary configuration of a plugin backend. Without the read-only configuration, discovery against a plugin store fails.

Verifying a backend

corium store verify acme-gcs '{"bucket":"verification"}' \
  --store-plugin /opt/corium/plugins/libacme_gcs.so

The command runs the blob and root conformance suite against a live backend. It creates uniquely named objects, exercises idempotence, listing, compare-and-set fencing, and deletion, and removes the objects afterward.

Use a disposable namespace. Do not run the verifier against a bucket or a database that holds live Corium data.

What a plugin costs

CAUTION: A plugin is native code that Corium runs in its own process. Install plugins only in directories that you control, and name them by explicit path.

  • A plugin can receive storage credentials, because backend configuration crosses the boundary as JSON.
  • A loaded library is never unloaded.
  • Each library carries its own async runtime and its own process globals, so a process that loads several drivers pays for each.
  • Encryption sits above the storage interface. A driver receives ciphertext and never receives key material.

Partly implemented. Only corium transactor and corium store verify load plugins. corium peer-server, corium console, and the other client commands do not, so --peer-bootstrap against a plugin backend fails with “storage backend is not available”. Use a built-in backend where a storage-aware peer is needed.

The plugin contract is documented in storage-plugins.md.

Keeping secrets out of the process arguments

Static secret fields are also read from the environment:

  • CORIUM_S3_READ_ONLY_ACCESS_KEY_ID
  • CORIUM_S3_READ_ONLY_SECRET_ACCESS_KEY
  • CORIUM_S3_READ_ONLY_SESSION_TOKEN
  • CORIUM_POSTGRES_READ_ONLY_URL
  • CORIUM_PLUGIN_READ_ONLY_CONFIG

Prefer the environment, or a protected configuration file, over a process argument.

Choosing a backend

SituationBackend
Demonstration or testmem
Single host, simple operationfs
High availability without a shared filesystempostgres or s3
Existing PostgreSQL operations practicepostgres
Large database, object storage economicss3
Single-file embedded deploymentturso
A service Corium does not supportA plugin

A high-availability pair on fs needs a shared filesystem. postgres and s3 remove that requirement, because they store the log natively.

The configuration file

corium transactor --config <path> reads storage selection and read-only discovery credentials from one EDN file. A flag on the command line overrides the value in the file.

The file holds one EDN map. Keys are plain keywords without a namespace. An unknown key is an error, and the message names the key.

Example

{:store :s3
 :data-dir "/srv/corium"
 :s3-bucket "corium-prod"
 :s3-prefix "corium/"
 :s3-region "us-east-1"
 :s3-read-only-role-arn "arn:aws:iam::123456789012:role/corium-reader"
 :s3-read-only-role-duration-seconds 900}
corium transactor --config /etc/corium/transactor.edn

Keys

KeyTypeEquivalent flag
:storeKeyword: :mem, :fs, :postgres, :turso, :s3--store
:data-dirString--data-dir
:turso-pathString--turso-path
:postgres-urlString--postgres-url
:postgres-read-only-urlString--postgres-read-only-url
:plugin-read-only-configString holding a JSON object--plugin-read-only-config
:s3-bucketString--s3-bucket
:s3-prefixString--s3-prefix
:s3-regionString--s3-region
:s3-endpoint-urlString--s3-endpoint-url
:s3-read-only-access-key-idString--s3-read-only-access-key-id
:s3-read-only-secret-access-keyString--s3-read-only-secret-access-key
:s3-read-only-session-tokenString--s3-read-only-session-token
:s3-read-only-role-arnString--s3-read-only-role-arn
:s3-read-only-role-session-nameString--s3-read-only-role-session-name
:s3-read-only-role-duration-secondsInteger--s3-read-only-role-duration-seconds
:s3-read-only-role-external-idString--s3-read-only-role-external-id

What the file does not hold

The file covers storage only. It does not hold the listen address, the owner identity, or the lease values. It does not hold the index pacing, the garbage collection schedule, the authentication flags, or the storage keys.

:store names a built-in backend only. A plugin backend needs --store <kind>:<json> on the command line, and the file carries no plugin paths. See storage plugins.

Put those on the command line, or in the unit file of the service manager.

Not implemented. There is no configuration file for the peer server, the PostgreSQL wire server, or the client commands. They take flags and environment variables only.

Protecting the file

The file can hold static secrets. Two rules apply.

  • Set the file mode so that only the transactor user can read it. For example, run chmod 600 /etc/corium/transactor.edn.
  • Prefer the file, or the environment variables listed in environment variables, over process arguments. Process arguments are visible to every user on the host.

The database catalog

One transactor serves many databases. The corium db commands operate the catalog. Every one of them talks to a running transactor.

Connection flags

Every client command shares the same connection flags.

FlagDefaultEffect
--transactor <url>http://127.0.0.1:4334Transactor endpoint. A comma-separated list gives failover order.
--token <secret>Shared development tokenBearer token. --token "" connects anonymously.
--ca <pem>NoneCA certificate to trust. Enables TLS.
--tls-domain <name>NoneDomain expected on the server certificate.
--peer-bootstrapOffRead the published snapshot from storage instead of replaying the log from basis 0.

CORIUM_TOKEN sets the token for every command.

Administrative commands use the first endpoint in the list. Peer connections fail over across the whole list.

Create a database

corium db create people --schema schema.toml

The command prints {:db "people" :created true}.

A database name holds 1 to 128 characters. Only ASCII letters, digits, -, and _ are allowed.

The schema file is EDN, or TOML when the path ends in .toml. Omit --schema to create an empty database. See schema management.

To encrypt every durable artifact of the database, add --storage-key:

corium db create people --schema schema.toml --storage-key file:/etc/corium/storage.key

Encryption is fixed at creation. See encryption at rest.

db create is idempotent, and it does not update an existing database. An existing name prints {:db "people" :created false}, and the schema file is ignored.

To change the schema of a database that already exists, use corium schema update. See schema management.

List databases

corium db list

The command prints the names that the transactor serves.

Inspect a database

corium db stats people

The command connects a peer, syncs it, and prints one EDN map:

{:basis-t 1240 :index-basis-t 1200 :datoms 91234 :entities 20114
 :attributes 37 :index-lag 40 :tx-count 1240 :tx-failures 2
 :tx-queue-depth 0 :gc-runs 17 :gc-swept-blobs 214}
FieldMeaning
:basis-tNewest committed transaction that the peer has seen.
:index-basis-tTransaction covered by the published indexes.
:datoms, :entities, :attributesCounts in the current value.
:index-lagTransactions committed after the published index basis.
:tx-count, :tx-failuresTransactor counters since process start.
:tx-queue-depthCommit queue depth now.
:gc-runs, :gc-swept-blobsGarbage collection counters since process start.

db stats replays from basis 0 unless --peer-bootstrap is given. On a large database that is slow. Add --peer-bootstrap when the client can reach the storage backend.

Partly implemented. db stats does not print the lease owner. The Metrics panel of corium tui shows lease ownership and the advertised endpoint, from the same Status call.

Delete a database

corium db delete people

The command prints {:db "people" :deleted true}.

CAUTION: db delete asks for no confirmation, and it cannot be undone. The command deletes the database root, the metadata root, the key manifest, and every log record at once. Blobs stay until garbage collection sweeps them. Take a backup first.

Fork a database

corium db fork creates a new database that duplicates an existing one at a transaction basis. Use it for a writable sandbox against real data. See forking a database.

Index publication

corium db request-index and corium db index-policy control when the transactor publishes fresh index trees. See index publication.

Schema management

A schema declares attributes. An attribute has a name, a value type, a cardinality, and optional properties.

The schema is data. Corium keeps it in the same log as every other fact. The schema of a database at basis t is therefore a question you can ask.

Two commands install attributes.

CommandPurpose
corium db create --schema <file>Install the first schema with the database.
corium schema update <db> --schema <file>Compare a file with the installed schema, and apply the plan you reviewed.

Both commands read the same file. The CLI reads TOML when the path ends in .toml. Every other extension is read as EDN.

corium db create people --schema schema.toml
corium schema update people --schema schema.toml

The rest of this chapter describes the file format, the properties, and the update procedure.

TOML format

The TOML format is an authoring layer over the flat attribute model.

schema-version = 1

[[entity]]
name = "person"

[entity.attributes]
id      = { type = "uuid", unique = "identity" }
name    = { type = "string", index = true }
age     = "long"
tags    = { type = "keyword", many = true }
address = { type = "ref", component = true }

[[entity]]
name = "organization"

[entity.attributes]
name      = "string"
employees = { type = "ref", cardinality = "many" }

The first block declares :person/id, :person/name, :person/age, :person/tags, and :person/address. A bare string is shorthand for that type with cardinality one.

An [[entity]] block is an authoring group only. It supplies the keyword namespace. It does not create an entity type. It does not constrain which attributes can appear together, and it does not constrain the target of a reference attribute.

Each group name can appear in at most one [[entity]] block.

schema-version is the version of the file format. It is not a migration number, and Corium does not compare it between runs.

Flat attributes

Top-level declarations express ungrouped attributes, or add attributes to a group without an entity block:

[[attribute]]
name = "created-at"
type = "instant"
index = true

[[attribute]]
group = "audit"
name = "created-by"
type = "ref"

These declare :created-at and :audit/created-by. Declaring one canonical attribute through both syntaxes is an error.

Attribute options

Every detailed declaration requires type.

OptionValuesDefault
typeboolean, long, double, instant, uuid, keyword, string, bytes, refRequired
manyBoolean cardinality shorthandfalse
cardinality"one" or "many""one"
unique"identity" or "value"Unset
indexBooleanfalse
componentBooleanfalse
no-historyBooleanfalse
docDocumentation stringUnset
protectionA declared class, as "protect/<name>"Unset

Use only one of many and cardinality on a declaration. A unique attribute receives index coverage whether or not index = true is present.

A [protect.<name>] section declares a protection class. Read attribute protection for the class options and for what protection costs.

Name rules

Group and attribute names are preserved exactly. They must be valid EDN keyword components, so that the resulting idents work in queries, transactions, and console input.

A name cannot start with a digit. It cannot contain whitespace, /, :, or EDN delimiter and reader-macro punctuation.

A quoted TOML key carries an EDN-valid name that is not a valid bare TOML key. Quoting does not bypass the validation.

[entity.attributes]
"active?" = "boolean"

EDN format

The EDN format is the Datomic-style attribute map. The file holds one vector of maps, or a sequence of bare maps.

[{:db/ident :artist/gid
  :db/valueType :db.type/uuid
  :db/cardinality :db.cardinality/one
  :db/unique :db.unique/identity}
 {:db/ident :artist/name
  :db/valueType :db.type/string
  :db/cardinality :db.cardinality/one
  :db/index true
  :db/doc "Credited name of the artist."}
 {:db/ident :medium/tracks
  :db/valueType :db.type/ref
  :db/cardinality :db.cardinality/many
  :db/isComponent true}]
KeyValuesDefault
:db/identKeyword. Required.None
:db/valueType:db.type/ plus boolean, long, double, instant, uuid, keyword, string, bytes, ref. Required.None
:db/cardinality:db.cardinality/one or :db.cardinality/manyone
:db/unique:db.unique/identity or :db.unique/valueUnset
:db/indextruefalse
:db/isComponenttruefalse
:db/noHistorytruefalse
:db/docStringUnset
:db/protectionA declared class, for example :protect/piiUnset

What each property does

:db/unique. :db.unique/identity makes a tempid collision an upsert. :db.unique/value makes a collision an error. Both give the attribute AVET coverage, so a lookup reference [:person/email "a@b.c"] works.

:db/index. The attribute gets AVET coverage. A query can then seek it by value or by range. Without it, a value lookup is a bounded scan of AEVT. Index coverage costs write throughput and storage.

:db/isComponent. A reference attribute whose targets are retracted with the parent by :db/retractEntity, and pulled recursively by default. Use it for owned sub-entities, not for shared references.

:db/noHistory. The attribute is not retained in the history indexes. Use it for high-churn values whose past does not matter, for example a counter. The past of that attribute cannot be recovered afterward.

:db/doc. Free documentation text. It has no effect on queries.

:db/protection. Values on the attribute are sealed by the writing peer under the key of the named class. Read attribute protection.

Reference attributes are always covered by VAET. A reverse-reference query therefore needs no extra declaration.

Protection cannot be combined with :db/index, :db/unique, or :db.type/ref. Ciphertext order is not value order, so a protected attribute can never appear in a value-ordered index.

Enumerated values

An entity outside the :db.part/db partition can carry :db/ident as an ordinary name. An ordinary transaction writes it, and a keyword in a reference position resolves through it.

[{:db/ident :status/active}
 {:db/ident :status/retired}]

A :db.type/ref attribute can then name :status/active as its value. Where this shape is not needed, a plain :db.type/keyword value is simpler.

Updating an installed schema

corium schema update compares a schema file with the schema installed in a database. It prints the changes, their cost, and their meaning.

corium schema update people --schema schema.toml

The command is read-only without --apply. It plans against one immutable database value, so every count in the plan is measured at one basis and one schema.

The plan

Each difference is one property-level change with an execution class.

ClassMeaningExamples
additiveNo existing fact is inspected or rewritten.A new attribute. Cardinality one to many.
validate-reindexExisting facts stay valid, but a bounded scan, a constraint validation, or an index rebuild is needed.Add index or unique. Change the uniqueness mode. Toggle isComponent. Retire an attribute.
rewriteCurrent facts must change first.Collapse cardinality where an entity holds several values.
destructiveInformation or historical meaning is lost.Change the value type of an attribute in place.

Risk is reported beside the class, and the two are independent. An AVET backfill is expensive and semantically harmless. A metadata-only isComponent change is cheap and changes the meaning of every live reference.

A plan carries a digest. A later schema change, or a failed safety condition, invalidates the digest. Ordinary data writes do not.

database: people  basis: 418  schema-generation: 3
desired:  sha256:709f947a…
plan:     sha256:ad5cb3c4…

ADDITIVE
  + :person/email string cardinality-one
  + :person/tags keyword cardinality-many

VALIDATE-REINDEX
  ~ :person/address component false -> true
      live refs: 8109
      [ack: component-enable] existing references acquire cascade retract and pull semantics
      note: existing facts are not rewritten; future pull and retract-entity semantics change

UNMANAGED
    :legacy/import-id    use --prune to retire

3 change(s) planned. Nothing was written.
To apply: corium schema update people --schema <file> --apply --plan sha256:ad5cb3c4… --allow validate-reindex --ack component-enable

The last line is the exact invocation to run. Copy it, and add the path of the schema file.

Acknowledgement codes

A change whose meaning changes carries a stable code. Pass the code back with --ack.

CodeWhat you accept
component-enableExisting references acquire cascade retract and pull semantics.
component-disableExisting references lose those semantics.
unique-mode-changeUpsert and conflict behavior changes for future writes.
no-history-enableHistory stops being recorded from this transaction onward.
no-history-disableHistory resumes. The interval already omitted cannot be reconstructed.
retire-live-attributeNew assertions are refused while existing facts stay readable.
protection-forward-onlyProtection changes are forward-only and cannot re-seal existing facts.

Allowing an execution class says which work can run. An acknowledgement says that you understood what the change means. The two are separate on purpose.

Partial files and retirement

A file manages the declarations that it contains. An installed attribute that the file does not name is reported as unmanaged, and the command leaves it alone. --prune turns every unmanaged attribute into a retirement request.

Retirement is not deletion. A retired attribute keeps its ident, its metadata, and its history. New assertions are refused. Retractions stay legal, which is what makes retirement a usable step when an application moves to a replacement attribute.

A retirement step prints the current datoms, the current entities, and the recorded history datoms. It needs --ack retire-live-attribute only when the attribute still holds live facts. Retracting those facts is separate work.

Idents are matched exactly. A removed ident and an added ident are two changes, never an inferred rename. An incorrect rename aliases two meanings permanently.

Engine attributes such as :db/txInstant are never managed by a file. A file that declares one is rejected with an explicit error.

Applying a plan

Run the same command again with --apply, the digest that the plan printed, and every allowance and acknowledgement that the plan asked for.

corium schema update people --schema schema.toml \
  --apply --plan sha256:ad5cb3c4… \
  --allow validate-reindex --ack component-enable

The transactor recomputes the plan under its writer queue. If the digest does not match, the transactor refuses the apply and changes nothing. Ordinary writes between the review and the apply are safe. Only a schema change or a failed condition invalidates the plan.

A successful apply prints the basis and the new schema generation:

Applied 3 change(s) to people at basis 419 (schema-generation 4).
  + :person/email
  + :person/tags

An apply that has already landed succeeds and prints No changes. Installing a change is what invalidates the digest that described it, so the command re-plans, finds nothing to do, and writes nothing. The command is therefore safe in a pipeline.

Applying needs the alter-schema authority. That authority is separate from transact, so an application writer cannot broaden its own vocabulary. Read authorization.

Flags

FlagEffect
--schema <path>The desired schema file. Required.
--pruneRetire the installed attributes that the file omits. Part of the digest.
--jsonPrint the versioned machine contract instead of the human report.
--detailed-exit-codeExit 0 for no change and 2 for changes planned.
--applyApply the plan. Requires --plan.
--plan <digest>The digest that the read-only plan printed.
--allow <class>Permit an execution class above additive. Repeatable.
--ack <code>Acknowledge a semantic change. Repeatable.

A successful plan exits 0 whether or not it finds changes, so && chains keep working. --detailed-exit-code changes that to 0 for no change and 2 for changes planned.

A failure exits 1. With --json the failure carries a stable code: parse-error, connect-error, plan-error, plan-mismatch, allow-required, ack-required, blocked-change, or apply-failed.

Scripts must read --json. The human report is not a contract.

The audit trail

Every applied schema transaction records the requester, both digests, the observed basis, the tool version, the execution classes, and the acknowledgements. The transaction entity carries them under :db.schemaUpdate/*.

These are ordinary queryable attributes, so the schema history of a database is a Datalog query. An ordinary transaction cannot write them, so a transaction cannot claim that it was a schema update.

What an update cannot do

Partly implemented. Every rewrite change is reported as blocked, and --allow rewrite does not enable it. Resolving cardinality conflicts, copying values to a replacement attribute, and sweeping current facts are jobs that do not exist yet. Do that work through ordinary transactions first, then plan again.

A blocked change stops the whole apply. Resolve it in the database, or remove it from the file, and then apply the rest.

A value-type change in place is destructive and can never run. The plan prints a replacement-attribute recipe instead. Follow it by hand.

  1. Add a new attribute with the wanted type.
  2. Convert the current values, and assert them under the new attribute.
  3. Compare the counts, and record the values that no conversion accepted.
  4. Move the application reads and writes to the new ident.
  5. Retire the old attribute with --prune.

An explicit rename is not specified yet. Excision, which erases historical facts, is separate work with its own approval path. schema update never hard-deletes anything.

Not implemented. A schema update points an attribute at an installed protection class. It cannot install a class. Decide the classes before you create the database.

Inspecting the installed schema

The console prints the schema:

:schema
:schema person/name

The Schema panel of corium tui shows the same table, and it filters with /.

SQL exposes the same data as relations:

SELECT * FROM corium_sys.attributes;

Reserved idents

The engine installs its own attributes first. Attribute entity ids below 100 in the :db.part/db partition are reserved.

:db/txInstant is installed by the engine at the same id that Datomic uses. A schema file that declares it is rejected.

An update allocates a new attribute id above the highest durable id. Database creation keeps its positional allocation, so a database built from one file is reproducible.

Index publication

The transactor republishes the covering indexes in the background. A cold peer then bootstraps from a snapshot instead of replaying the whole log.

Each index is published as content-defined leaf chunks under a small manifest. A publication uploads only the chunks that the changes landed in. Building the snapshot still costs processor time in proportion to the database, and pacing bounds that cost.

Indexing is never a durability requirement

The log append is the commit point. The transactor serves from its in-memory value whatever the index lag is.

Deferred publication has two costs, and no risk.

  • Cold-peer bootstrap gets slower, because the log tail after the published basis is replayed.
  • A backup is less fresh, because it reads through the published basis.

Pacing

FlagDefaultEffect
--index-interval-ms5000Base interval between publications.
--index-backoff4Minimum wait before the next publication, as a multiple n of the duration of the last one. 0 disables it.
--index-tail-threshold0Defer a due publication while fewer than this many datoms are pending. 0 publishes any pending work.
--index-tail-deadline-ms60000Longest that a below-threshold tail defers publication.

The backoff bounds indexing to at most 1/(1+n) of wall-clock time and of storage bandwidth as publications get slower. With the default of 4, indexing takes at most one fifth of the time.

The tail threshold makes small writes coalesce. Without it, a trickle of transactions rewrites the indexes on every interval.

Runtime overrides

All four values can be changed per database while the transactor runs. Omitted flags are unchanged. An override lasts until the process restarts.

corium db index-policy people --interval-ms 60000 --tail-threshold 1000000

Read the current policy back with no flags:

corium db index-policy people

The command prints one EDN map:

{:db "people" :interval-ms 60000 :backoff 4 :tail-threshold 1000000 :tail-deadline-ms 60000}

Publish now

corium db request-index people

This request bypasses pacing entirely. Use it after a bulk load, and before a backup, when the snapshot must be current.

Bulk loading

For a bulk load, follow this procedure.

  1. Raise the tail threshold, for example to one million datoms: corium db index-policy <db> --tail-threshold 1000000.
  2. Run the load. The backoff keeps the indexing duty cycle bounded as the database grows.
  3. Watch :index-lag in corium db stats, or the metrics endpoint.
  4. Run corium db request-index <db> when the load is complete.
  5. Restore the normal policy, or restart the transactor.

Without step 4, the final tail publishes within the tail deadline of the last transaction.

Partly implemented. On the native backends the per-transaction log objects are not yet sealed into chunks. Replay cost and list cost therefore grow with the tail since the last publication. A long deferral on postgres, turso, or s3 makes recovery and cold bootstrap slower than the same deferral on fs.

Watching the lag

Three surfaces report index lag.

  • corium db stats <db> prints :index-basis-t and :index-lag.
  • The Metrics panel of corium tui plots index lag.
  • The metrics endpoint exposes corium_transactor_index_duration_seconds. See monitoring.

A lag that grows without limit means that publication cannot keep up. Lower --index-backoff, or give the transactor faster storage.

Query console

corium console opens an interactive Datalog console. The console is a peer. Every query runs locally in the console process.

corium console people --transactor http://127.0.0.1:4334

Input

The console accepts three kinds of input.

EDN Datalog queries. Enter the query on one line:

[:find ?name ?age :where [?e :person/name ?name] [?e :person/age ?age]]

Pull forms.

(pull [:person/name :person/age] 1000)

Console commands. Every command starts with a colon.

Partly implemented. A console query takes database inputs only. A query with :in parameters other than $ is rejected. Use a client library for a parameterized query.

The console is read-only. There is no transact command. See getting started for the write paths.

Commands

CommandEffect
:basisPrint the basis and the active view.
:as-of <t>Fix the view at transaction <t>.
:as-of <timestamp>Fix the view at a UTC timestamp.
:since <t>Show only facts added after <t>.
:since <timestamp>The same, named by timestamp.
:history onShow every assertion and retraction.
:history offReturn to the current view.
:currentReturn to the current view.
:schemaPrint every attribute.
:schema <attr>Print one attribute, for example :schema person/name.
:statsPrint the basis and the datom, entity, and attribute counts.
:timing onReport time and datoms scanned after each query.
:timing offStop reporting them.
:watchTail live transaction reports until Ctrl-C.
:helpPrint the command list.
:quit, :exitLeave the console.

Timestamps

:as-of and :since accept a transaction number, or a UTC timestamp.

:as-of 10
:as-of 2026-07-25T09:30:00Z
:since 2026-07-25 09:30:00

A timestamp is YYYY-MM-DD, optionally with HH:MM, HH:MM:SS, or HH:MM:SS.mmm. A timestamp selects the last transaction committed at or before it. Resolution reads the :db/txInstant datom that every commit asserts.

The SQL shell accepts the same two forms with \as-of and \since.

Cost of a time view

Partly implemented. A distinct time view costs a fold of the whole history on first read, not a fold of the view. A :history on console session on a large database is slow and uses memory in proportion to total history. See time and database values.

Bootstrap

By default the console replays the log from basis 0. On a large database that is slow.

Add --peer-bootstrap when the console host can reach the storage backend:

corium console people --peer-bootstrap

The console then reads the published snapshot and subscribes from the index basis. The transactor supplies the storage connection details, using the read-only credential that you configured. See storage backends.

An encrypted database also needs --storage-key.

Watching transactions

:watch tails the transaction report stream. Each report prints t, the commit instant, and the datom count. Press Ctrl-C to stop the tail and return to the prompt.

The Transactions panel of corium tui shows the same stream with a datom detail pane.

Terminal dashboard

corium tui opens a full-screen dashboard over one database.

corium tui people --transactor http://127.0.0.1:4334
FlagDefaultEffect
--refresh-ms <n>2000Metrics sample interval. The minimum is 250.

The dashboard also accepts every connection flag.

The process owns the terminal, so it writes no tracing output.

Press Tab to cycle the panels. Press 1 to 4 to jump to one panel, from outside the query editor.

Quit with Ctrl-C anywhere, with q outside the query editor, or with :quit.

Query panel

An editor for EDN Datalog queries, (pull …) forms, and every console command.

Enter runs the form when its brackets balance. Otherwise Enter inserts a newline. Alt-Enter always inserts a newline.

A relation result renders as a scrollable table with :find headers. Every run reports wall-clock time, the datoms scanned, and the basis that it ran against. and recall the query history.

Metrics panel

Data-store statistics, sampled from the transactor Status call on the refresh interval:

  • Basis, index basis, and index lag.
  • Datom, entity, and attribute counts.
  • Commit queue depth, transaction totals, and failure rate.
  • Indexing and garbage collection counters.
  • Lease ownership and the advertised endpoint.

The panel also draws sparklines for transaction frequency, status round-trip latency observed by the peer, and index lag. It reports peer-side query latency as last, average, and maximum.

This panel is the only surface that shows lease ownership without reading the root record directly.

Transactions panel

A live feed from the transaction report subscription of the peer. Each row shows t, the commit time, and the datom count.

A detail pane shows the datoms of the selected transaction. Press f to toggle follow-newest.

Schema panel

The attribute table: ident, value type, cardinality, uniqueness, and the index, component, and history flags.

Press / to filter the table.

When to use the dashboard

Use the dashboard for live observation during a load, a failover test, or an incident. Use the metrics endpoint for recorded monitoring. See monitoring.

SQL shell and PostgreSQL server

Corium executes SQL inside a peer, against immutable database values. SQL does not change the storage model into tables. The relations are a projection.

The SQL dialect is the DataFusion dialect. Wire compatibility with PostgreSQL does not imply dialect compatibility or pg_catalog compatibility.

The relational projection

Attributes are grouped by keyword namespace. Given :artist/name, :artist/country, and :artist/tags, SQL sees:

corium.artist(e BIGINT, name TEXT, country TEXT, tags LIST<TEXT>)

The rules of the projection are:

  • e is the Corium entity id, and the name is reserved.
  • A cardinality-one column is a nullable scalar.
  • A cardinality-many column is a non-null list. An absent attribute is an empty list. Values are unique and ordered deterministically, but the order carries no meaning.
  • One entity can occur in several namespace tables. These are projections, not entity types.
  • An attribute without a namespace is grouped in corium._global.
  • Names are preserved exactly. Use double quotes for a name such as release-group.

Three system relations are available in every view.

RelationContent
corium_sys.datomse, a, attr, typed value columns, tx, t, added.
corium_sys.attributesThe schema.
corium_sys.identsEntity id to keyword ident.

Partly implemented. A history session exposes corium_sys relations only. Wide history tables are reserved for a later validity-interval design.

The SQL shell

corium sql people
corium sql people -c "SELECT * FROM corium.artist LIMIT 10"
corium sql people -f report.sql

An interactive statement ends with a semicolon. Each statement captures a fresh current database value, unless a time view is selected. Ctrl-C drops the running query.

The shell is read-only.

CommandEffect
\as-of <t>Fix later sessions at <t>, or at a UTC timestamp.
\since <t>Use a since view. Timestamps are accepted.
\history onExpose history events.
\history offReturn to the current view.
\currentReturn to the current view.
\basisPrint the basis and the view.
\dtList relations.
\d <table>Print the result columns of a relation.
\timing onReport execution time.
\qQuit.

List functions come from DataFusion:

SELECT e, name FROM corium.artist WHERE array_has(tags, 'ambient');

The shell takes no key flag, so it prints <redacted> for a value on a protected attribute.

The PostgreSQL wire server

corium postgres-server --listen 127.0.0.1:5432

One server exposes the whole database catalog of the transactor. A connection picks its database with the standard startup database parameter. It can switch at any time with USE <database>. SHOW DATABASES lists what is available.

psql 'host=127.0.0.1 port=5432 dbname=people' \
  -c "SELECT e, name FROM corium.person ORDER BY name LIMIT 10"
FlagDefaultEffect
--listen <addr>127.0.0.1:5432Listen address.
--database <name>AllRestrict the exposed set. Repeatable.
--password <secret>NoneRequire this cleartext password. Ignored once authentication is configured.
--allow-writesOffEnable guarded DML.

The server also takes the connection flags, the serving flags, and --storage-key.

Databases are opened lazily and cached. One peer connection is shared by every client that uses that database.

The server supports the simple and the extended query sub-protocols, including $1 bound inputs. Common scalar parameters accept text and binary encodings. Results support both encodings.

Not implemented. Array inputs are not supported on the wire.

Writes through SQL

corium postgres-server is read-only by default. --allow-writes enables a narrow DML subset.

corium postgres-server --listen 127.0.0.1:5432 --allow-writes

In autocommit each statement is one transaction. An expected-basis fence rejects a stale read-modify-write plan before it commits.

  • Only existing corium.<namespace> projections are writable. corium_sys, the time views, DDL, and schema changes are read-only.
  • INSERT requires an explicit column list. It supports VALUES or a query source. Omit e for a tempid. An explicit e must not already occur in that projection. A NULL input omits the attribute.
  • UPDATE supports one plain target table, predicates, expressions, and RETURNING. Assigning NULL clears a cardinality-one attribute. Assigning ARRAY[...] replaces the whole cardinality-many set.
  • DELETE supports one plain target table, predicates, and RETURNING. It retracts every attribute in the target namespace, and it preserves attributes of other namespaces on the same entity.
  • RETURNING works for all three. Delete rows come from the pre-commit snapshot. Insert and update rows come from the committed value.

Not implemented. Joined and multi-table mutations, conflict clauses and upserts, ordered or limited mutations, new keyword interning, and DDL are deferred.

Explicit transactions

An explicit BEGIN block pins the database value of its first statement. DML is staged against a provisional value, so a later statement in the block reads what the earlier ones wrote.

ROLLBACK discards the staged forms. COMMIT submits them as one atomic Corium transaction. A concurrent basis change fails the commit with SQLSTATE 40001, which a client reads as a serialization failure and retries.

SET, RESET, and DISCARD are compatibility no-ops.

Object-relational mappers

The server answers the PgJDBC metadata probes for SQL keywords, current schema and catalog, and transaction isolation. Hibernate therefore selects its PostgreSQL dialect on its own.

The runnable postgres-hibernate example exercises Hibernate ORM 7.4 with PgJDBC 42.7. It inserts with a generated id, reads, updates, runs an HQL query, and deletes. Every step uses an ordinary Hibernate transaction.

Not implemented. Broader pg_catalog introspection, DDL-based schema management, savepoints, COPY, and sequences are absent. Declare the schema with corium schema update rather than with the schema tool of the mapper.

Security of the wire server

CAUTION: The PostgreSQL wire server does not terminate TLS. It rejects --tls-cert and --tls-key rather than accept flags it cannot honor. Put a TLS-terminating proxy in front of it, or bind it to loopback.

Restrict the exposed set with --database when only some databases must be reachable.

A SQL client is a Corium principal

Set any of --serve-token, --oidc-*, or --authz-db, and the server authenticates each client for itself.

PostgreSQL has no bearer-token field, so the password field carries the token of the caller. The startup user is informational.

corium postgres-server --listen 127.0.0.1:5432 \
  --oidc-issuer https://issuer.example --oidc-audience corium \
  --authz-db corium_authz

psql "host=127.0.0.1 port=5432 dbname=people user=alice password=$JWT"

CAUTION: The token crosses the wire in the clear. The server prints this warning at startup whenever authentication is configured.

Every statement is then authorized as that principal. SELECT needs query. DML needs transact. SHOW DATABASES lists only what the principal can inspect.

Reads are answered through the view of the principal and through its own protection class keys. A column that the policy hides keeps its declared type, reports NULL, and never takes a pushed-down predicate. A principal whose view hides attributes cannot write. Read authorization and attribute protection.

--password still applies when no authentication flag is set. It is one shared secret and it maps to no principal.

Partly implemented. A write still commits through the peer connection of the server, so the transactor additionally applies the bearer principal of that connection. Give that connection an identity that can transact every database the server exposes.

Peer server and thin clients

A peer server is a peer hosted as a standalone process. It exposes query, pull, transact, datom scans, and transaction ranges over gRPC.

Use it for a language that has no peer library. For a language with a peer library, embed the peer instead. An embedded peer queries in-process.

Start a peer server

corium peer-server --db people --listen 0.0.0.0:4336 \
  --transactor http://127.0.0.1:4334

One process hosts one database. Run one process per database.

FlagDefaultEffect
--db <name>None. Required.Database to host.
--listen <addr>127.0.0.1:4336gRPC listen address.
--max-fuel <n>10000000Ceiling on datoms touched per query.
--metrics-listen <addr>NonePrometheus endpoint at /metrics.

The peer server takes the same connection flags as the other client commands, and the same serving flags as the transactor.

Query fuel

Fuel bounds a runaway query. A client can request less fuel. fuel = 0 requests the server default. The server clamps every request to --max-fuel.

An exhausted budget returns INVALID_ARGUMENT.

Storage bootstrap

By default the peer server replays the log from basis 0 at startup. On a large database that is slow.

corium peer-server --db people --peer-bootstrap \
  --storage-key file:/etc/corium/storage.key

--peer-bootstrap reads the published snapshot from storage and subscribes from the index basis. The peer needs network reach to the storage backend, and a build with the matching storage feature.

The transactor supplies the connection details through its GetStorageInfo call, using the read-only credential that you configured. See storage backends.

Segment cache

A peer server can keep a local SSD cache of segments.

FlagDefaultEffect
--segment-cache-dir <path>NoneDedicated directory for the cache.
--segment-cache-capacity <size>None. Required with the directory.Disk capacity, for example 256GiB.
--segment-cache-memory <size>64MiB, or the capacity when smallerMemory front tier.

A size accepts the suffixes B, KiB, MiB, GiB, TiB, kB, MB, GB, and TB.

The cache requires --peer-bootstrap. Without it, the process fails at startup with a clear message.

Give the cache a dedicated directory. Corium manages the contents.

Failover

Pass every transactor endpoint, active first:

corium peer-server --db people \
  --transactor http://txor-a:4334,http://txor-b:4334

The peer rotates the list on failure. A standby rejects subscriptions with a standby status, which the peer treats as a reason to try the next endpoint. See high availability.

The thin-client contract

The wire contract is documented in thin-client-protocol.md. The canonical schema is crates/corium-protocol/proto/corium.proto.

Six rules matter to an operator.

  • Every transact and subscribe request sends a protocol_version. This build speaks version 3 and accepts version 1 and later. An unsupported version is FAILED_PRECONDITION, never a silent downgrade.
  • Malformed input is INVALID_ARGUMENT. An unknown database or entity is NOT_FOUND. Upstream loss is UNAVAILABLE.
  • Query results stream in chunks. A client concatenates them and stops at last = true.
  • Subscribe.from_basis_t is exclusive. The server backfills every later transaction without gaps, then continues live.
  • The subscription handshake advertises the heartbeat interval. A client treats silence for a few multiples of it as a dead upstream.
  • Transact gives read-your-writes on the serving peer before it responds.

A client is conformant when it reproduces the behavioral corpus in tests/conformance.

Version 3 adds the sealed value tag. A client older than version 3 never receives a sealed value. An unopened value reaches it as the tagged EDN element #corium/redacted, which every EDN reader parses.

Serving many principals

A peer server serves every client from one process and one database value. Which facts a client sees is a policy question, not a process question.

A principal whose view hides attributes cannot Transact, and it cannot Subscribe. Transaction data is opaque bytes by the time authorization runs. A subscription proxies the stream of the transactor. Neither call can honor a filter, so the server refuses rather than serve unfiltered data.

Language clients

ClientLocation
Rustcorium-peer, corium-client
Pythonclients/python
Javaclients/java
Clojurecorium-cljrs, the corium.api namespace

The Python and Java clients each offer two peers behind one interface.

PeerWhere it runs
LocalPeerEmbeds a full peer in the process. It indexes and queries in process, and it talks to a transactor directly.
RemotePeerConnects to a peer server over gRPC.

Both produce the same database values, and every time view works the same way on both. Only a remote peer can join across databases in one query.

An embedded peer can also read published segments straight from storage, and it can hold its own class keys. Use it where the keys must stay in the application process.

Authentication and TLS

Every network surface establishes a request-scoped principal. Authentication answers who is calling. Authorization answers what that caller can do.

The default is permissive

A server with no authentication flags does two things.

  • It recognizes the shared development token, and it gives that caller the identity operator, vouched for by the provider static-token, with the role admin.
  • It also admits an anonymous caller.

This default makes a local database usable with no flags. It is not safe on a shared network.

CAUTION: Never expose a default-configured transactor or peer server outside a trusted network. The shared development token is a compiled-in constant, and anonymous callers are admitted.

Strict mode

Any of three flags switches a server to strict mode. In strict mode an absent or unrecognized credential is rejected.

FlagEffect
--serve-token <secret>Require this exact bearer token. It replaces the development token.
--require-authRequire the development token, or --serve-token. Reject anonymous callers.
--oidc-issuer <url>Accept tokens signed by this issuer, and the static token.

--serve-open goes the other way. It disables authentication completely, and every request arrives as anonymous. It conflicts with --serve-token, --require-auth, --oidc-issuer, and --authz-db.

Read the secret from CORIUM_SERVE_TOKEN rather than from a process argument:

CORIUM_SERVE_TOKEN=$(cat /etc/corium/serve.token) \
  corium transactor --data-dir /srv/corium --require-auth

Client tokens

A client sends its token with --token, or with CORIUM_TOKEN.

CORIUM_TOKEN=$(cat /etc/corium/serve.token) corium db list

--token "" connects anonymously. With no flag and no variable, the client sends the shared development token.

OIDC

--oidc-issuer <url> accepts bearer tokens signed by an OIDC issuer. The static token keeps working alongside it.

FlagEffect
--oidc-issuer <url>Issuer URL.
--oidc-audience <aud>Accepted audience. Repeatable. Set it.
--oidc-jwks-file <path>Read the JWKS from a file instead of fetching it.

Two Cargo features apply. oidc verifies against a JWKS file. oidc-discovery also fetches the JWKS from the issuer over HTTP. A binary without the feature rejects the flags at startup and names the feature.

Set at least one audience. Without it, a token minted for another service of the same issuer is accepted.

TLS

A server terminates TLS when both certificate flags are present:

corium transactor --data-dir /srv/corium \
  --tls-cert /etc/corium/tls/server.pem \
  --tls-key /etc/corium/tls/server.key

A client enables TLS by naming a CA, a domain, or both:

corium db list --transactor https://txor-a:4334 \
  --ca /etc/corium/tls/ca.pem --tls-domain txor-a.internal

Three surfaces have no TLS of their own.

  • The metrics endpoint. Keep it on a private operations network.
  • The PostgreSQL wire server. Put a proxy in front of it.
  • The storage backends use their own transport security. PostgreSQL uses sslmode in the URL. S3 uses HTTPS.
DeploymentSettings
Laptop, single userNo flags.
Shared development host--require-auth, or --serve-token.
Production, machine clients--serve-token, TLS, --authz-db.
Production, human identities--oidc-issuer with --oidc-audience, TLS, --authz-db.

Authentication alone permits every action. Add authorization to restrict what a principal can do.

Authorization

Servers authorize every request permit-all by default. --authz-db <name> switches them to a relationship policy that is stored in an ordinary Corium database.

The model is relationship-based, in the style of Google Zanzibar and OpenFGA. A decision is a bounded, cycle-safe walk over relationship tuples.

The policy database is an ordinary database. Backup, restore, fork, as-of, and the log API all work on it.

Bootstrap

Bootstrap is two steps, in this order.

Step 1. Against a transactor started without --authz-db:

corium authz init --admin alice --provider oidc
corium authz grant 'group:eng#member' writer database:music
corium authz grant bob member group:eng
corium authz check bob transact --database music

authz init creates the database corium_authz, installs the reserved schema, installs the default permissions, and grants the first administrator owner on catalog:* and on database:*.

Step 2. Restart the surfaces with enforcement on:

corium transactor  --data-dir /srv/corium --authz-db corium_authz
corium peer-server --db music --authz-db corium_authz

The default administrator

authz init defaults its administrator to operator, pinned to the provider static-token. That is the identity a token client presents, so the CLI keeps working after enforcement is on.

FlagDefaultEffect
--db <name>corium_authzPolicy database name.
--admin <id>operatorSubject id of the first administrator.
--provider <name>static-tokenProvider that must vouch for the administrator. any accepts every provider.
--no-adminOffInstall schema and permissions only. Grant nobody anything.

Subjects, relations, and objects

A tuple says subject relation object.

corium authz grant alice writer database:music
corium authz revoke alice writer database:music
PositionForms
Subjectuser:alice, group:eng, role:ops, or the userset group:eng#member. A bare name reads as user:<name>.
RelationA name, for example owner, writer, viewer, member, parent.
Objectdatabase:music, tenant:acme, catalog:*, database:*.

Relation names are data, not built-in values. The permissions decide which relation satisfies which action.

Actions

Fifteen actions exist. Each belongs to one class.

ClassActions
Readquery, pull, datoms, tx-range, subscribe, inspect, list-databases
Writetransact
Admincreate-database, delete-database, fork-database, garbage-collect, manage-index, manage-keys, alter-schema

alter-schema is deliberately separate from transact. An application writer that can add facts must not be able to broaden the vocabulary that it writes them under. The action is admin-class and database-scoped, so the default permissions give it to a database owner only. Read schema management.

The default permissions that authz init installs bind classes to relations.

Object typeClassRelations that satisfy it
databasereadviewer, writer, owner
databasewritewriter, owner
databaseadminowner
catalogreadviewer, owner
catalogwriteowner
catalogadminowner

A permission entity can also name one action instead of a class, and * matches any action or any object type.

Testing a decision

corium authz check bob transact --database music

The command runs the same evaluator that a server runs, and it prints the matched path. Use it before and after a change.

FlagEffect
--database <name>Target database. Omit it for catalog-wide actions.
--provider <name>Provider that vouched for the subject. Defaults to oidc.
--role <name>A role that the credentials of the principal assert. Repeatable.
--claim <key>=<value>A claim that the principal carries. Repeatable.

Status

corium authz status

The command prints the compiled basis, :authz-t, and the entity counts.

Operating notes

Fail closed. A surface that cannot read or compile the policy denies every request. It does not refuse to start. It logs the remedy and recovers on its own when the database appears, so an ordering mistake is not fatal.

Changes propagate without a restart. Each server watches the policy database and recompiles off the request path. A grant takes effect in milliseconds.

Every decision is logged with its basis. The tracing target is corium_authz::audit. Denials log at info. Grants log at debug.

--authz-fresh-writes makes write and admin actions re-read the policy before they decide. The cost is one snapshot read per such request. Reads keep using the pinned snapshot.

--authz-max-depth <n> bounds the relation hops of one check. The default is 8.

--authz-break-glass-role <role> admits a role while the policy is unreadable. It never overrides a deny.

Recovering from a lockout

A policy that denies everyone cannot be repaired through the policy.

  1. Stop the transactor.
  2. Start it again without --authz-db.
  3. Fix the tuples with corium authz grant.
  4. Stop it, and start it again with --authz-db.

Break-glass does not help here, because the policy is readable. It denies.

Interaction with authentication

--authz-db conflicts with --serve-open, because that flag authorizes requests whose identity was never established.

An anonymous caller is still admitted in permissive mode. With an authorizer in place, that caller arrives as user:anonymous. Public read with authenticated write is therefore a policy question, not a flag.

Views: attribute and key filtering

A view narrows what a successful read returns. A binding attaches a view to one relation on one object.

;; Hide every attribute outside the allowlist.
{:authz.view/name "support"
 :authz.view/filter-type "attribute-allowlist"
 :authz.view/attribute [":person/name" ":person/city"]}

{:authz.binding/relation "support"
 :authz.binding/object "database:people"
 :authz.binding/view "support"}
AttributeMeaning
:authz.view/nameName a binding refers to. Unique.
:authz.view/filter-typeattribute-allowlist or attribute-denylist. Optional.
:authz.view/attributeAttribute idents the filter names. Repeatable.
:authz.view/keyProtection class key ids the view permits. Repeatable.
:authz.binding/relationRelation the view attaches to.
:authz.binding/objectObject the view attaches to. type:* is allowed.
:authz.binding/viewName of the view to apply.
:authz.binding/unfilteredMarks the relation as granting full attribute visibility.

A view can name attributes, key ids, or both. :authz.view/filter-type is needed only when the view names attributes.

Hidden means hidden

A datom on a hidden attribute is dropped inside the scan. It never binds, it never joins, and it never satisfies a predicate.

Every path that reaches an index directly is closed with it. A pull omits the key. SQL reports NULL and refuses to push a predicate down that column. get-else falls to its default. missing? reports missing. A lookup reference does not resolve. Reverse-reference traversal returns nothing.

Each of those is an existence test over the attribute that the view withholds.

Combining views

When several relations succeed, their views intersect. Holding one more relation can never reveal more than holding it alone.

:authz.binding/unfiltered is the escape. Use it for a relation such as owner that must see everything.

CAUTION: :authz.binding/unfiltered grants attributes and no keys. Keys are named by key id, and that binding names none. A relation that must read protected values names those key ids on a view.

Key grants

:authz.view/key names the protection class keys that a principal can use. Read attribute protection, which states the key policy modes and the strict default.

Surfaces that cannot filter refuse

A principal whose view hides attributes cannot write on any surface. The peer server refuses Transact and Subscribe. The transactor refuses a write. SQL refuses DML.

Transaction data is opaque bytes by the time authorization runs, and Subscribe proxies the stream of the transactor, so neither can honor a filter. All four ask the same question, so one principal under one policy gets one answer whichever surface it reached.

A view that names every attribute takes nothing away, so it is not treated as hiding. A view that restricts only keys is refused nowhere.

Protecting the policy database

The policy database is governed by the policy that it holds. The database:* ownership of the administrator is what keeps corium authz grant working.

Back it up like any other database. Read backup and restore.

Encryption at rest

Every durable artifact of an encrypted database is sealed. Index blobs, transaction-log record payloads, and cached segments are all covered.

The seal uses a per-database data key. That data key is itself wrapped by a key-encryption key, written KEK, that Corium never stores.

Encryption is fixed at creation

corium db create people --schema schema.toml --storage-key file:/etc/corium/storage.key

A database created without a storage key stays unencrypted forever. A database created with one stays encrypted forever.

There is no in-place migration. Migrating an unencrypted database means a backup and a restore into a new database.

Partly implemented. corium backup refuses an encrypted database. Backup format 1 cannot carry the key manifest, so no restore can open the resulting archive. An encrypted database therefore has no supported backup path today. Protect it with storage-level replication and snapshots until backup format 2 lands.

Key identities

A key identity is a URI.

SchemeResolvesContent
file:<path>Yes32 raw bytes, or 64 hexadecimal characters.
env:<NAME>YesThe same two forms, from an environment variable.
awskms:, gcpkms:, vault:NoRecognized and rejected as unsupported.

Surrounding whitespace is ignored, so a key file with a trailing newline works.

Not implemented. KMS key identities are recognized, but no keyring resolves them. A process that names one fails at startup. Use file: or env: today.

Create a key:

head -c 32 /dev/urandom > /etc/corium/storage.key
chmod 400 /etc/corium/storage.key

CAUTION: Corium never stores the KEK. If you lose it, the database cannot be read again. Keep the key off the machine that holds the data, and keep a copy in a separate system.

Which processes need the key

Every process that reads storage directly needs the key.

corium transactor  --data-dir /srv/corium --storage-key file:/etc/corium/storage.key
corium peer-server --db people --peer-bootstrap --storage-key file:/etc/corium/storage.key

--storage-key is repeatable, because one node can host databases under different KEKs. CORIUM_STORAGE_KEY sets it from the environment.

A process resolves every named key at startup. A process without a key that its database needs therefore fails at open, and the error names the key. It does not fail later at its first read.

Thin clients and peer-server callers need no key. They receive plaintext over TLS.

corium db create --storage-key names the key for the transactor to resolve. No key material leaves the transactor host.

Offline commands

Two offline commands read blob and log content, so they need the key.

corium log --data-dir /srv/corium --db people --storage-key file:/etc/corium/storage.key
corium gc  --data-dir /srv/corium --storage-key file:/etc/corium/storage.key

Offline garbage collection refuses to run without the key. Without the key, the command cannot follow the index chunks, and a sweep deletes them.

Inspecting keys

corium keys status people

The command prints the KEK, the storage-key epochs, their states, and the share of the nonce budget that each epoch has spent.

FieldMeaning
:encryptedWhether the database is encrypted at all.
:kekThe key-encryption key that the manifest names.
:rotation-duetrue when the active epoch has spent half its nonce budget.
:keys-unavailableThis node cannot load a manifest change.
:keys-fencedThis node cannot load the epoch that the manifest opened.
:storage-keysOne map per epoch: state, algorithm, KEK epoch, opening t, records sealed, budget used, live objects.

Rotation

corium keys rotate people

Rotation opens a new storage-key epoch. New writes use it at once. It rewrites no stored object.

An older epoch stays readable. It drains as ordinary re-indexing rewrites its objects. An epoch retires only when no live object carries it.

Rotate when corium keys status reports :rotation-due true. That fires at half the log-record nonce budget. A log record uses a random 96-bit nonce, so an epoch must seal well under 2³² records. That count is the span of t that the epoch covers.

Re-wrapping

corium keys rewrap people --kek file:/etc/corium/storage-2026.key

Re-wrapping re-encrypts the data keys under a new KEK. It reads, rewrites, and re-encrypts no stored object.

The transactor must resolve both KEKs at once. Follow this procedure.

  1. Start the transactor with both --storage-key flags.
  2. Run corium keys rewrap <db> --kek <new>.
  3. Confirm the new KEK with corium keys status <db>.
  4. Restart the transactor with the new key only.

When a node cannot load a key change

A key change made elsewhere is picked up within one lease-renewal tick. When that load fails, the effect depends on which change it was.

StateCauseEffect
:keys-unavailable trueThe manifest changed, and this node cannot load it. Usually a re-wrap to a KEK it cannot resolve.Warning only. Reads and writes continue under the keys it already holds. The corium_keys_unavailable gauge rises.
:keys-fenced trueThe manifest opened an epoch that this node cannot load.Writes refuse with FAILED_PRECONDITION, naming both epochs. Reads, index publication, and the lease continue.

The difference is deliberate. A re-wrap leaves the data keys unchanged. A write refusal therefore turns a key-service outage into a write outage for no confidentiality gain. A rotation is different in kind: records sealed under a closed epoch draw on a budget that has stopped counting them.

Both states clear as soon as a load succeeds. The fix is the same for both.

  1. Give the process a --storage-key that resolves the KEK that the manifest now names.
  2. Restart the process.

Only the fenced state stops writes while you do this.

The second layer

Storage encryption protects the medium. It does not protect a fact from a reader that Corium serves.

The second layer is attribute protection. Values on a protected attribute are sealed by the writing peer under a class key. Only a process whose keyring resolves that key sees them in the clear.

The two layers are independent. A database can use either, both, or neither. --storage-key supplies the keys of both, because one process keyring holds key-encryption keys and class keys alike.

Attribute protection

Encryption at rest protects the medium. Attribute protection protects facts from readers.

Values on a protected attribute are sealed by the writing peer under the key of a protection class. Only a process whose keyring resolves that key sees them in the clear. The transactor does not. A peer without the key does not. An operator with storage credentials does not.

Declaring a class

A protection class names a key. It never holds key material. Declare classes in the schema file that creates the database.

[protect.pii]
key = "file:/etc/corium/pii.key"
padding = 64
on-missing-key = "redact"

[[entity]]
name = "person"

[entity.attributes]
name = "string"
ssn  = { type = "string", protection = "protect/pii" }

A [protect.<name>] section declares the class :protect/<name>.

OptionValuesDefault
keyKey identity, for example "file:/etc/corium/pii.key"Required
algorithm"aes-256-gcm-siv""aes-256-gcm-siv"
scope"attribute" or "entity""attribute"
paddingBytes to round the plaintext up to. At least 16.Unset
on-missing-key"redact", "hide", or "error""redact"
legacy-plaintext"redact" or "pass-through""redact"
epochKey epoch that new values seal under1

The EDN form declares the same class as an entity with :db.protect/* attributes, and points an attribute at it with :db/protection.

Not implemented. A schema update points an attribute at an installed class. It cannot install a class. Decide the classes before you create the database.

Partly implemented. scope = "entity" parses and stores, and a writing peer refuses to seal under it. Use the default attribute scope today.

What each option costs

scope decides what the sealing determinism leaks. Sealing must be deterministic, because the transactor compares values as bytes and holds no key. Under "attribute" a reader without the key can tell that two entities share a value on that attribute. Under "entity" the seal also binds the entity, so it leaks only the repeated value of that one entity.

padding rounds the plaintext up to a multiple of that many bytes before sealing. It costs storage and removes the length side channel for short, guessable values.

on-missing-key decides what a reader who cannot open a value gets.

PolicyResult
redactThe value binds in redacted form. EDN prints #corium/redacted. SQL prints NULL.
hideThe datom is dropped from every scan. The entity leaves the join.
errorThe read fails.

Under all three, an unopenable value never satisfies a constant and never satisfies a predicate. It binds, it disappears, or it raises. It never matches by accident.

legacy-plaintext decides the same question for a value that was written before the attribute became protected.

Protection is not free

A protected attribute cannot also carry :db/index, :db/unique, or :db.type/ref. Ciphertext order is not value order, so a protected value can never appear in a value-ordered index. The schema rejects the combination rather than surprising a range query later.

An attribute that has ever been protected can never gain :db/index or :db/unique, even after it is unprotected again.

Where an application needs an indexed lookup on a protected field, add a second unprotected attribute that holds a keyed hash of the value. That leak is then explicit.

Which processes need class keys

--storage-key is the keyring of the process. It resolves key-encryption keys and protection class keys alike. Give a class key to the processes that must read the class, and to no others.

corium peer-server --db people \
  --storage-key file:/etc/corium/storage.key \
  --storage-key file:/etc/corium/pii.key

A process with no class keys is a fully working peer. It commits, it indexes, it syncs, and it answers every query that touches no protected value identically. Protected values come back redacted, hidden, or refused, as the class policy says.

corium console, corium sql, and corium tui take no key flag. They print protected values in redacted form.

Serving many principals from one process

A peer server and a PostgreSQL wire server serve many principals from one process. Which of the keys of that process a request can use is a policy question.

Name the key ids on a view, and bind the view to the relation that can read them:

{:authz.view/name "pii-reader" :authz.view/key ["file:/etc/corium/pii.key"]}
{:authz.binding/relation "hr" :authz.binding/object "database:people"
 :authz.binding/view "pii-reader"}

A guarded server defaults to the strict key policy. A principal whose decision names no key id hydrates nothing.

ModeA decision that names no key idDefault when
strictGrants no class key.Authentication is configured.
server-wideGrants the whole keyring of the process.Authentication is off.

--key-policy strict and --key-policy server-wide override the default on peer-server and on postgres-server.

Three rules matter.

  • Key grants combine across successful paths by intersection. One more relation can never widen a key set.
  • Granting a key id that the process does not hold does nothing. Policy narrows the keyring of the process. It never extends it.
  • :authz.binding/unfiltered grants full attribute visibility and no keys. Keys are named by key id, and that binding names none. A relation that must read protected values names them.

CAUTION: This is authorization, not cryptography. A key-holding server still holds the plaintext and is choosing not to disclose it. A compromised or misconfigured server defeats it.

For a genuinely less-trusted deployment, run the server with no class keys. Let each entitled application embed a peer with its own keyring, so the keys stay in the process that owns them.

Not implemented. Seal-through mode, in which the server forwards sealed values and the thin client opens them itself, does not exist. A key set is always resolved in the server.

Upgrading a guarded server that holds keys

Turning authorization on changes what a key-holding server discloses. Under the derived strict default, a principal whose policy names no key id stops seeing protected plaintext and starts seeing redactions.

That is the safe direction, and it is loud. Plan the change.

  1. List the relations that must read each class.
  2. Create one view per class, naming its key ids on :authz.view/key.
  3. Bind each view to its relation with :authz.binding/*.
  4. Restart the server with --authz-db.

To keep the old behavior while you write those grants, pass --key-policy server-wide deliberately.

Changing protection

Protection is not fixed at database creation. corium schema update can protect an attribute, unprotect it, or move it to another class.

The change is forward-only. Values written from that basis onward take the new form. Every value already stored keeps the form it had.

  • Protecting an attribute does not seal the plaintext already stored.
  • Unprotecting an attribute does not open the ciphertext already stored.

The plan reports both consequences and requires --ack protection-forward-only. It also reports that lookup references and value-ordered reads through the attribute stop working from that basis on.

Not implemented. Sweeping the current values into the new form is a rewrite job that does not exist. There is no corium keys protect, corium keys unprotect, or corium keys audit. Class key rotation and crypto-shredding commands are also absent.

What a protected value looks like on the wire

A sealed value needs thin-client protocol version 3. An older client never receives one. Values reach a thin client as boundary EDN, which renders an unopened value as #corium/redacted, a tagged element that every EDN reader parses.

High availability

One transactor holds the write lease per database. A warm standby polls the lease and takes over when it lapses.

This is an active and standby pair. It is not a consensus protocol. The root store is the single arbiter.

Starting a pair

Start both members identically, each with its own identity and endpoint:

corium transactor --data-dir /srv/corium --ha \
  --owner txor-a --advertise http://txor-a:4334 --listen 0.0.0.0:4334
corium transactor --data-dir /srv/corium --ha \
  --owner txor-b --advertise http://txor-b:4334 --listen 0.0.0.0:4334

Whichever starts first becomes active. The other stands by.

The standby rescans the catalog every lease-renewal interval, so a database created on the active is picked up. It rejects client work with a standby FAILED_PRECONDITION that names the current lease holder.

Give each member a stable --owner. A restarted member then re-acquires its own unexpired lease at once.

Storage requirements

Both members must see the same blob store, root store, and log.

StoreShared storage for a pair
fsNeeds a shared filesystem for <data-dir> on both members.
postgres, s3Shared by construction. No shared filesystem is needed.
mem, tursoNot usable for a pair.

CAUTION: Never run two members against diverged copies of a data directory. The store is the source of truth.

Peer failover

Peers list both endpoints and fail over automatically:

corium peer-server --db people \
  --transactor http://txor-a:4334,http://txor-b:4334

A library peer passes the same list through ConnectConfig::with_failover. A peer with storage credentials can also rediscover the advertised endpoint of the current holder from the database root.

Guarantees

  • Takeover is ordinary crash recovery. The standby acquires the lapsed lease, which atomically fences the deposed writer, replays the log tail, and serves.
  • No acknowledged transaction is ever lost or duplicated. A post-append ownership check runs before every acknowledgement.
  • A deposed transactor can never publish. Every root write is a compare-and-set on the record that holds the lease.
  • Peer subscriptions reconnect and backfill without gaps.
  • A deposed member returns to standby on its own. No operator action is needed after a garbage collection pause or a partition.

Unavailability window

Writes are unavailable from the crash until takeover. The bound is the sum of three terms.

  1. One lease time-to-live, because the last renewal of the active must expire.
  2. One standby poll interval, which is one third of the time-to-live.
  3. Reconnect backoff on the peer.

With the defaults that is about 6.7 seconds plus backoff.

Ambiguous transactions

A transact call that fails before it reaches the commit point is retried transparently within failover_timeout. Standby rejection and connection refusal are both in this class.

A call whose connection died mid-request is ambiguous. The transaction is committed, or it is absent. The call surfaces an error, exactly like a transactor crash between durability and reply.

On an ambiguous error, run sync and read before you resubmit. A blind retry can write the data twice.

Tuning

KnobDefaultEffect
--lease-ttl-ms5000Failover detection bound. Renewals run at one third of it.
--heartbeat-ms10000Subscription heartbeats. A peer presumes the transactor dead after 3 missed intervals.
Peer reconnect_min / reconnect_max100 ms / 5 sReconnect backoff while endpoints rotate.
Peer failover_timeout30 sHow long a safe-to-retry transact failure rides out a takeover.

A lower lease time-to-live gives faster takeover. It also costs more root-store traffic, and it tolerates shorter garbage collection or input and output pauses on the active.

Keep the heartbeat interval at or below the lease time-to-live. The heartbeat is what detects a partition that TCP has not noticed.

Clock skew between members shifts detection latency only. It never affects safety, because the root store is the single arbiter.

Runbooks

The runbooks chapter holds the procedures for planned failover, a crashed active, split-brain suspicion, and both members down.

Fleet topology

Not implemented. A fleet topology is specified. The design distributes databases across overlapping candidate sets and gives clients one load-balanced address, with the same lease and failover guarantees. It does not change the commands in this chapter. See transactor-fleet.md.

Backup and restore

Backup is online. Restore is offline.

How backup works

The backup contacts the running transactor once. That call fixes the current transaction basis, and it returns the connection details of the underlying storage.

The backup then reads the storage log independently, through that basis only. Transactions committed while the backup runs are left for the next incremental run.

corium backup --transactor http://127.0.0.1:4334 people /backups/people.corium
FlagDefaultEffect
--transactor <url>http://127.0.0.1:4334Transactor used for storage discovery.
--token <secret>Development tokenBearer token. --token "" connects anonymously.
--ca <pem>, --tls-domain <name>NoneTLS for the transactor connection.

The positional arguments are the database name and the destination file.

Incremental backup

Run the same command with the same file for an incremental refresh.

The backup reads only the transaction records after its existing checkpoint, and it appends one new checkpoint frame. The report prints :replayed-transactions.

The first run embeds the immutable snapshot blobs and retains that index snapshot as a replay base. Later runs do not repeat that work.

The report is one EDN map:

{:db "people" :backup-format 1 :writer-version "…" :basis-t 1240
 :index-basis-t 1200 :replayed-transactions 40 :copied-blobs 12 :reused-blobs 480}

The archive

A backup has exactly one representation: a binary .corium archive.

The header carries an independent backup-file format version, and the Corium version that created it. Every incremental checkpoint records the version that appended it.

An unsupported future format fails before restore, and the error names its writer.

--log-format human|json controls diagnostic logging only. It never changes the artifact.

Not implemented. There is no dump command. Human, JSON, and EDN export belong in one, not in backup or restore.

Where a backup can run

StoreRequirement
fs, tursoRun where the absolute local storage path of the transactor is reachable.
postgres, s3Connect to the same native storage that the transactor advertises. S3 credentials come from the standard AWS environment.
memRejected. A separate process cannot open process-local memory storage.

The advertised PostgreSQL connection is read and write in this version. A future release can substitute read-only credentials without a protocol change.

Partly implemented. corium backup refuses an encrypted database. Backup format 1 cannot carry the key manifest. See encryption at rest.

Restore

Restore is offline, and it refuses to overwrite a database. The target transactor must be stopped.

corium restore /backups/people.corium --data-dir /srv/corium-restored --as-db people

Restoring under a new name creates a clone:

corium restore /backups/people.corium --data-dir /srv/corium --as-db people-staging
FlagEffect
--data-dir <path>Target transactor data directory. Required.
--as-db <name>Target database name. It can differ from the source name. Required.

Restore writes a filesystem data directory. It does not write to a postgres, turso, or s3 store directly.

Backup-container and database-storage versions are checked separately before publication.

After a restore

  1. Start the target transactor on the restored data directory.
  2. Wait until :index-lag in corium db stats reaches zero.
  3. Compare the basis with :basis-t in the backup report.
  4. Compare datom, entity, and attribute counts.
  5. Run a known query and compare the result.
  6. Redirect peers only after those checks pass.

Backup policy

Three rules make a backup useful.

  • Run corium db request-index <db> before a backup when the snapshot must be current. A backup reads through the published basis.
  • Keep the first full archive and its incremental chain together. An incremental run needs the checkpoint in the same file.
  • Test a restore on a schedule. An untested backup is not a backup.

For a database that corium backup refuses, back up the underlying storage instead. See the runbooks.

Forking a database

A fork creates a new database that duplicates an existing one at a transaction basis. The result is a sandbox wound back to a point in time.

Use a fork to debug against real data, or to try an alternative approach, without touching the original.

Unlike backup and restore, forking is online. It runs against the live transactor through the catalog service.

Commands

corium db fork people people-debug --as-of 1234
corium db fork people people-scratch

Without --as-of, the fork is taken at the current basis.

The command prints one EDN map:

{:db "people-debug" :forked-from "people" :basis-t 1234 :created true}

What a fork copies

The fork copies only the transaction-log prefix through the requested basis. Every t up to the basis of the source names a transaction, so any value in range is exact.

Schema metadata is shared. Index segments deduplicate by content address in the blob store, so a fork is cheap in storage.

The new database replays that prefix, publishes its own indexes, and then transacts completely independently of its source.

Rules

  • A basis ahead of the source is refused.
  • An existing target is never overwritten. The command prints :created false, and nothing is changed.
  • The fork is a full database. It accepts writes, it needs its own backups, and garbage collection covers it.

When not to fork

A read-only view of a point in time does not need a fork. A peer gets one locally with as-of, at no storage cost.

Fork only when the sandbox must accept writes.

Cleaning up

Delete a finished fork like any other database:

corium db delete people-debug

Blobs shared with the source stay reachable from the source root. Garbage collection sweeps only what no live root reaches. See garbage collection.

Garbage collection

Old index roots keep old segments alive until no reader needs them. Garbage collection deletes the segments that no live root reaches.

Collection is epoch-based, and it is never urgent.

The retention rule

A sweep deletes an unreachable blob only when the blob is older than the retention window. The window must cover any reader that can still hold a stale root.

The default window is 72 hours.

Scheduled collection

The transactor runs collection on a schedule.

FlagDefaultEffect
--gc-interval <duration>1hInterval between sweeps. off disables the duty.
--gc-window <duration>72hRetention window.

Collection is serialized with index publication. The two never run at the same time.

Manual collection

Online collection asks a running transactor to sweep:

corium gc --transactor http://127.0.0.1:4334 --window 72h

Offline collection reads a data directory directly. The transactor must be stopped:

corium gc --data-dir /srv/corium --window 72h

Both use the same retention rule. --data-dir and --transactor conflict.

An encrypted database needs the key for offline collection:

corium gc --data-dir /srv/corium --window 72h --storage-key file:/etc/corium/storage.key

Offline collection refuses to run on an encrypted database without the key. Without the key, a sweep deletes the index chunks that it cannot follow.

Zero window

CAUTION: Use --window 0 only when no stale root and no in-flight reader can exist. A zero window deletes a blob as soon as it is unreachable. A reader that still holds an older root then fails.

A safe use is a stopped system with no peers running.

Why collection is safe

Deletion is the only mutation, and it touches only unreachable data. A bug in the mark phase can therefore strand garbage. A generous window makes data loss a non-risk.

Deleting a database deletes its root, and the sweep reclaims the blobs afterward.

Monitoring

Three counters report collection.

SourceFields
corium db stats <db>:gc-runs, :gc-swept-blobs
Metrics endpointcorium_transactor_gc_runs_total, corium_transactor_gc_swept_blobs_total, corium_transactor_gc_retained_blobs_total
corium tuiThe Metrics panel

A retained count that grows steadily means that the window is longer than it needs to be, or that a root is pinning old segments.

Tuning

SituationSetting
Peers hold long-lived database valuesRaise --gc-window above the longest reader lifetime.
Storage cost matters more than reader toleranceLower --gc-window, and watch for reader errors.
Bulk load in progressSet --gc-interval off, and run one manual sweep afterward.
A pause on the active transactor causes failoverRaise --lease-ttl-ms, or lengthen --gc-interval.

Collection competes with index publication for processor time and storage bandwidth. On a busy system, run it less often rather than with a shorter window.

Monitoring

Three surfaces report the state of a Corium system: the metrics endpoint, corium db stats, and the tracing log.

The metrics endpoint

Pass --metrics-listen to a transactor or a peer server:

corium transactor --data-dir /srv/corium --metrics-listen 127.0.0.1:9464

The endpoint serves Prometheus text at /metrics.

CAUTION: Keep the metrics listener on a private operations network. The endpoint has no bearer-token authentication.

Transactor metrics

MetricTypeMeaning
corium_transactor_transactions_totalCounterCommitted transactions.
corium_transactor_transaction_failures_totalCounterRejected transactions.
corium_transactor_transaction_latency_secondsHistogramCommit latency.
corium_transactor_queue_depthGaugeCommit queue depth.
corium_transactor_index_duration_secondsHistogramIndex publication duration.
corium_transactor_gc_runs_totalCounterCollection runs.
corium_transactor_gc_swept_blobs_totalCounterBlobs deleted.
corium_transactor_gc_retained_blobs_totalCounterUnreachable blobs kept by the window.
corium_keys_unavailableGaugeNodes that cannot load a key manifest change.

Peer server metrics

MetricTypeMeaning
corium_peer_queries_totalCounterQueries served.
corium_peer_query_latency_secondsHistogramQuery latency.
corium_peer_query_fuel_spent_totalCounterDatoms touched.

Segment cache metrics

A peer server with --segment-cache-dir adds these.

MetricLabelsMeaning
corium_peer_segment_cache_requests_totalresult, tierHits and misses per tier.
corium_peer_segment_cache_native_fetches_totalresultFetches that went to storage.
corium_peer_segment_cache_bytes_read_totalsourceBytes read per source.
corium_peer_segment_cache_admissions_totalresultAdmissions and rejections.
corium_peer_segment_cache_used_bytestierBytes in use.
corium_peer_segment_cache_capacity_bytestierConfigured capacity.

On-demand statistics

corium db stats people

The command prints the basis, the index basis, the index lag, the counts, and the transactor counters. See the database catalog.

The transactor Status call carries the same data. The Metrics panel of corium tui samples it live, and it is the only surface that shows lease ownership.

Auditing schema changes

Every applied schema update records its requester, its digests, its observed basis, and its acknowledgements on the transaction entity. Those are ordinary attributes, so the schema history of a database is a query.

[:find ?when ?who ?tool
 :where [?tx :db.schemaUpdate/requester ?who]
        [?tx :db.schemaUpdate/tool ?tool]
        [?tx :db/txInstant ?when]]
[[#inst 1785899778642 "static-token:operator" "corium-cli/0.1.0"]]

The requester is the authenticated principal. A caller never supplies it.

An ordinary transaction cannot write :db.schemaUpdate/*, so no transaction can claim to have been a schema update. See schema management.

Logging

Tracing is human-readable by default. --log-format json writes structured logs, and RUST_LOG filters them.

RUST_LOG=corium_transactor=info,corium_peer=warn \
  corium --log-format json transactor --data-dir /srv/corium

--log-format is a global flag, so it comes before the subcommand.

Useful targets:

TargetContent
corium_transactorCommit pipeline, indexing, leases, garbage collection.
corium_peerConnection, subscription, failover.
corium_authz::auditEvery authorization decision, with its basis. Denials at info, grants at debug.

Log lines worth an alert:

  • standby took over write lease — a failover happened.
  • deposed — this member lost the lease. It stands down on its own.
  • standing by; lease held elsewhere — normal on a standby at startup.

What to alert on

SignalConditionMeaning
Index lagGrows without limitPublication cannot keep up.
corium_transactor_queue_depthStays highThe write path is saturated.
corium_transactor_transaction_failures_totalRises sharplyValidation errors, or a fenced writer.
corium_keys_unavailableAbove zeroA node cannot load a key manifest change.
corium_transactor_gc_retained_blobs_totalGrows steadilyStorage is not being reclaimed.
Lease ownerChanges unexpectedlyAn unplanned failover.

Peer memory is not exported as a metric. Watch the resident memory of the process, because a peer holds the whole history. See indexes and storage.

Runbooks

Each runbook is a procedure. Read the whole procedure before you start it.

Planned failover

Use this before maintenance on the active member.

  1. Stop the active member with Ctrl-C, or with SIGINT. It releases its leases on the way out.
  2. Watch the log of the standby for standby took over write lease. Takeover happens within one third of the lease time-to-live.
  3. Do the maintenance.
  4. Start the member again with the same --owner and --ha. It rejoins as standby.

If the supervisor sends SIGTERM, the process dies without releasing the lease. Takeover then costs one full lease time-to-live.

Crashed active

Nothing is required for service. The standby takes over within the time-to-live plus one third.

  1. Confirm the takeover. Watch the basis advance with corium db stats, and read the lease owner in the Metrics panel of corium tui.
  2. Start the crashed member again under its supervisor with --ha. It rejoins as standby.
  3. Investigate the crash afterward, not before.

Split-brain suspicion

Both members print ownership messages in their logs.

This is not possible for durable state. The root record is owned by exactly one lease version, and every publish and acknowledgement is fenced by it.

  1. A member that logs deposed is the loser. It stands down on its own.
  2. Trust the root record, not the process logs. corium tui reads the lease owner from the Status call.
  3. Take no other action.

Both members down

  1. Start either member. Prefer the one with the newest data-directory modification times if storage is not shared.
  2. The member waits out any unexpired lease. Without --ha it waits up to --lease-wait-ms. With --ha it waits without limit.
  3. It recovers by log replay, and it serves.
  4. Start the second member. It becomes standby.

Recovery from a backup

  1. Stop the affected transactor, and preserve its data directory. Do not delete it.
  2. Restore the newest backup into an empty directory, or under a new name: corium restore <file> --data-dir <empty-dir> --as-db <name>.
  3. Start a transactor on the restored directory.
  4. Wait until :index-lag in corium db stats reaches zero.
  5. Compare the basis, the datom count, the entity count, and the attribute count with the backup report.
  6. Run a known query, and compare the result.
  7. Redirect peers only after those checks pass.

Transactor will not start

Read the error first. Four causes are common.

Error namesCauseFix
A lease holderAnother transactor holds the lease.Add --ha to stand by, or stop the other member.
A storage keyThe process cannot resolve a named KEK.Give it a --storage-key that resolves.
A Cargo featureThe binary lacks the storage or OIDC feature.Rebuild with the feature.
A missing data directory--data-dir is absent.Pass it, or set :data-dir in the configuration file.

The process fails at startup for a key it cannot resolve. That is by design. It does not fail later at the first read.

Writes refuse with FAILED_PRECONDITION

Two causes give this status.

A standby. The message names the current lease holder. Point the client at that endpoint, or pass the whole endpoint list.

A fenced key. Run corium keys status <db>. If :keys-fenced is true, the manifest opened an epoch that this node cannot load.

  1. Give the transactor a --storage-key that resolves the KEK that the manifest names.
  2. Restart the transactor.
  3. Confirm that :keys-fenced is false.

See encryption at rest.

An ambiguous transaction

The result of a transact call whose connection died mid-request is unknown. The transaction is committed, or it is absent.

  1. Do not resubmit yet.
  2. Run sync on the connection.
  3. Read the data back and decide from the result.
  4. Resubmit only if the write is absent.

Changing the schema of a live database

Writes continue throughout. Only a blocked change stops the procedure.

  1. Edit the schema file. Keep every attribute that must survive, because a file that omits one reports it as unmanaged.
  2. Plan it: corium schema update <db> --schema <file>. Nothing is written.
  3. Read every execution class, every count, and every acknowledgement code.
  4. Run the invocation that the last line of the plan prints, and add the path of the schema file.
  5. Confirm the new schema generation in the output of the apply.
  6. Compare :attributes in corium db stats with the expected count.

A plan is invalidated by a schema change or a failed condition, never by ordinary data writes. Re-plan if the digest is refused.

See schema management.

A schema plan is blocked or refused

Read the reason that the plan prints under the change.

ReasonMeaningFix
value-type-mutationA value type cannot change in place.Follow the replacement-attribute recipe that the plan prints.
unique-duplicatesDuplicate values exist.Retract the duplicates, then plan again.
cardinality-conflictsAn entity holds several values where the file asks for one.Choose a winner per entity and retract the rest.
ever-protectedThe attribute has been protected at some time.It can never gain index or unique. Drop them from the file.
protection-conflictThe declaration combines protection with index, unique, or ref.Drop the conflicting property from the declaration.

An apply can also fail after a clean plan.

Error codeMeaningFix
plan-mismatchThe schema changed between the plan and the apply.Plan again and read the new plan.
allow-requiredA change needs --allow <class>.Add the exact allowance the plan names.
ack-requiredA change needs --ack <code>.Add the exact code the plan names.

Every request is denied

The policy denies, or the policy is unreadable.

  1. Run corium authz status. A missing basis means the policy is unreadable.
  2. If it is unreadable, a break-glass role admits an operator. See authorization.
  3. If the policy denies, stop the transactor.
  4. Start it again without --authz-db.
  5. Fix the tuples with corium authz grant. Test each one with corium authz check.
  6. Restart with --authz-db.

Index lag grows without limit

  1. Read :index-lag in corium db stats, and the publication duration in the metrics endpoint.
  2. Lower --index-backoff, so publication takes a larger share of wall-clock time.
  3. Lower --index-tail-threshold if a large threshold defers the work.
  4. If neither helps, the storage backend is the limit. Give it faster storage, or reduce the write rate.

Index lag never risks durability. It lengthens cold-peer bootstrap and it makes a backup less fresh.

A peer uses too much memory

A peer holds every datom that it has seen, including retractions.

  1. Confirm the cause. Memory tracks total history, not the size of the live database.
  2. Restart the peer with --peer-bootstrap, so it starts from the published snapshot rather than replaying the log from basis 0.
  3. Avoid opening many distinct time views in one process. Each distinct view costs a fold of the whole history.
  4. Split the workload across more peer processes.

See indexes and storage.

Backing up an encrypted database

corium backup refuses an encrypted database.

  1. Stop the transactor, or accept a crash-consistent copy.
  2. Copy the underlying storage with its own tool. Use a filesystem snapshot, a PostgreSQL dump, or S3 replication.
  3. Copy the KEK separately, and keep it in a different system.
  4. Test the restore path on a separate host before you rely on it.

Storage is full

  1. Run a manual sweep: corium gc --transactor <url> --window 72h.
  2. If that reclaims little, read corium_transactor_gc_retained_blobs_total. A large retained count means that the window is holding the blobs.
  3. Lower --gc-window only when no reader holds a root older than the new window.
  4. Delete finished forks and staging clones with corium db delete.

Emergency: recreate a database from the log

The log is the source of truth. A data directory with an intact log recovers by replay.

  1. Stop every transactor that touches the directory.
  2. Preserve a copy of the whole directory.
  3. Start one transactor on the directory. Startup replays the log tail after the last published index basis.
  4. Compare corium db stats with the last known values.

Never edit or delete files under <data-dir>/logs by hand. Old lease-version files are inert history that readers must merge.

Command reference

This chapter lists every corium command. Run corium <command> --help for the authoritative flag list of a build.

Global

corium [--log-format human|json] <command>

--log-format comes before the subcommand. corium tui writes no tracing output, because it owns the terminal.

Shared flag groups

Connection flags apply to peer-server, postgres-server, every db subcommand, schema update, every authz subcommand, every keys subcommand, console, tui, and sql.

FlagEnvironmentDefault
--transactor <url>http://127.0.0.1:4334
--token <secret>CORIUM_TOKENShared development token
--ca <pem>None
--tls-domain <name>None
--peer-bootstrapOff

Serving flags apply to transactor, peer-server, and postgres-server.

FlagEnvironment
--serve-token <secret>CORIUM_SERVE_TOKEN
--require-auth
--serve-open
--oidc-issuer <url>
--oidc-audience <aud>
--oidc-jwks-file <path>
--authz-db <name>CORIUM_AUTHZ_DB
--authz-fresh-writes
--authz-break-glass-role <role>
--authz-max-depth <n>
--key-policy strict|server-wide
--tls-cert <pem>
--tls-key <pem>

--key-policy decides which protection class keys a caller can use. Only peer-server and postgres-server act on it. postgres-server rejects --tls-cert and --tls-key, because it does not terminate TLS.

Key flags apply to transactor, peer-server, postgres-server, gc, and log.

FlagEnvironment
--storage-key <uri>CORIUM_STORAGE_KEY

One keyring serves both purposes. It resolves key-encryption keys and protection class keys. The flag is repeatable, and the environment variable takes a comma-separated list.

Servers

corium transactor

Runs a transactor over a data directory. See the transactor.

Storage flags: --config, --store, --store-plugin, --plugin-store, --plugin-read-only-config, --data-dir, --turso-path, --postgres-url, --postgres-read-only-url, --s3-bucket, --s3-prefix, --s3-region, --s3-endpoint-url, --s3-read-only-access-key-id, --s3-read-only-secret-access-key, --s3-read-only-session-token, --s3-read-only-role-arn, --s3-read-only-role-session-name, --s3-read-only-role-duration-seconds, --s3-read-only-role-external-id.

Process flags: --listen, --owner, --advertise, --metrics-listen.

Lease flags: --ha, --lease-ttl-ms, --lease-wait-ms, --heartbeat-ms.

Index flags: --index-interval-ms, --index-backoff, --index-tail-threshold, --index-tail-deadline-ms.

Collection flags: --gc-interval, --gc-window.

Function flags: --db-fn-fuel, --db-fn-memory-bytes.

corium peer-server

Hosts one database for thin clients. See peer server and thin clients.

Flags: --db, --listen, --max-fuel, --metrics-listen, --segment-cache-dir, --segment-cache-capacity, --segment-cache-memory.

corium postgres-server

Serves the catalog over the PostgreSQL wire protocol. See SQL shell and PostgreSQL server.

Flags: --database (repeatable), --listen, --password, --allow-writes.

Storage administration

corium store verify <kind> <config>

Runs the blob and root conformance suite against a live backend. <config> is the JSON configuration object of the backend. Flag: --store-plugin (repeatable). See storage backends.

Use a disposable namespace. The command writes and deletes objects.

Catalog

corium db create <name>

Creates a database. Flags: --schema <path>, --storage-key <uri>.

corium db delete <name>

Deletes a database. It asks for no confirmation.

corium db list

Lists the databases that the transactor serves.

corium db stats <name>

Connects a peer and prints statistics.

corium db fork <name> <target>

Duplicates a database at a basis. Flag: --as-of <t>. See forking a database.

corium db request-index <name>

Publishes the indexes now, bypassing pacing.

corium db index-policy <name>

Reads or overrides the pacing of one database. Flags: --interval-ms, --backoff, --tail-threshold, --tail-deadline-ms. With no flags it prints the current policy.

Schema

corium schema update <db> --schema <path>

Compares a schema file with the schema installed in a database and prints the plan. It writes nothing without --apply. See schema management.

Flags: --schema, --prune, --json, --detailed-exit-code, --apply, --plan, --allow (repeatable), --ack (repeatable).

Authorization

See authorization.

CommandEffect
corium authz initCreates the policy database with schema, permissions, and a first administrator. Flags: --db, --admin, --provider, --no-admin.
corium authz grant <subject> <relation> <object>Asserts a relationship tuple. Flag: --db.
corium authz revoke <subject> <relation> <object>Retracts a relationship tuple. Flag: --db.
corium authz check <subject> <action>Prints what the policy decides. Flags: --database, --provider, --role, --claim, --db.
corium authz statusPrints the compiled basis and entity counts. Flag: --db.

Keys

See encryption at rest.

CommandEffect
corium keys status <db>Prints the KEK, the epochs, and the nonce budget.
corium keys rotate <db>Opens a new storage-key epoch.
corium keys rewrap <db> --kek <uri>Re-wraps the data keys under another KEK.

Data care

corium gc

Sweeps unreachable blobs. Flags: --data-dir or --transactor (they conflict), --token, --ca, --tls-domain, --window, --storage-key. See garbage collection.

corium backup <db> <destination>

Creates or refreshes a backup from a live transactor. Flags: --transactor, --token, --ca, --tls-domain. See backup and restore.

corium restore <source>

Restores a backup offline. Flags: --data-dir, --as-db. Both are required.

Interactive surfaces

CommandEffect
corium console <db>Datalog console. See query console.
corium tui <db>Dashboard. Flag: --refresh-ms. See terminal dashboard.
corium sql <db>SQL shell. Flags: -c/--command, -f/--file. See SQL.

Offline inspection

corium log

Prints committed transactions from a filesystem data directory. The transactor does not need to be running.

Flags: --data-dir, --db, --from, --to, --storage-key.

--from is inclusive. --to is exclusive, and 0 means open-ended.

Commands that do not exist

Not implemented. There is no corium transact command. Writes come from a client library, or from corium postgres-server --allow-writes.

Not implemented. There is no corium dump command. Human, JSON, and EDN export are deferred.

Not implemented. corium schema has only the update subcommand. status, history, and job inspection are planned and absent.

Not implemented. There is no corium keys protect, corium keys unprotect, or corium keys audit. Protection changes go through corium schema update. See attribute protection.

Environment variables

Every variable below has an equivalent flag. A flag on the command line overrides the variable.

Corium variables

VariableEquivalent flagUsed by
CORIUM_TOKEN--tokenEvery client command.
CORIUM_SERVE_TOKEN--serve-tokentransactor, peer-server, postgres-server.
CORIUM_AUTHZ_DB--authz-dbtransactor, peer-server, postgres-server.
CORIUM_STORAGE_KEY--storage-keytransactor, peer-server, postgres-server, gc, log.
CORIUM_STORE_PLUGINS--store-plugintransactor, store verify.
CORIUM_PLUGIN_READ_ONLY_CONFIG--plugin-read-only-configtransactor.
CORIUM_POSTGRES_READ_ONLY_URL--postgres-read-only-urltransactor.
CORIUM_S3_READ_ONLY_ACCESS_KEY_ID--s3-read-only-access-key-idtransactor.
CORIUM_S3_READ_ONLY_SECRET_ACCESS_KEY--s3-read-only-secret-access-keytransactor.
CORIUM_S3_READ_ONLY_SESSION_TOKEN--s3-read-only-session-tokentransactor.
CORIUM_S3_READ_ONLY_ROLE_ARN--s3-read-only-role-arntransactor.
CORIUM_S3_READ_ONLY_ROLE_EXTERNAL_ID--s3-read-only-role-external-idtransactor.

CORIUM_STORAGE_KEY accepts a comma-separated list, because one process can hold several keys. The same keyring resolves key-encryption keys and protection class keys.

CORIUM_STORE_PLUGINS accepts a path-separator-delimited list of files and directories. Corium searches a directory for platform dynamic libraries only, and it never adds the working directory.

Rust and AWS variables

VariableEffect
RUST_LOGTracing filter, for example corium_transactor=debug,corium_peer=info.
HOSTNAMESupplies the default --owner value, transactor-$HOSTNAME.
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEYPrimary S3 credentials.
AWS_PROFILE, AWS_REGION, AWS_ENDPOINT_URLStandard AWS configuration.

The transactor takes its primary S3 credentials from the standard AWS chain, which also covers instance and task roles.

Handling secrets

Process arguments are visible to every user on the host. Three rules follow.

  • Put a token, a password, or a connection URL in an environment variable, or in the configuration file.
  • Set the mode of a key file and of a configuration file so that only the service user can read them.
  • Prefer a file: storage key over an env: storage key. A file has a mode. An environment variable is inherited by child processes.

Default values

Network

SettingDefault
Transactor listen address127.0.0.1:4334
Peer server listen address127.0.0.1:4336
PostgreSQL server listen address127.0.0.1:5432
Metrics listen addressDisabled
Client transactor endpointhttp://127.0.0.1:4334

Storage

SettingDefault
--storefs
--data-dirNone. Required.
--turso-path<data-dir>/store.db
--s3-prefixBucket root
S3 read-only role duration900 seconds

Lease and availability

SettingDefault
--lease-ttl-ms5000
--lease-wait-ms15000
--heartbeat-ms10000
--haOff
--ownertransactor-$HOSTNAME
Peer reconnect backoff100 ms to 5 s
Peer failover timeout30 s

Index publication

SettingDefault
--index-interval-ms5000
--index-backoff4
--index-tail-threshold0
--index-tail-deadline-ms60000

Garbage collection

SettingDefault
--gc-interval1h
--gc-window72h
corium gc --window72h

Query and function budgets

SettingDefault
--max-fuel (peer server)10000000
--db-fn-fuel1000000
--db-fn-memory-bytes16777216 (16 MiB)
--authz-max-depth8

Segment cache

SettingDefault
--segment-cache-dirDisabled
--segment-cache-capacityNone. Required with the directory.
--segment-cache-memory64 MiB, or the capacity when smaller

Security

SettingDefault
AuthenticationPermissive: development token accepted, anonymous admitted
AuthorizationPermit-all
TLSOff
Encryption at restOff
Authorization database namecorium_authz
authz init --adminoperator
authz init --providerstatic-token
Attribute protectionOff
--key-policystrict when authentication is configured, server-wide when it is not
on-missing-key (protection class)redact
legacy-plaintext (protection class)redact
scope (protection class)attribute

Schema

SettingDefault
corium schema update modeRead-only. --apply is needed to write.
Permitted execution classadditive only
--pruneOff. Absent attributes are reported as unmanaged.
Exit code with changes planned0, or 2 with --detailed-exit-code

Interactive surfaces

SettingDefault
corium tui --refresh-ms2000, minimum 250
corium log --from0
corium log --to0, meaning open-ended
corium postgres-server write modeRead-only

Duration and size formats

A duration accepts one of the suffixes ms, s, m, h, and d, for example 1h, 30m, 72h. A bare number is seconds. off disables the scheduled collection duty.

A byte size accepts B, KiB, MiB, GiB, TiB, kB, MB, GB, and TB. A bare number is bytes.

Glossary

Attribute. An entity in :db.part/db that declares a name, a value type, a cardinality, and optional properties. See schema management.

AEVT. The covering index sorted by attribute, entity, value, transaction. It holds all current datoms.

AVET. The covering index sorted by attribute, value, entity, transaction. It holds the datoms of indexed and unique attributes only.

Acknowledgement code. A stable, kebab-case name for a schema change whose meaning changes, passed back with --ack.

Basis. The transaction number, written t, that a database value covers.

BLAKE3. The hash function that addresses blobs.

Blob store. The immutable, content-addressed half of the storage service.

Cardinality. Whether an attribute holds one value or many per entity.

Chunk. A content-defined run of a sorted key stream. A published leaf is exactly one chunk.

Class key. The key that seals values on the attributes of one protection class. A process resolves it through its own keyring.

Covering index. An index that holds whole datoms, so an answer needs no second lookup.

Database root. The record in the root store that names a database: its basis, its index roots, its log root, and its write lease.

Database value. An immutable snapshot of a database at one basis.

Datom. One fact: entity, attribute, value, transaction, and assert or retract.

EAVT. The covering index sorted by entity, attribute, value, transaction. It holds all current datoms.

Entity id. A 64-bit number: a 22-bit partition and a 42-bit sequence.

Epoch (storage key). One generation of the per-database data key. New writes seal under the newest open epoch.

Execution class. How much work a schema change needs: additive, validate-reindex, rewrite, or destructive.

Fence. The mechanism that stops a deposed writer. Every root write is a compare-and-set on the record that holds the lease.

Fork. A new database that duplicates an existing one at a basis, and then diverges. See forking a database.

Fuel. A budget on the datoms a query touches, or on the work a database function does.

Group commit. Committing concurrent transactions as one batch under one durability boundary, while each keeps its own t and its own acknowledgement.

Ident. A keyword that names an entity, above all an attribute.

Index basis. The transaction number that the published index trees cover.

Index lag. The difference between the basis and the index basis.

KEK. Key-encryption key. It wraps the per-database data key. Corium never stores it.

Key policy. Whether a serving process hydrates a caller with the class keys that policy names (strict) or with its whole keyring (server-wide).

Lease. The right to write one database. It lives in the database root and is renewed by compare-and-set.

Lease version. The generation of a lease. It names the log file, or the log key prefix, that its holder appends to.

Lookup reference. A [attribute value] pair on a unique attribute, usable anywhere an entity id is expected.

Partition. The high bits of an entity id. Entities of one partition sort together in EAVT.

Peer. A library, or a process, that holds a database value and queries it locally.

Peer server. A peer hosted as a standalone process for thin clients.

Plan digest. The hash of a schema plan. --apply refuses a digest that no longer describes the change.

Principal. The identity of a request, produced by an identity provider.

Protection class. A named key identity and sealing policy. An attribute that names one has its values sealed by the writing peer. See attribute protection.

ReBAC. Relationship-based access control, the authorization model that --authz-db enables.

Retirement. The schema change that refuses new assertions on an attribute while keeping its ident, its metadata, and its history readable.

Root store. The small, mutable, strongly consistent half of the storage service. It is updated only by compare-and-set.

Schema generation. A per-database counter that advances once for each committed transaction containing a schema change.

Segment. An immutable, content-addressed node of an index tree.

Standby. A transactor that polls a lease held elsewhere and takes over when it lapses.

Storage plugin. A dynamic library that registers a storage backend at run time. See storage backends.

Tempid. A transaction-local placeholder for an entity id, resolved at commit. A collision on a :db.unique/identity attribute becomes an upsert.

Transaction report. The record that a commit broadcasts to peers: the basis before, the basis after, the datoms, and the tempid map.

Transactor. The single writer for a database.

Unmanaged attribute. An installed attribute that the desired schema file does not name. corium schema update leaves it alone unless --prune is given.

Upsert. Unifying a tempid with an existing entity through a :db.unique/identity attribute.

VAET. The covering index sorted by value, attribute, entity, transaction. It holds reference-typed datoms.

View. A policy object that hides attributes from a principal, names the class keys the principal can use, or both.

t. The sequence part of a transaction id, and the name of a basis.

:db/txInstant. The commit time of a transaction, asserted as a datom on the transaction entity.