How it works
It reads your contract,
not your test file.
A product team already owns the one document that says what its service does: the wire contract. SODL treats that document as the source of truth and derives everything it understands from it — fresh, on every compilation.
Nothing is authored. Nothing is stored. Nothing test-related goes into
your contract. sodl.yaml names where the contract lives;
SODL links to it at compile time and re-derives from scratch.
The pipeline
Four stages, every run
The runtime infers nothing. Every binding was resolved at compile time, which is why a scenario either compiles against today’s contract or does not run at all.
Composed once per run. There is no cached model to drift.
What gets derived
- Every operation and its request / response field shapes
- Entities, and which field carries an entity’s identity
- How entities correlate across operations
- Enum-backed lifecycles — the states a thing can be in
- Which operations are observable, and which are not
What is never derived
- Anything the contract does not state. Ambiguity stays ambiguous until you resolve it by name
- A merged meaning for two resources that merely share a name across providers
- A guessed endpoint. An operation from an OpenAPI contract with no declared HTTP endpoint fails honestly, naming what is missing
The compile gate
An invalid scenario is rejected before it runs
This is the difference that makes the rest possible. Your scenario is type-checked against the derived surface — so when the contract moves, the scenario stops compiling, rather than passing against a system that no longer exists.
Before — the field exists
$ sodl check
Checking contracts & derivation... ✓ PASS contracts resolved; semantics derived cleanly
Running scenarios... ✓ PASS 14 scenarios, 14 passed
Checking coverage... ✓ 31 of 34 operations exercised
────────────────────────
Verification: passedAfter — someone removed it
$ sodl check
Checking contracts & derivation... ✓ PASS contracts resolved; semantics derived cleanly
Running scenarios... ✗ FAIL 1 file does not compile
────────────────────────
Verification: failed
Findings
✗ Execution: 1 file does not compile
✗ scenarios/create-and-approve.sodl — does not compile:
unknown field "discount_code" on CreateOrderOutput shape is SODL’s own; the names are a worked example. Nothing was executed — the failure is a compile-time fact.
A test engine would have run this scenario, sent a request missing a field it believed in, and reported whatever the server said. SODL never got that far.
The five capabilities
Four of these are impossible without the first
The dependency runs one way. That is also the answer to the obvious question — why can’t an existing tool just add this? Bolting derivation onto an authoring tool gets you a linter. The five only exist together.
Derive
The verifiable surface is computed from the contract, never written
by hand. sodl analyze reports what SODL understood
— and, just as importantly, what it could not.
The same model powers the editor
- Completion offers the operations SODL derived from your contract — not a snippet list someone maintained
- Hover shows each fact tagged with the source it came from, and says so when a fact is inferred rather than stated
- An ambiguous name is shown fully qualified, every candidate listed, rather than resolved by guess
Prove
Run the scenario against the live system. This is the one output a test engine also produces — but SODL reports it in six honest states rather than two, because “did not pass” and “failed” are different facts.
One failure never stops another file from running. Only a failed setup skips its own subtree — and says so.
$ sodl run
PASS scenarios/billing/create-and-approve.sodl
PASS scenarios/billing/refund-partial.sodl
FAIL scenarios/billing/refund-full.sodl
refund-full.sodl:12:3 — INVALID_ARGUMENT: refund exceeds captured amount
NOT SERVED scenarios/identity/rotate-key.sodl
SKIPPED scenarios/reporting/monthly.sodl
Illustrative names; the states, their meanings and the
file:line:col failure line are SODL’s own.
Coverage
Because SODL knows the whole surface, it has a denominator. It can name the operations no scenario exercises, the entities nothing asserts on, and the lifecycle states nothing ever reaches.
Coverage is disclosure, never a gate. No threshold, no score, no effect on the exit code. A number that can block a release becomes a number people manage instead of a number people read.
$ sodl coverage
Operations: 31 of 34 exercised
billing.BillingService: 18 of 19 exercised
identity.IdentityService: 13 of 15 exercised
Entities: 4 of 5 asserted-on, 1 untouched
Order.status (OrderState): 4 of 6 states assertedLine formats are SODL’s own; the service and entity names are a worked example.
Drift
SODL snapshots the contract a release was verified against, then re-checks today’s contract against it. A removed operation, removed field, changed type or removed enum state fails the gate even when no scenario covers it.
The gate never guesses: a change whose baseline position is unknown is reported without a location rather than with an invented one.
$ sodl drift ./release-bundle -against .
Uncovered breaking changes (no scenario in the bundle exercises these):
CreateOrder.discount_code: field removed (billing/v1/billing.proto:64)
Order.status: state CANCELLED removed (billing/v1/billing.proto:112)
Informational changes (1):
GetOrderHistory: operation added (billing/v1/billing.proto:88)
Counts: 0 broken, 0 pre-broken, 2 breaking changes, 1 informational changes
$ echo $?
1Exit 0 no drift · 1 drift detected · 2 usage or load error.
Evidence
Every run writes an immutable JSON record answering five questions: what ran, against which contracts, in which environment, when, and by which version of SODL.
Written by temp-file and rename, never reopened. Re-running writes a new timestamped file; it never edits an old one.
$ ls .sodl/runs/
20260801T093012Z.json
20260801T141955Z.json
$ sodl report -open # the latest record, as a pageA principle
Six states, because “not passed” is not one thing
Most tools have two outcomes and quietly file everything awkward under the second. That habit blames your system for the tool’s own limits. SODL refuses to do it.
| State | Band | What it actually means |
|---|---|---|
| PASS | passed | The scenario ran and every expectation held. |
| FAIL | failed | The scenario ran and something did not hold. The first failing position and the message print verbatim beneath it. |
| SKIPPED | not verified | Its folder’s own setup failed, so the subtree beneath it never ran. Nothing was proved — and nothing is claimed. |
| NOT SERVED | not verified | The deployment does not implement the operation at all. A fact about the deployment, never confused with a failed assertion. |
| UNSUPPORTED | not verified | SODL itself cannot run the construct in this version. Our boundary, not your system’s fault — and counted, so the boundaries that actually cost people something are the ones we fix first. |
| UNCOMPILABLE | not verified | The file never cleared the compile gate, so nothing in it ran. The diagnostics print beneath it. |
The exit code stays simple. 0 only when
every scenario passed; 1 if anything failed, was skipped, was
not served, was unsupported or did not compile; 2 on a usage
or tree-walk error. The nuance is in the report, not in the gate.
Next
The evidence model is where derivation pays off — and the boundaries page says plainly what SODL will not do for you.