Invariants
An invariant is a condition that must stay true during the whole run. The registrar checks each invariant while the run continues, not at the end.
invariants: - { layer: letter, no_edits_outside: ["src/**"] } - { layer: letter, untouched: "tests/test_sync.py" } - { layer: judgment, clause: "Changes stay minimal.", checkpoint: each_edit }An invariant is different from a satisfaction clause. A satisfaction clause says when the work is complete. An invariant says what the agent must not do while the work continues.
Letter invariants
Section titled “Letter invariants”A letter invariant is a machine check. The registrar makes the check before each action. There are two forms.
no_edits_outside
Section titled “no_edits_outside”The agent can change a file only when a pattern in the list matches the file.
invariants: - { layer: letter, no_edits_outside: ["src/**", "docs/*.md"] }untouched
Section titled “untouched”The agent cannot change the named file.
invariants: - { layer: letter, untouched: "tests/test_sync.py" }This form stops the most common way that an agent cheats a test: the agent changes the test instead of the code.
An invariant takes exactly one of these two forms. To state both conditions, write two invariants.
Judgment invariants
Section titled “Judgment invariants”A judgment invariant is a clause in prose. A judge model reads the clause and the actions, and then makes a decision.
invariants: - { layer: judgment, clause: "Changes stay minimal.", checkpoint: each_edit }You must configure a judge before you use this layer.
checkpoint
Section titled “checkpoint”The checkpoint field states when the judge makes the check. The default
checkpoint is at_satisfaction.
| Value | The judge makes the check |
|---|---|
each_edit |
After each change to a file. |
at_satisfaction |
When the satisfaction clauses pass. |
{ each_n_iterations: 5 } |
After each 5 iterations. |
A frequent checkpoint finds a problem early, and costs more. Each check spends part of the cost bound.
Assent invariants
Section titled “Assent invariants”An assent invariant asks a person.
invariants: - { layer: assent, clause: "A person approves each change to the schema." }The run stops until a person answers. You must configure an assent transport.
Prefer the letter layer
Section titled “Prefer the letter layer”Where you can, state a worry as a letter invariant. A machine check is exact, it costs nothing, and it applies to each action.
Take the worry “the agent must not cheat the test”. As a judgment clause, a model decides what “cheat” means. As a letter invariant, the condition is exact:
invariants: - { layer: letter, untouched: "{failing_test_file}" }Read more about the three check layers.
What happens at a breach
Section titled “What happens at a breach”An invariant breach starts the on_invariant_breach remedy. The default
remedy is inform with 3 strikes: the registrar tells the model which clause
failed, and the run continues. After 3 breaches, the run stops.
Read more about remedies.