The stamp
- Ask did the suite still print green
- Stamp SYS-REQ-010 valid. lemma cache still timeout
- Why the YAML still looks proved. The tests still run
Topic · formalization lemma verdict consistency
Gist
Formalization lemma verdict consistency is a SYS-REQ that still says formalization_status: valid while its lemma cache is timeout, unknown, translation_error, or solver_crash. Proof runs proof audit --check formalization_lemma_verdict_consistency. valid should mean PROVED. Jama still authors.
proof audit --check formalization_lemma_verdict_consistency
Keep the live shall if it is still true. Keep Jama if it already holds it. Neither one fails the merge when the status and the cache live in separate files.
01 · The silent valid
You can keep verification.formalization_status: valid after the cache for that lemma is still timeout. The suite still runs. The YAML still looks proved.
The check is formalization_lemma_verdict_consistency. It is verify-stage. Severity of a conflict is warning, not fail. The inspect command is the same audit, plus proof verify-lemma --no-cache ./....
The hop joins SYS-REQs whose formalization_status is valid against a lemma the YAML names in traces.verified_by_extra as LEMMA:<scope>/<name>, or a Go lemma that names the SYS-REQ with // verifies:. The cache is .proof/lemma-cache.json, filled by proof verify-lemma. A conflict is a most-recent verdict of timeout, unknown, translation_error, or solver_crash. The honest status for that row is inconclusive.
The help file teaches the silent overstatement first. Status and solver cache live in separate files. Nothing in the suite asked whether they contradicted.
# specs/system/requirements/SYS-REQ-010.req.yaml
id: SYS-REQ-010
verification:
formalization_status: valid
traces:
verified_by_extra:
- LEMMA:resolve/resolvable_reset_clears_walker_state
# .proof/lemma-cache.json
# resolvable_reset_clears_walker_state: timeout
# proof audit --check formalization_lemma_verdict_consistency
# [VERIFICATION] formalization_lemma_verdict_consistency
# SYS-REQ-010 claims formalization_status=valid but lemma
# "resolvable_reset_clears_walker_state" is inconclusive (verdict=timeout)
# silent valid: the suite still printed green
Downgrade to inconclusive if the lemma is still undecided. Re-run proof verify-lemma --no-cache ./... if a rewrite should decide it. Remove the LEMMA: row only when the lemma is gone, then proof verify-lemma --prune-cache ./.... Then re-run the same check. Do not delete a live lemma reference to make the warning disappear. That leaves valid standing on nothing.
proof verify-lemma --no-cache ./...
proof verify-lemma --no-cache --timeout 120s ./...
proof verify-lemma --prune-cache ./...
proof audit --check formalization_lemma_verdict_consistency --verbose
02 · The exhibit
One valid stamp. A lemma named in the YAML. Cache still timeout. Click the tabs.
The stamp
This hop
Nobody asked whether the lemma named on this SYS-REQ had a decided verdict. A green suite is not a solver. The finding kind is this hop.
Cache unreadThe stamp
Keep the live shall. Keep the Jama cell. That is not this hop.
Keep the recordProof
Same SYS-REQ. A silent valid, or this hop. Click the tabs.
| Surface | What they do | What Proof does | What we lose |
|---|---|---|---|
| Green suite | The tests that still ran. | Whether a valid SYS-REQ names a lemma whose cache is still undecided. | We do not rerun the suite here. A green stamp is not this hop. |
| Z3 / Kind2 on one function | Run the solver on a lemma you already wrote. | Whether the YAML still claims proved after that run timed out. | Not the solver hop. See Z3/Kind2 on one function. |
| Assume contract consistency | Whether a source assume still names a callee lemma. | Whether a SYS-REQ that already names a lemma still matches the cache. | Not the assume hop. See assume contract consistency. |
| DO-333 | Formal methods in the DO-178C supplement. | One cache join on valid vs timeout. |
Not FM.3. Not SPARK. See DO-333. |
| Jama cell | A shall, and a note if you type it. | A warning the audit can name next to the requirement id. | Not Jama's V&V. Jama still authors. We have not run a frozen Jama pack. |
The teaching graph is still one SYS-REQ whose board called the formalization done and whose lemma cache never decided. Close it by writing inconclusive, by re-running the solver until it PROVES, or by dropping a lemma that is genuinely gone. The same hop also walks the reverse // verifies: annotation. A pair that appears both ways is one finding. A missing cache file is a skip, with a hint to run verify-lemma. An absent lemma name in the cache is a separate informational finding, not this mis-classification. Do not treat proof verify-lemma --allow-unknown as this hop. That flag changes the CI exit code. It does not make valid true.
# specs/system/requirements/SYS-REQ-010.req.yaml
verification:
formalization_status: inconclusive
traces:
verified_by_extra:
- LEMMA:resolve/resolvable_reset_clears_walker_state
# proof audit --check formalization_lemma_verdict_consistency
# no valid-on-undecided join. this hop is quiet
The solver hop stays on Z3/Kind2 on one function. The assume hop stays on assume contract consistency. Jama still authors. Proof vs Jama.
03 · The honest loss
A quiet proof audit --check formalization_lemma_verdict_consistency can still mean there was no lemma cache, or no valid SYS-REQ with a lemma. Jama still authors.
Warning, not fail. Not blocking. No project loaded is a pass. No .proof/lemma-cache.json is a skip, not a fail. A SYS-REQ with valid and no lemma reference is a silent pass: a FRETish-only formalization may claim that. The hop looks at formalization_status, LEMMA: traces, // verifies: comments, and the cached verdict. It does not re-run Kind2. It does not re-run Z3. It does not write the YAML. It does not prove the Go. Raising severity to error in proof.yaml is a strictness setting, not a remediation. We have not scored this floor against a frozen Jama pack or a ReqIF export. The loss is named, not scored.
The solver hop stays on Z3/Kind2 on one function. The engagement stays on software correctness audit. Jama still authors.
04 · Nearby questions
What is formalization lemma verdict consistency? Same question. Same URL.
Is this Z3 or Kind2 on one function? No. That hop runs the solver. This hop asks whether a SYS-REQ that already claims valid still matches that cache. See
Z3/Kind2 on one function.
Is this lemma binding freshness? No. That hop is a separate-file binding whose SHA-256 no longer matches the live function. This hop is a SYS-REQ that still says valid while the lemma cache is timeout. See
lemma binding freshness.
Is this lemma branch coverage? No. That hop is a production AST branch with an empty covered_by. This hop is a SYS-REQ that still says valid while the lemma cache is timeout. See
lemma branch coverage.
Is this assume contract consistency? No. That hop is a source // reqproof:assume whose callee lemma must exist. This hop is a SYS-REQ that already names a lemma. See
assume contract consistency.
Is this DO-333? No. That hop is the DO-178C formal-methods supplement. This hop is one cache join. See DO-333.
Is this design by contract? No. That hop matches YAML assume/guarantee pairs. This hop is valid vs the lemma cache. See
design by contract.
Does a tree with no lemmas pass? Yes. No valid SYS-REQ with a lemma is a pass. That is not a proof that the Go is correct. It is a proof that this hop had nothing to join.
Does a green hop prove the code matches the shall? No. The hop observes YAML fields and a cache file. It does not prove the Go.
Is Proof a Jama alternative for the shall? No. Jama still authors. Proof vs Jama.