What each method can establish
| Method | Useful evidence | Remaining gap |
|---|---|---|
| Compilation / type checks | The program meets configured syntax and type rules. | Well-typed code can implement the wrong policy. |
| Unit / integration tests | Observed behavior for selected inputs and environments. | Untested paths and incorrect expected outputs. |
| Static security analysis | Patterns or data flows reported by configured queries. | Unsupported frameworks, modeling gaps and false alerts. |
| Repository-aware AI review | Candidate defects interpreted in project context. | Findings can be wrong; missing findings are not a proof. |
| Sandbox evaluation | Execution results within configured limits and test cases. | A weak judge or weak isolation can invalidate the result. |
| Formal verification | A proof obligation checked against stated specifications and assumptions. | The specification and environment may omit the real failure. |
CodeQL’s official documentation explains its database-and-query analysis model. Dafny’s reference describes specifications and verification. Read the tool notes before choosing a deployment.
Read the alert context
A scanning alert includes the rule, highlighted operation and analysis context. In GitHub’s example, affected-branch metadata sits beside the code. Confirm that you are looking at the branch and configuration relevant to the change before acting on the result.
Severity helps prioritize review. It does not replace checking the data path, the surrounding controls or the scope of the analysis.
Check how you decide a result is correct
A string comparison is appropriate when the contract requires an exact string. It can reject a valid answer when the task allows several solutions.
The ScaleBox paper illustrates this with special judges. Judge logic itself needs tests: known valid solutions, known invalid solutions, boundary inputs and malformed output.
For application tests, use the same discipline. An expected value should follow from the specification, not just from the current implementation.
When formal verification fits
A proof-oriented workflow needs a property precise enough to express. Preconditions state what callers must provide; postconditions state what a successful implementation guarantees. Loop invariants describe what remains true across an iteration.
For example, you might specify that a function returns a sorted permutation of its input. “The function works well” is too vague to verify. The proof still depends on the model and assumptions, including how external systems behave.
The Dafny language reference documents contracts and verification constructs. This site does not run a verifier or produce proof certificates.
Choose the first check
Start with a concrete failure and the cheapest check that can expose it. A serialization change needs compatibility examples. Authorization needs adversarial access cases. A mathematical routine may benefit from properties and proof obligations.
When the failure crosses a service boundary, add an integration test. When untrusted input reaches a sensitive sink, inspect data flow and consider static security analysis. Use AI review to propose additional cases, then verify the findings.
Turn this into a plan ↗