Verify the change.
Understand the system.

Choose the scope first

“AI large scale code verification” can describe three different jobs: reviewing AI-assisted changes in a large codebase, evaluating many generated programs, or proving properties of a program. They require different evidence.

For application work, start with the proposed change and its callers. OpenAI’s code verification research reports that repository access and execution context improved its reviewers compared with diff-only review. Treat that as a result about the authors’ evaluation, not a guarantee for your repository.

Build a verification loop

1. Describe the contract. Write the expected behavior, failure cases and compatibility constraints in the pull request. Include what must stay true.
2. Map the affected boundary. Read callers, schemas and configuration. A function signature may be unchanged while its behavior differs.
3. Run deterministic checks. Compile or type-check, lint, run focused tests, then exercise the integration boundary. Keep the exact commands.
4. Challenge the test oracle. Ask whether the assertions would fail for a plausible wrong implementation. Tests copied from the same generated answer can repeat its assumptions.
5. Inspect security-sensitive paths. Review trust boundaries, authorization, secrets, dependencies and input handling. CodeQL can support security analysis for supported languages; its alerts still need interpretation.
6. Resolve findings with evidence. Require a failing example or a concrete path to impact. Record the owner of any accepted gap.

Trace an alert from source to sink

For a data-flow alert, open the reported path and follow each step from the input source to the sensitive operation. Check whether the input can reach that operation in the relevant configuration, and whether a validation or sanitization step changes the result.

GitHub’s example identifies a database query built from user-provided data. Use the path and linked rule documentation to form a reproducible test. The reported path is a candidate problem to confirm in context; it does not automatically prove that your application is exploitable.

A worked example: updating an address

Suppose an assistant changes an endpoint that updates a customer’s billing address. A successful update test covers only one branch. Write the invariant before expanding the suite:

Given an authenticated customer A
When A submits an update for customer B
Then the server rejects the request
And B's stored address remains unchanged
And the response does not expose B's personal data

Exercise the public endpoint rather than calling only its helper. Include an expired session, a missing customer, malformed input and a repeated request. Check the stored value after rejection. If the update publishes an event, confirm that a rejected request publishes none.

These are editorial recommendations for this scenario, not a complete security specification. Your system may require additional controls, including auditing and approval.

Keep checks useful as the repository grows

Use fast checks on every pull request and broader suites on an appropriate schedule. Dependency-aware selection can shorten the immediate loop, but changed-file selection alone misses shared configuration and indirect callers. Run a full suite periodically and compare what the narrower suite missed.

Separate infrastructure errors from test failures. Preserve the seed, environment, tool version and failing input when an intermittent failure appears. Retrying until green can hide a defect; track retries as evidence instead.

For generated-program evaluation, the execution environment also needs resource limits and a suitable judge. ScaleBox addresses that setting. Its throughput results should not be used as an estimate for an application CI pipeline.

Before merging

• The stated behavior has a test or a documented verification argument.
• Failures are reproducible, or the unresolved uncertainty has an owner.
• Security findings and compatibility changes have been reviewed.
• The person approving the change can explain why the evidence is sufficient.

Choose the next check from the change

Use the method-selection guidance for the property you need to check, then generate a plan for the review.

A green job means its configured checks passed. It does not establish that every relevant property was checked.