---
title: "How to make a coding agent keep working until your Solana tests pass · Barestate"
description: "Stop AI coding agents from declaring a Solana task done before the tests pass: use a check the model cannot change, with Claude Code hooks, CI, or Oh My Sol’s /goal and independent checker."
url: https://barestate.xyz/solana/make-a-coding-agent-work-until-solana-tests-pass
---

# How to make a coding agent keep working until your Solana tests pass

Short answer

Take the decision away from the model. Give the agent one command that proves the work, such as cargo test or anchor test, and have something else run it: a Claude Code Stop hook, CI, or a checker. In Oh My Sol, /goal <text> --check "<command>" keeps the model going until that check passes, and only an independent checker on a fresh offline Surfpool chain can close the task.

Who it’s for

Solana developers whose AI agent stops early, skips the tests or edits them to pass, and who want a loop that ends only when the tests really pass.

Last checked

November 16, 2026, against Oh My Sol 0.1.0

## Why do coding agents stop before the tests pass?

In most coding agents, the model that writes the code also decides when the work is finished. That is the failure mode. A model can stop after the program compiles, report tests it never ran, or change a failing assertion instead of the code. On Solana, code that looks right can still fail when it runs on a chain, so the test run is the part that matters.

The fix is the same whatever tool you use: define “done” as a command that exits 0, and let something other than the model run it and read the result.

## How do I set this up with any coding agent?

1. **Write or pick the check first.** `cargo test` runs in-process tests such as LiteSVM or Mollusk quickly. `anchor test` builds the program, deploys it to a local validator and runs the tests.
2. **Give the agent the exact command** and say the task is not done until it exits 0, and that it must not edit the test files.
3. **Make the check run outside the model.** In Claude Code, a Stop hook can run the tests and exit with code 2 on failure, which prevents Claude from stopping and continues the conversation. Check `stop_hook_active` so a failure that will never resolve does not loop. In any agent, a CI job on the branch does the same job after the fact.
4. **Protect the tests.** Review the diff for changes under `tests/`, or keep the test files somewhere the agent cannot write.
5. **Run the check yourself** before you merge.

Sources: [Anchor installation](https://www.anchor-lang.com/docs/installation), [Claude Code hooks](https://code.claude.com/docs/en/hooks).

## How does Oh My Sol keep working until a check passes?

Oh My Sol (OMS) is built around this split. Every task has a requirements file, its own Git worktree and a budget. The model edits files in the worktree, but its only completion tool is `propose_done`, which moves a requirement from `active` to `proposed`. It has no tool that closes one.

A separate checker then starts a fresh offline Surfpool chain from a pinned snapshot and runs the requirement’s checks against a saved copy of the code, not your live worktree. A pass closes the requirement. A fail sends it back to `active` and the model tries again if budget remains. When the task was created, OMS recorded digests of the contract, the verifier files and the evaluator config; if any of them changes, the evaluator refuses to run.

To set a goal with your own check inside OMS:

```
/goal <text> --check "<command>"
/status      # requirements, proof and budget
/diff        # task changes against the base
/verify      # rerun the independent check, no model turn
/goal clear  # remove the goal
```

## What does a run look like?

```
/goal Add a withdraw instruction that rejects non-owners --check "cargo test"
```

The model edits the program and proposes that it is done. The checker runs `cargo test`. If a test fails, the requirement goes back to `active`, the check output is passed to the model (through a redactor), and it tries again. While it works you can run `/status`, `/usage` and `/diff`. When the check passes, the requirement closes as verified and the session settles.

OMS stops and hands control back to you when any of these happens:

| Stop reason | What it means | What to do |
| --- | --- | --- |
| `repeated_failure` | The same failure came back twice in a row | Read the feedback, change the requirement or the code, run `/goal` again |
| Attempt cap | Attempts reached the cap, 5 by default | Check `/status` and `/diff` |
| `blocked` | The checker reopened the requirement 3 times | Look at the check output with `/status` and `/diff` |
| Budget used up | Defaults: 2,000,000 tokens, $5, 30 minutes | See `/usage` |
| Checker unavailable | OMS stops rather than let the work through unchecked | Run `/verify` again, then `oms doctor` to check Surfpool |

## Does the check make a difference?

Yes. With the same models, OMS beats both Codex CLI and Claude Code on eight fresh Solana tasks, each graded by a hidden check: 96% of runs passed against 76% for Codex CLI with GPT-6.1 Sol, 92% against 79% for Claude Code with Claude Opus 5.5, and 71% against 46% for Claude Code with Claude Sonnet 5.5. It also used 42 to 62% fewer input tokens per solved task. The [benchmarks](https://barestate.xyz/oh-my-sol#benchmarks) show every model.

OMS runs on macOS and Linux, or Windows under WSL 2, with Node 22.19.0 or newer and Git. You bring your own model access.

## Frequently asked questions

Why does my AI coding agent say it is done when the Solana tests still fail?

Because in most agents the model itself decides when the work is finished. It can stop after a build that compiles, after tests it never ran, or after editing a test so it passes. The fix is to let something other than the model decide: a command that must exit 0, run by a hook, CI or a separate checker.

How do I keep Claude Code working until my tests pass?

Add a Stop hook that runs your test command and exits with code 2 when it fails. Exit code 2 on a Stop hook prevents Claude from stopping and continues the conversation. Check the stop_hook_active field so a failure that will never resolve does not loop forever.

What does /goal do in Oh My Sol?

/goal <text> --check "<command>" tells Oh My Sol (OMS) to keep working until that check passes, or until a cap stops it. It stops when the same failure repeats twice in a row, when attempts reach the cap (5 by default), or when the budget runs out. /goal clear removes the goal.

Can the model in Oh My Sol mark its own task as done?

No. The model’s only completion tool is propose_done, which moves a requirement from active to proposed. A separate checker closes it, and only when its checks pass against a fresh offline Surfpool chain. The checker also refuses to run if the pinned contract or verifier files were changed after the task was created.

What does "verified" mean?

It means the pinned checks passed for a saved copy of the code on a fresh chain. The checks are yours, so write them for the behavior you care about and review the diff before you merge.

How much can a single Oh My Sol task spend?

The defaults are 2,000,000 tokens, $5 of API spend, 30 minutes and 32,768 output tokens per response. A normal interactive launch has no idle expiry; pass --bounded to apply the caps to the whole session. /usage and /status show the numbers as you go.

## Try it on your own program

Oh My Sol 0.1.0 runs on macOS and Linux, and on Windows through WSL 2. You need Node 22.19.0 or newer and Git. You bring your own model access: an API key, a ChatGPT subscription or a Claude subscription.

```
npx @barestate/oms doctor --install
npx @barestate/oms
```

[Read the docs](https://barestate.xyz/docs) · [See the benchmarks](https://barestate.xyz/oh-my-sol#benchmarks)
