---
title: "Oh My Sol · Barestate"
description: "A terminal app for building on Solana with the model you already use. References, a local chain and toolchain checks are in the box."
url: https://barestate.xyz/oh-my-sol
---

# The Solana harness, batteries included

A terminal app for building on Solana with the model you already use. References, a local chain and toolchain checks are in the box.

[Explore the harness](https://barestate.xyz/docs) · [How it works](https://barestate.xyz/oh-my-sol#how)

```
npx @barestate/oms doctor --install
npx @barestate/oms
```

## How it works

Models were trained on older Solana, and the details they get wrong are usually the ones that changed since. Oh My Sol keeps the current references, program error decoding and toolchain checks next to the model while it works, and gives it a local Surfpool chain to run your program against.

Each task runs in its own Git worktree. When the model thinks it is finished, OMS runs your requirements on a fresh chain before the task closes, and if they fail, the model tries again within the budget you set.

## Your keys stay out of reach

On macOS, every command OMS runs goes through a Seatbelt sandbox. It blocks reads from your home folder, apart from the toolchains OMS needs, and it always blocks keypair files and folders like `~/.ssh` and `~/.config/solana`, so a command the model writes can’t read your keys even if it tries.

## Benchmarks

Solana Bench scores an agent by how many distinct Solana instructions it gets to execute successfully on a mainnet fork. On the same model, GPT-6.1 Sol, Oh My Sol scored higher than Codex CLI in both the basic and the swap environment.

**Solana Bench scores**

*Mean score, higher is better. OMS and Codex CLI ran GPT-6.1 Sol and Claude Code ran Claude Opus 5.5, in the same batches on one VM: basic 8 OMS and 6 Codex CLI, swap 16 and 12, and 2 Claude Code runs in each. Permutation test on the difference of means: p = 0.011 (basic), 0.010 (swap).*

It also got there on less. In the basic environment OMS used 18.2K total input tokens for every point it scored, against 33.7K for Codex CLI, which is 46% fewer.

Models are trained on older Solana, so we also built eight tasks around changes newer than the models themselves. On those, Oh My Sol passed more runs than each model’s own harness on every model we tried.

**Pass rate on fresh Solana tasks**

Runs passed. GPT-6.1 Sol: OMS 92 of 96, Codex CLI 73 of 96. GPT-6 Astra: OMS 22 of 24, Codex CLI 17 of 24. Claude Opus 5.5: OMS 22 of 24, Claude Code 19 of 24. Claude Sonnet 5.5: OMS 17 of 24, Claude Code 11 of 24.

*Same model, same eight tasks and the same hidden check for both harnesses.*

The gap holds up run by run. If you line up each task and run and keep only the pairs where one harness passed and the other failed, it is almost always OMS that passed.

**Matched runs that only one harness passed**

*Runs are paired by task, run and batch. Pairs that both harnesses passed, or both failed, are left out.*

It also reads less to get there: 42 to 62% fewer input tokens for every task it solves than each model’s own harness.

**Input tokens per solved task**

*Total input tokens across all runs, cache reads and writes included. Lower is better.*

## Run it on your machine

Oh My Sol 0.1.0 runs on macOS and Linux, and on Windows through WSL 2. You need Node 22.19.0 or newer and Git, and the first command is a one-time setup that downloads Surfpool and checks your toolchain.

```
npx @barestate/oms doctor --install
npx @barestate/oms
```

[Explore the harness](https://barestate.xyz/docs) · [Join the waitlist](https://barestate.xyz/oms/waitlist) · [Sign in](https://barestate.xyz/oms/login)

## Benchmark data

The same figures as JSON, with units: [GET https://barestate.xyz/api/benchmarks](https://barestate.xyz/api/benchmarks). Methodology: [https://barestate.xyz/docs/benchmarks/methodology](https://barestate.xyz/docs/benchmarks/methodology).

### Solana Bench scores

| Statistic | OMS (GPT-6.1 Sol) | Codex CLI (GPT-6.1 Sol) |
| --- | --- | --- |
| Basic, mean score | 377 | 273 |
| Swap, mean score | 366 | 283 |
| Basic, median score | 346 | 284 |
| Swap, median score | 325 | 282 |

*Mean score, higher is better. OMS and Codex CLI ran GPT-6.1 Sol and Claude Code ran Claude Opus 5.5, in the same batches on one VM: basic 8 OMS and 6 Codex CLI, swap 16 and 12, and 2 Claude Code runs in each. Permutation test on the difference of means: p = 0.011 (basic), 0.010 (swap).*

### Pass rate on fresh Solana tasks

| Model | OMS passed | Own harness | Own harness passed |
| --- | --- | --- | --- |
| GPT-6.1 Sol | 92/96 (96%) | Codex CLI | 73/96 (76%) |
| GPT-6 Astra | 22/24 (92%) | Codex CLI | 17/24 (71%) |
| Claude Opus 5.5 | 22/24 (92%) | Claude Code | 19/24 (79%) |
| Claude Sonnet 5.5 | 17/24 (71%) | Claude Code | 11/24 (46%) |

*Same model, same eight tasks and the same hidden check for both harnesses.*

### Matched runs that only one harness passed

| Model | Compared with | Only OMS passed | Only the other passed | Sign test p |
| --- | --- | --- | --- | --- |
| GPT-6.1 Sol | Codex CLI | 21 | 2 | 0.0001 |
| GPT-6 Astra | Codex CLI | 5 | 0 | 0.0625 |
| Claude Opus 5.5 | Claude Code | 3 | 0 | 0.2500 |
| Claude Sonnet 5.5 | Claude Code | 6 | 0 | 0.0312 |

*Runs are paired by task, run and batch. Pairs that both harnesses passed, or both failed, are left out.*

### Input tokens per solved task

| Model | OMS | Own harness | Difference |
| --- | --- | --- | --- |
| GPT-6.1 Sol | 134,945 | 230,696 (Codex CLI) | 42% fewer |
| GPT-6 Astra | 131,487 | 267,644 (Codex CLI) | 51% fewer |
| Claude Opus 5.5 | 183,079 | 475,987 (Claude Code) | 62% fewer |
| Claude Sonnet 5.5 | 198,726 | 488,286 (Claude Code) | 59% fewer |

*Total input tokens across all runs, cache reads and writes included. Lower is better.*