PRESENTED BY OUMI
Your AI stopped learning the day it shipped
A general-purpose model doesn’t learn from your production failures, operator corrections, or the work that makes your business different.
On August 11, Oumi is launching the AI Factory that runs the complete loop: Evaluate, Synthesize, Train, Deploy, Compound. We’ll show it live on a real task and reveal three product announcements.
Join Manos Koukoumidis and the Oumi team tomorrow at 10:00 AM PT / 1:00 PM ET.
We are now building complex harnesses around AI agents. Imagine a system prompt says “always verify the result.” A CLAUDE.md says “run the tests before reporting success.” A skills file lays out a 12-step procedure. Then there are tools, memory, guardrails, evaluation loops, and instructions that keep getting longer.
We put all of this into the model’s context, give it the task, and assume the agent will follow it. But a document is not an enforcement mechanism. That matters more as agents run longer, act more autonomously, and start changing the systems around them.
A set of papers published over the past month points at the same underlying problem.
Agents skip rules that exist only as text.
Agents can improve against tests they also control.
Agents can throw away information that later steps still need.
Three different research areas but has one thing in common: We are giving the model responsibility for things the harness should be responsible for.
That changed how I think about agent design.
1. Start with the rules
Consider a simple instruction: “Do not report success until all tests have been run.” To a human engineer, this is a rule. But to a model, it is another piece of text in a large context.
Nothing in the runtime prevents the agent from skipping the test, changing the order of the steps, or deciding that a different check is good enough.
SIGIL tested this across 30 agent skills. When the procedures were given to the model as prose, GPT-4o executed 56% of the required steps. GPT-5 improved that to 68%. The complete procedure was followed in only 28% of runs. HANDBOOK.md also tested agents against operating manuals ranging from 20 to 124 pages. The best configuration passed 36.2% of trials when every requirement had to be satisfied!
Some of the most frequently skipped steps were verification steps.
That creates an uncomfortable situation. If the agent skips a verification, there is no verification failure to report. The result can look fine even though part of the procedure never happened.
So the problem is not that models sometimes disobey instructions.
The problem is that prose has no enforcement power.
SIGIL takes a different approach. Instead of asking the model to interpret and remember every part of the procedure, it turns the parts that can be enforced into executable code.
Take the test example again.
In prose, we write “run all tests before reporting success.”
In the harness, the runtime runs the tests and stores the result. The success path checks that result. If the tests did not run, the agent cannot reach the success state.
The model can still decide what to do next. It cannot decide whether a mandatory, machine-checkable action happened.
With this approach, SIGIL reports 86% (up 30%) compliance with mandated steps, and the full procedure is completed 2.3 times as often.
This is also why using a better model does not solve the problem.
A stronger model may follow instructions more often. But you are still depending on the model to remember and execute a rule the system could have enforced directly.
None of this means prompts are useless. Prompts are where we express intent, explain goals, and leave room for judgment. But if a rule is important enough that failure is unacceptable, we should ask why its enforcement is left to the model.
2. Then we gave the agent control of the scoreboard
Once some responsibilities move out of the model and into the harness, a second question appears. Who controls the harness? Increasingly, the answer is the agent!
A common way to improve an agent is to let it modify its own harness. It changes the instructions, writes tests, runs evaluations, reads the results, and tries again.
At first this sounds reasonable.
But the agent is now changing the system while using that same system’s score to decide whether the change worked. Remember the recent OpenAI incident where their agent breached HuggingFace infrastructure!?
Rethinking the Evaluation of Harness Evolution asks a simple question about this. If harness evolution is itself a search process, how much of the improvement comes from a better harness, and how much comes from giving the model more chances to search? Under the same compute budget, harness evolution does not consistently win. In some settings, plain test-time scaling does better.
That does not make harness evolution useless. It means part of the reported improvement may be search rather than a genuinely better agent.
There is a stranger failure mode.
Phantom Guardrails gives an agent a failure that does not exist and asks it to build a guardrail for it. The right answer is to conclude there is nothing to fix.
In 15 of 60 runs, the agent invented the failure and built the guardrail anyway!
That is what happens when an optimizer is rewarded for finding something to improve. It can start producing the evidence that gives it something to work on.
So the fix is not to make the agent better at evaluating itself. Sometimes it is to take the evaluation away from it.
SEAL does that. The agent can write its own tests and propose changes, but a fixed check outside the agent decides whether the new version is accepted. The agent cannot inspect or rewrite that check. It gets an accept or a reject.
The agent proposes. The harness decides.
This is the same lesson as the rules section, one level up. There, we did not want the model deciding whether a mandatory test had happened. Here, we do not want it deciding whether its own change was an improvement.
There is one more responsibility we have not looked at.
3. And then we let it decide what to forget
An agent has been working for 30 minutes. Its context holds tool calls, files, test results, decisions, intermediate plans, and previous mistakes.
Eventually it gets too large.
The easy solution is to truncate the beginning.
But truncation is not memory management. You are not asking what information is no longer useful. You are asking what you can delete because you ran out of space. Those are different questions. And an agent can only make a good decision if it still has the information that decision depends on. Which makes context another harness responsibility.
Self-GC treats context as a collection of objects rather than one long transcript. It can fold, mask, or remove parts of the history while keeping recoverable copies and safe points.
It prunes 43.95% of prefix tokens while leaving 84.85% of later continuations unaffected. In a production-derived evaluation, its no-impact rate reached 91.27% to 94.58%, compared with 77.71% to 87.46% for the baselines. In production, it reduced average daytime input tokens by 10% to 15%.
The metric that matters is not how many tokens were removed. If deleting a tool result saves 1,000 tokens but the agent fails ten steps later, it was not a good deletion.
The question is what the next step still needs.
CompactionRL takes this in another direction by training the model to compact its own history during reinforcement learning. On SWE-bench Verified, it improved Pass@1 by 7.0 points for GLM-4.5-Air. So context management may eventually be partly learned by the model.
For now, context is state, and state is part of the harness. Let’s summarize today’s discussion!
Ask these four questions before shipping an agent
1. Which rules are mandatory? Move enforcement out of prose and into executable checks where possible.
2. Who controls the scoreboard? Do not let the system being optimized rewrite the mechanism that decides whether it improved.
3. What state does the next step need? Judge compaction by downstream task success, not by tokens removed.
4. Which decisions actually require judgment? Keep those with the model. Do not encode a decision in rigid code just because it can be encoded.
Agents are going to keep getting longer prompts, better tools, larger memory, and more freedom to modify their own instructions while nobody is watching.
That makes the boundary between the model and the harness more important. The goal is not a harness that tells the model everything it must do. It is a harness that guarantees the things the model should not be trusted to guarantee itself.
Let the model decide and let the harness enforce.
Very Insightful, Thank you!
Soon we’ll create a formal way of specifying what we want the ai to do and a harness that controls it.
Congratulations, you’ve reinvented programming languages and compilers but made them shittier.