1Washington University in St. Louis · 2Google Cloud AI Research
3Google Cloud · 4University of North Carolina at Chapel Hill
*This work was done while Chengsong interned at Google Cloud AI Research · †Corresponding authors
Agents got a harness. Now the environment gets one.
Building environments is expensive, and once built they sit still: the same tasks, the same feedback, for every agent forever. Instead of generating new environments, EnvHarness customizes existing ones through the standard reset() / step() interface alone. Every customized task inherits the trusted, human-built verifier of its source benchmark. Three component types cover three kinds of control, and because each preserves the interface, they compose freely: E″ = w₂(w₁(E)). You’ll meet all three, as live, editable code, in the playground below.
Both scale by adding external layers, so the core system never changes. Same trick, other side of the loop.
The task is the 24 game: combine the four cards with + − × ÷, two at a time, until one number is left. Make it 24 and the episode is solved. Play a hand, then open the tabs on the right. The line under each editor says what that component customizes. Edit one, apply it to the stack, and compare the two panels on the left: what the policy observes, and what the environment actually holds.
An LLM designer agent runs the loop below. It watches the policy interact with the environment, diagnoses a systemic flaw in the resulting trajectories, writes a component that targets it, and stacks the candidate provisionally so the same policy can be tested in the customized environment. Components that teach are kept; the rest are revised. The reset()/step() contract, the tasks and the verifiers are untouched throughout.
$ edit sklearn/impute.py $ git commit -m "fix SimpleImputer" → submit patch # tests never run ✗ FAIL: hidden test breaks
Rollouts come back as trajectories τ, successes and failures alike.
“Systemic weakness: the agent submits patches without running the failing test first. Success depends on luck, not verification.”
Diagnosis targets the pattern, not one bad episode.
class _Contract(Contract): def modify_transition(self, action, response, env_state): cmd = bash_command(action) if "pytest" in cmd: env_state.extras["ran_tests"] = True if is_submission(cmd) and not env_state.extras.get("ran_tests"): return failed(response, "pre-commit hook failed: run the test suite before submitting.") return response
A plug-in layer over the frozen env, with tasks and verifiers untouched.
rollout 3/8 · submit → blocked: "run the test suite…"
rollout 3/8 · $ pytest tests/ … 1 failed
rollout 3/8 · fix → pytest ✓ → submit ✓
verdict: teaches verification → ACCEPT
Same interface, new lessons. The policy trains on it, and the cycle turns again.
The pattern above holds across policy models. Skills mined in EnvHarness environments beat both baselines for every model we tested, on the same held-out tasks and with the same verifiers.
The gain over original-env skills stays nearly constant, 2.9 to 3.7 points, while the no-skill baselines span 30.7 to 67.2. The designer shares the policy model in every setting.
Plus nine targeted-weakness case studies across three benchmarks, and the full formalism for why components compose.
Work on self-evolving agents has agents rewriting their prompts, growing skill libraries and editing their own scaffolds. In nearly all of it, the world the agent evolves against stays fixed.
An agent that keeps improving against a frozen benchmark eventually stops learning from it. The start states, the rules and the feedback are the same for every learner at every stage, so once the agent can solve the tasks, the environment has nothing further to teach. The missing piece is not a better agent but control over the environment it learns in.
EnvHarness supplies that half the way the agent harness did for the model: a frozen core that external layers make capable. Every component preserves the reset()/step() contract and inherits the benchmark’s human-built verifiers, so reshaping is cheap, trustworthy, and composable. An LLM designer does it automatically, against the specific policy being trained.
The result is a loop that evolves on both sides of the interface: the environment targets the learner’s current weaknesses, serves difficulty on demand, and co-evolves round after round as the policy improves, climbing 47.7 to 54.8 on SWE-bench over three rounds. We ship three component types; the interface admits many more.
@inproceedings{envharness2026,
title = {EnvHarness: Awakening Static Worlds for Agent Learning},
author = {Anonymous Authors},
booktitle = {Under review},
year = {2026},
}
The full framework is open source. The repository contains the ActionableEnv contract, the per-benchmark Bridges, the three component types, and the designer agent, with presets for every benchmark on this page. The README covers setup and how to reproduce each figure.