Rationale

I maintain dspy-go, a Go implementation of DSPy. Every so often, I need to update the capabilities of the underlying LLM layer: add vision, support another provider, expose new request controls, or preserve more provider evidence.

I eventually moved pkg/llm into a dedicated library, llm-go. When dspy-go first came out, there was not much development around LLM clients in Go. Splitting the package let dspy-go stay focused while the provider layer evolved on its own.

After the port, the main question was how to battle-test it. dspy-go was the only substantial consumer, which meant many bugs would only appear when I changed dspy-go alongside it.

Around then, Tau, a Python port of Pi, came to my attention. That led to an obvious question: why not build my own coding agent? I could exercise llm-go through real tool use while also adding the agent features I find useful.

That became Mu.

Mu

Building the first version of an agent is fairly straightforward. At a high level, what I wanted Mu to provide out of the box was:

  • Self-evolving capabilities
  • Client/host separation
  • Durable operations
  • Get rid of lossy compaction
  • Background communication support
  • Useful debugging so I can inspect what actually happened

The checklist is short, but each item changes the shape of the runtime.

There is a lot of hype around recursive self-improvement (RSI), but what does it mean for my personal harness? Systems such as Hermes Agent, which can create and refine skills from experience, are one approach. What other directions could a harness take? Could a dedicated controller agent manage that evolution?

Client/host separation means the UI is not the owner of the work. I should be able to detach a client without killing the session, reconnect later, and let one workspace host own multiple isolated sessions.

Durable operations mean that a model request or tool call is not merely a line printed to a terminal. Its admission, exact arguments, result, usage, and terminal state need to survive process boundaries. Recovery should continue from committed facts rather than guesses.

“Get rid of compaction” does not mean sending an infinitely growing prompt. It means keeping the journal as the authoritative history instead of replacing old work with a lossy summary. The runtime can select or retrieve context under pressure, but the original evidence remains available. This is similar in spirit to one of Codex’s newer experimental features.

Background communication is the unfinished part. Launching work is easy; giving it clear ownership, cancellation, completion, and restart semantics is where it becomes a lifecycle problem rather than another tool.

The debugging requirement has probably shaped Mu the most. Provider requests, stream settlement, token usage, tool bytes, recovery decisions, and continuity choices should be inspectable. Without that evidence, it is too easy for an agent—or its author—to declare success based on a final message.

Benchmarking

Inspired by Alma’s Kimi K3 result, I used the frozen 30-task FrontierHarness Eval v1 as a local stress test to see where Mu stood. The point was not to publish a leaderboard entry. I wanted to see whether the Mu runtime and llm-go provider path could survive long, unfamiliar repository work and still leave enough evidence to explain failures.

Mu's provisional position in FrontierHarness Eval

Mu’s orange star is a provisional local observation, not a rank. The published baseline frontier does not include Mu.

Setup

  • FrontierHarness Eval commit: e837a70
  • Mu commit: 246c76c
  • DeepSWE commit: 435ee89
  • Model: Kimi K3 through Fireworks
  • Harness adapters: Harbor 0.22.0 and Pier 0.3.1
  • Workload: 9 DeepSWE tasks and 21 Terminal-Bench tasks
  • Execution: serial, with no automatic retries
  • Per-task ceiling: $12
  • Mu mode: native Chat Completions with continuity disabled

The adapter called Mu’s coding runtime directly. This exercised model requests, tools, editing, provider accounting, and long agent loops. It did not exercise the interactive TUI, the full live-host lifecycle, or Mu’s continuity path.

DeepSWE task containers had no task internet access; Pier allowed only api.fireworks.ai for the model path. Terminal-Bench containers used public egress. Every task ran in a fresh Docker container, but Docker Desktop replaced the benchmark’s Runta checkpoint infrastructure.

Result

With the explicitly authorized retry of task 8 included, Mu passed 23 of 30 tasks:

Suite Passed Total
DeepSWE 6 9
Terminal-Bench 17 21
Overall 23 30

The seven failures were:

  • arktype-json-schema-refs-dependencies
  • expr-try-catch-errors
  • meriyah-explicit-resource-declarations
  • chess-best-move
  • dna-insert
  • gcode-to-text
  • largest-eigenval

Three of the DeepSWE failures were near misses under binary grading. The ArkType patch passed 23/25 new feature tests and all 1,679 regression tests. The Expr patch passed 74/79 feature tests and all 66,265 regressions. The Meriyah patch passed 45/49 feature tests and all 51,469 regressions. These were useful engineering signals, but each task still received zero reward.

largest-eigenval hit its 900-second agent timeout. Its twenty-fourth model request had started streaming when Harbor canceled the agent, so the final usage record was never observed.

Cost

The cost numbers need more explanation than the score:

Number Value Meaning
Selected-outcome cost at least $29.5090 Frozen repricing for the selected 30 outcomes, including the settled lower bound for the timed-out task
Website-aligned chart metric at least $1.2830/pass Available selected-outcome cost divided by 23 passes
Settled usage across all attempts at least $32.0897 Includes the first, unscored attempt of task 8; task 20’s final request remains unknown
Conservative control-ledger charge $42.5187 of $51 Includes full-cap charges when accounting failed closed

The chart inherits the repository’s Median cost per task axis label, but that is not the statistic it plots. The website-aligned implementation divides the sum of available task costs, including failures, by the number of passes. I kept the label to make the generated chart match the source repository, and I am calling out the mismatch here.

Local Caveats

This run is local, unranked, and non-comparable to the published leaderboard:

  1. Docker Desktop replaced fresh Runta checkpoint restores.
  2. Linux/amd64 images ran under emulation on an arm64 Mac.
  3. Network policies differed, and the published baselines did not retain enough egress evidence to establish parity.
  4. The aggregate authorization grew from $21 to $41 and then $51. That changed how much budget later tasks could reserve.
  5. Task 8’s first attempt stopped under a reduced remaining cap. I explicitly reran it with a $12 cap and selected that passing result.
  6. Task 20 timed out with one unsettled model request, so its cost is a lower bound.
  7. Most tasks were run only once, so there are no variance estimates.

The defensible claim is therefore narrow: Mu passed 23 of 30 tasks under this local, manually continued setup. It is not evidence that Mu ranks above Codex, Pi, Alma, or any other published harness.

Acknowledgements

Mu did not emerge in isolation. Several agents and harnesses gave me concrete ideas to study, test, and disagree with:

  • Pi showed how small and inspectable a capable coding-agent loop can be. Its tool surface and extension model were useful references while I was deciding how much machinery Mu actually needed.
  • Amp set a high bar for the interactive experience. It influenced how I think about keeping an agent fast to use even when the runtime underneath it is doing more durable work.
  • Codex provided an unusually valuable open implementation to compare against, especially around context management, tool execution, patch application, and session lifecycle.
  • Maka made the idea of the log as the runtime concrete. Its work on projections, archived tool results, context pressure, and supervised execution helped sharpen Mu’s durability design.
  • Hermes Agent gave me a concrete reference for self-improvement at the harness level: creating and refining skills from experience, persisting useful knowledge, and searching past sessions. It helped me think about RSI as an inspectable feedback loop rather than an abstract model capability.

Thank you to the authors and contributors of all five projects for making their ideas, code, and experiments available. Mu’s implementation choices—and its mistakes—remain my own.

What I Am Taking Away

The coding loop was the easy part. The harder and more interesting work is around it:

  • which effects are safe to replay
  • how history remains authoritative under context pressure
  • how background work joins or cancels
  • how costs remain bounded when a stream ends ambiguously
  • how a failure leaves enough evidence to diagnose itself
  • how the harness can improve itself using evidence and lessons learned

That is the harness I want Mu to become.