178 Green Tests, 100% Coverage, Crashes on First Connection

# A Paradox

178 green tests. 100% coverage. 100% broken in production.

All three facts were true at the same time. That’s what a late-night commit (2026-06-18) in the DeepSeek Harness repository recorded. The commit message opens with this:

Two independent bugs made the ACP server crash the moment an editor (Zed) connected, despite 178 green unit tests at 100% coverage.

Connect Zed editor, send the first request, get back cannot get property "agents" without inject. Every test green, every line covered, feature dead on arrival.

This wasn’t a testing discipline problem—the repo enforced per-file 100% coverage gates from day two. The problem was elsewhere: the test world and the real world were disconnected.

Every test hides an assumption no one verified. I assumed manual plugin mounting equals real loading. I assumed calling from the test top-level equals the real call topology. I assumed the test ran against current code, not stale builds. 178 green tests mean those 178 assumptions held in the test world. Feature crashes mean those assumptions broke in the real world.

Three disconnects, three independent failure modes. All from the repo’s postmortem 0001, with commit SHAs you can verify.

# First Disconnect: Manual Mounting ≠ Real Loading

Start with what broke. The ACP server—a JSON-RPC bridge exposing agent capabilities through Agent Client Protocol—crashed on the first editor connection. The direct cause looks trivial in hindsight: one extra line at the end of packages/acp/acp/src/index.ts.

export default apply   // ← the bug

In this codebase, namespace plugins export name, inject, Config, and apply as separate named exports—every other plugin does it this way. But this file added a default export, which triggered Cordis Loader’s normalization logic (vendor/loader/src/index.ts):

unwrapExports(exports: any) {
  if (isNullable(exports)) return exports
  exports = exports.default ?? exports   // ← prefer default when present
  ...
}

When a default export exists, the entire module namespace collapses to the bare function. The inject declaration vanishes. The plugin builds with an empty inject, and the first line of apply trying to read ctx.agents throws during load.

Here’s the key: this bug cannot reproduce in unit tests by design. All 178 tests manually assemble plugins with ctx.plugin({ name, inject, apply })—the test provides inject itself, so unwrapExports never runs. The test world and the shipping world diverge from line one.

# Second Disconnect: Top-Level Call ≠ Real Topology

Delete that export line, session/new works—but session/load still crashes. Same error string. Completely independent second bug.

AgentLoop’s static inject deliberately excludes sessionPersistence—including it would make non-persistence demos wait forever for a backend that won’t arrive. So resume() reads directly: this.ctx.sessionPersistence, uses it if present, ignores if absent. The problem is the read itself. At runtime, this read goes through Cordis’s context proxy (vendor/cordis/src/reflect.ts), which looks up services by walking only ancestor fibers. sessionPersistence isn’t on an ancestor—it’s on a sibling branch. Walking up from AgentLoop’s fiber to root never encounters it.

Why did tests pass? Because tests call ctx.agents.resume(...) directly from the top level, outside any plugin fiber. The proxy takes a completely different shortcut—queries the global service store directly, ignoring topology, finds the service.

Same line of code: called from test top-level, passes; called from a real plugin fiber through the proxy, fails. Tests thought they verified the call. They verified a different world’s call instead.

# Third Disconnect: Stale Build ≠ Current Code

There’s a third layer, the most hidden one. The only test in the repo that actually drives these two RPCs requires an API key—CI has no key, silently skips it. The local keyed test “passed” once; investigation revealed it was fake: the test ran in a temp directory where tsx couldn’t find the repo root’s tsconfig paths, silently fell back to stale build artifacts in lib/. Old code had no bug, so green light. Test thought it was testing new code. It tested old builds.

Three disconnects, each breaking differently: first on the loading path, second on call topology, third on time. But the blindness mechanism is the same—a test’s “pass” only proves the test’s own world is self-consistent. It cannot prove the test world matches the real world.

# Where Coverage Fits

So what does 100% coverage measure? It measures every line was executed—that’s true, no lies there. It doesn’t measure “executed the way production executes it”—also true. These two statements don’t contradict, which is why “100% coverage” and “100% broken” can both hold.

The repo’s AGENTS.md codified this after the postmortem: line coverage is not behavior coverage.

The fix didn’t stop at deleting bugs. Added a keyless, real-Loader, real-stdio e2e for session/new—and the postmortem documented the acceptance test: add export default back, this test must go red. A regression test’s credential is turning red when the original bug comes back.

# The Most Valuable Moment in the Investigation

One moment from the investigation is worth isolating. The team initially suspected Cordis’s traceable/shadow mechanism—that suspicion was actually correct (it was the culprit for the second bug) and perfectly explained the error. They followed that elegant theory for hours. The turn came after instrumenting fiber walks and running one real subprocess: the trace showed the throw happened on the ROOT fiber, no shadow at all. Theory falsified on the spot. The real culprit—that one export line—surfaced minutes later. The postmortem says:

Trust the trace, not the theory. … a fiber-walk console.error found in minutes after hours of plausible-but-wrong reasoning.

The lesson isn’t just “add logging.” It’s deeper: when you have a theory, an agent explanation, and persisted event logs, only the last one is a trace. The first two share a property—they help you “rationalize” what you’re seeing. In this incident, a correct theory (shadow mechanism) delayed finding a simpler truth (one extra export). A right theory applied to the wrong object is more dangerous than a wrong theory.

# Why This Matters More in the AI Era

Stories like this are getting denser. The recent pgrust project—two people using AI to rewrite PostgreSQL in Rust—passed all 46,066 official regression tests, then got broken by fuzzing in one afternoon. 46,066 tests and 178 tests are two sides of the same coin: one proves “official tests all pass” doesn’t stop fuzzing, the other proves “100% coverage” doesn’t stop the first real connection. Tests measure self-consistency in their own world. They never measure isomorphism with the real world.

When tests and code come from the same context, verification easily degrades into repetition. The most comfortable path for an agent writing tests is exactly these three disconnects: in-memory stubs are fastest (first disconnect), top-level calls are most certain (second disconnect), running on existing builds is cheapest (third disconnect). Agent-generated tests and agent-generated code share the same assumptions—it doesn’t know to question “equivalence” because “equivalence” is what it wrote down.

The rules don’t need to be complex. Three:

1. Every feature needs at least one test through the real entry path—real loader, real process, real stdio. Acceptance: manually restore the known bug, it must go red.
2. Audit every silent fallback in the test environment—stale builds, skipped key gates, mocked loaders. Every silent fallback is an unsigned assumption.
3. When debugging, read the trace first, then listen to explanations—including your own, including the agent’s.

178 green tests, 100% coverage, 100% broken in the real world. When these three 100%s hold simultaneously, coverage becomes a placebo—it proves code was run, not that code was run correctly.

# Evidence Anchors

This article is an independent analysis of an open-source repository under MIT license, in developer preview with rapid iteration. Line numbers and paths are anchored to the main branch as of 2026-08-14.