# The most valuable thing an agent did for me was not code

> The fix was four lines, but the value came before it. I used coding agents to reproduce reported behavior, build test setups, measure real app flows and disprove their own hypotheses.

- Author: Jonas Rohde (@edhor)
- Published: 2026-08-31
- Canonical: https://edhor.me/blog/the-most-valuable-thing-an-agent-did-was-not-code/
- Tags: ai-agents, agentic-coding, debugging, performance, testing

The most valuable thing an agent did for me was not code.

Someone reported that an offline-capable app did not start properly without a
network connection. That was weird. Offline use was one of the cases the app
was built for.

So I let the agent work like a lab. It built the release APK, drove the
emulator, toggled airplane mode and wrote a small proxy which killed only the
API host while keeping the rest of the network alive. Then it dumped the
on-device cache and compared screenshots.

The first finding was not the actual reason. But it found the location of the
problem in one run.

A compression function produced unpaired UTF-16 surrogates for the cache,
while the actual store persisted UTF-8. The invalid surrogates disappeared and
so did the cache.

The fix is four lines. The value was before it.

## TL;DR

* I did not use the agent to write code or search for a possible reason → I
  used it for the complete diagnosis
* Give the agent a wide investigation scope without giving it an equally wide
  implementation scope
* Define repeatable evidence before it starts changing things
* Autonomous execution still needs a human deciding what counts as proof
* If you don't want agents writing production code yet, let them reproduce,
  measure and disprove

I didn't use the agent to write code or just search in the codebase for a
possible reason. I used it for the complete diagnosis.

And once that worked, I wanted to know how far I could take the method.

## Start with a measurement contract

The next problem was about performance. An action in a mobile app could take up
to ten seconds under load.

My own suspicion pointed mainly at the native integration layer. That turned
out to be only part of the story.

The useful part of my prompt was not the suspicion. It was the definition of
evidence. I wanted Android, the real app, a local backend, a production-shaped
dataset, idle and load scenarios, and the same measurements before and after a
change.

My instruction was roughly: continue step by step until the run is complete,
and save every result.

This gave the agent a wide investigation scope without giving it an equally
wide implementation scope. It could build fixtures, add measurements, drive
the app and disprove my hypothesis. It could not turn every interesting thing
it found into a product change.

That boundary matters. Investigation benefits from freedom. Production code
benefits from a very clear reason to exist.

## What the first run found

The first issue was database contention during background work.

With a large local dataset, the background process started one complete job per
selected group. The concurrency limit I expected to protect us did not apply
because these requests used synchronous `execute()` calls.

The lookup itself was fast while the system was idle. Under active background
work, about 79.7% of its time was spent waiting behind concurrent database
work. Limiting that work to three jobs changed the lookup p95 from about 110 ms
to 11 ms. The maximum fell from 157 ms to 14 ms.

That was already useful, but it did not explain all of the reported delay.

A physical device reported a successful hardware input while the first
JavaScript handler sometimes ran seconds later. So I attached the device and
asked the agent to go as deep as it did before.

The agent injected package-scoped test events through Android debugging tools.
They followed the same native receiver and event path as the hardware input,
but stopped before creating application data. This made 45 identical probes
per run possible.

The hardware was not the problem. The native event emission took 0.21 ms.
Outside a stall, the event reached JavaScript in 1 to 3 ms. During cache
persistence it waited behind `compressToUTF16`, which blocked the JavaScript
thread for an average of 8.22 seconds on the large fixture. Writing the result
to storage needed about 21 ms.

With the existing serializer, 25 of 45 probes waited at least one second and
the maximum was 8.06 seconds. With the same data and raw JSON, the p95 fell
from 7.66 seconds to 5 ms. Not one probe waited a second.

That result did not come from reading the compression code and calling it
expensive. The agent had to prove where the input was waiting, separate dataset
shape from payload size, and run the same input through both implementations.

## Repeat the method, not the conclusion

After the first run I created a handoff and started a second investigation in
another mobile app.

The prompt was intentionally broad: take over the complete local end-to-end
investigation, build whatever profiling and fixtures are needed, and stay
local.

The important part was what the agent did *not* do. It did not treat the first
result as proof for another app. It treated it as a hypothesis.

On a physical device, compressing a production-shaped 1 MB cache took about
2.66 seconds.

Separately, a system trace on a recent Android test target showed the
JavaScript thread running on CPU for 98.41% of the measured compression
interval. That ruled out storage I/O and lock waiting.

Still not enough.

I asked what a user would actually notice. Was this work effectively in the
background, or did it block something people do? I asked for a deterministic
difference in a real app flow, not only a theoretical serializer number.

That changed what counted as finished.

The agent built a local authenticated flow, made a real tap collide with the
persistence window, and measured native touch to the JavaScript handler to the
first visible frame.

These Android test targets are a separate evidence lane from the physical
device measurement above.

Across an older and a newer Android target, compression added hundreds of
milliseconds before the tap handler ran. On the older target the complete
interaction took more than a second. Switching to raw JSON removed around 96%
of the wait before the handler. The later rendering time barely changed on the
newer target. The tap itself had been waiting behind compression.

The final product fix removed 12 lines across three files.

Again, the value was before it.

## A low-risk place to start

If you don't want to use agents for production code yet, this is a very
low-risk place to start: let them reproduce, measure and disprove.

A wrong hypothesis costs you a rerun, not a rollback.

Take a bug you understand well. Ask the agent to run the real app, reproduce
the behavior and show you exactly where its explanation comes from. Let it add
temporary instrumentation. Let it drive the simulator or the browser while it
checks the logs and console. Ask for the same scenario before and after the
change.

And if it hits a limit, don't stop at "the agent cannot do this." What is
actually missing? Is the model not capable enough? Is access missing? Does the
app have no test data? Is there no way to observe the behavior?

In one investigation the agent could not test an external integration locally,
so I asked it to build a mock server from the available protocol
documentation. For the performance work I let it add a lot of logs and
measurements because the code alone could not tell us the runtime behavior.

Think about what you would do next, then ask the agent to do it for you.

## Autonomous does not mean hands-off

The two sessions delegated dozens of tasks across multiple agents and
subagents. Together the agent runs added up to several hours of active work.
Because they ran in parallel, that was not the same amount of wall-clock time.

The number sounds interesting. It was not the reason this worked.

The investigation had a measurement contract, and I kept changing that
contract when the evidence was too weak. I asked for another check when a
database conclusion looked suspicious. I rejected the first report because it
showed numbers without explaining the cause. In the second run I pushed the
agent from serializer time to visible product impact.

The agent also rejected its own bad evidence. It discarded truncated Logcat
records, stale app bundles, mixed run boundaries and taps which missed the
target. It kept physical-device and emulator claims separate. Both long runs
were paused with handoffs and resumed from the same measurement boundary.

That is what autonomous meant here. The execution continued without me
driving every command. The judgment did not disappear.

You need to keep your hands on the steering wheel. If I had only asked the
agent to search for a reason for these regressions, it would never have found
or proven all of this. Agents are becoming more capable, but also better at
following instructions. They will not start an investigation like this by
accident.

You need to come up with the idea. You need to decide what evidence connects
the technical result to the product. And you need to challenge the first
answer when it does not explain enough.

That is also why this method fits the [orchestrator
pattern](https://edhor.me/blog/orchestrate-dont-operate/). The agent can execute a huge amount
of work, but somebody still owns the goal and decides what comes back into the
product.

Give the agent broad room to investigate, build fixtures and disprove
hypotheses. Keep the write boundary narrow. And keep your hands on the steering
wheel.