Watching it think

September 7, 2026 · 7 min read · #ai-security, #red-team, #llm, #tooling, #golang

Part of my EVA series, where I build the cockpit I pilot GORED runs from. The last post was the seam that lets a piloted run and a headless CI run be one loop. This one turns the dial down and actually watches a run go. Code in t0ul/GORED; write-ups in t0ul/ai-security-engineering.

Intro

My first run page was embarrassing. It used a meta refresh, the old trick where the page reloads itself every couple of seconds, and watching a run through it felt like watching a kettle by blinking. You would catch the state at two-second strobes, the page would jump to the top on every reload, and a fast run would be half over before the first refresh landed. It worked, in the sense that the numbers were eventually right, but it was no way to watch a model think.

The fix is the part I like, because it did not require touching the engine at all. The run already produces a perfect, ordered, tamper-evident record of itself as it goes. All EVA had to do was read that record out loud. This post is how the live feed works, and a thing that falls out of it for free: because the prompt a run used is written into that same record, every run carries the exact instructions it ran under, so you can reproduce it later without guessing.

A piloted run against the benign demo target. The feed on the left is the trace streaming in live as the model works; the findings in the centre end in a captured broken-access-control bug, with the disclosing response and a repro curl; the map on the right is the surface it built. Watching a run go is the whole point of this post.

I. The run already narrates itself

Every GORED run writes a trace through gledger, which is a dependency-free, hash-chained audit log. Each step the run takes becomes a record under one trace id: the model was called and here is what it said, this tool was invoked with these arguments, this observation went on the blackboard, the phase advanced from recon to exploit. Every record links to the one before it with a hash, so the log is tamper-evident; edit a line after the fact and a verify step catches the break. That property is why I trust a replay weeks later, which is a later post, but it matters here too.

The point for the live feed is quieter than tamper-evidence. It is that the record already exists, in order, written as the run happens, by the engine, for its own reasons. The engine does not write it so a browser can watch. It writes it because a run has to be auditable whether anyone is watching or not. So EVA does not need the engine to grow a notion of a spectator. It needs to read a file that is already being written.

II. Tailing the trace, not instrumenting the loop

The live page opens a stream to one route, and that route tails the trace file. In Go the whole thing is a handler that sets the server-sent-events content type, seeds the page with the trace so far, and then pushes each new line as it appears:

mux.HandleFunc("GET /run/{id}/events", s.events)

func (s *Server) events(w http.ResponseWriter, r *http.Request) {
	flusher, ok := w.(http.Flusher)
	if !ok {
		return // this client cannot stream
	}
	w.Header().Set("Content-Type", "text/event-stream")
	// seed from the trace so far (works for a live run or a finished one),
	// then tail the file and push each new event as it lands:
	for ev := range events {
		data, _ := json.Marshal(ev)
		fmt.Fprintf(w, "data: %s\n\n", data)
		flusher.Flush()
	}
}

Server-sent events are the right tool here and a smaller one than most people reach for. It is a plain HTTP response that never ends, with each message written as a data: line and flushed immediately, and the browser’s built-in EventSource reconnects on its own if the connection drops. No websockets, no second protocol, no library. The engine does not import any of this and does not know it exists. It writes its trace; EVA tails it; the browser renders it. Three parties, each minding its own business, joined by a file.

The nice consequence is that live and finished are the same code path. A run that ended an hour ago has a complete trace on disk, so opening its page seeds every event and then simply finds no more to tail. A run happening right now seeds what has landed so far and keeps going. I did not write a “replay a finished run” feature and a “watch a live run” feature. I wrote one feature that reads a trace, and whether the trace is still growing is not its problem.

III. Three lenses on one stream

The same stream feeds three views of the run, because the raw feed alone is a firehose. There is the feed itself, every step in order, which is what you read when you want to know what the model actually did. There is a surface map that accumulates the paths and endpoints the run discovered, which is what you watch when you want to see the app’s shape emerge. And there is the verdict, the capture state the oracle reached, which is the only one of the three that answers did it work.

The recon run in the screenshot above is the one that sold me on the whole cockpit. Pointed at an app it had never seen, with nothing handed to it, the model read the pages, dug through the single-page app’s JavaScript, and surfaced dozens of real paths it was never told about. I wrote about that result in my GORED series, because reading an app turns out to be the one thing a small local model is reliably good at. Watching it happen, the map filling in on the right while the feed scrolls on the left, is a much better way to understand that than reading a number at the end.

IV. The prompt is a knob, and the knob is recorded

There is one run control I want to call out here because it lives right next to the feed. The instruction a stage runs with, the skill prompt, is editable per run. You can sharpen the brief, change the framing, and launch, without touching code, through one option the orchestration layer passes down:

opts = append(opts, appsweep.WithSkillInstruction(inst))

That by itself is just a text box. The part that makes it trustworthy is where the instruction goes. When the run starts, the engine writes the skill instruction it is actually using into the trace, in the opening record, labelled as the prompt of record:

r.emit(traceID, "run", "run_start", gledger.F{
	// ...
	"skill_instruction": r.Skill.Instruction, // the prompt of record, reproducible from the trace
})

So a run does not just remember its result, it remembers the exact instruction that produced it, in the same tamper-evident log as everything else. When a run does something surprising, I do not have to reconstruct what I typed; it is in the trace. And when I want to compare two prompts honestly, the thing I am comparing is recorded on both sides, not remembered. This is the small beginning of treating prompts as governed data rather than hardcoded strings, which is a theme that comes back when the cockpit grows a factory.

Conclusion

So the live feed is not really a feature of the engine at all. The engine writes an ordered, hash-chained account of its own run because it must; EVA tails that account and streams it, seeding the page the same way whether the run is live or long finished; and the browser renders three lenses on it with nothing fancier than server-sent events. The prompt rides along in the trace, so a run is reproducible from its own record. The engine still does not know anyone is watching, which is exactly how I wanted it.

Next I will let a few of these runs pile up and face the obvious question: where do they go when I restart the cockpit? That is the sortie log, the little SQLite registry that turns a scatter of trace files into a comparable history.

← Back to the blog