A factory I wouldn't fake

October 5, 2026 · 6 min read · #ai-security, #red-team, #llm, #tooling, #golang

Part of my EVA series, where I build the cockpit I pilot GORED runs from. Last time I gave runs a durable home so they survive a restart. This post is about the one surface I refused to fake, and what it took to make it real. Code in t0ul/GORED; write-ups in t0ul/ai-security-engineering.

Intro

Every cockpit wants a factory behind it, and EVA’s is Studio: the place the skills a run is built from are meant to be authored, tried, and saved. I built it, and then for a while I would not let you save. You could list every skill, read it, pick it, and the edit button was not there, on purpose. That restraint is the first half of this post, because it is the same standard I hold every number in this project to. The second half is the payoff, because the piece that was missing has since landed, and the editor is real now in the one way that matters: a skill you change actually changes the next run.

The reason I waited is worth more than the editor itself, so I will spend most of the post on it. It is a trap I have fallen into before and watched a hundred tools fall into: shipping a control that looks like it works and quietly does nothing.

I. What a factory would be

Laid out fully, the factory has three inventories. Tools are the primitives the agent can call during a run, an HTTP request, a fingerprinter, an endpoint digger, a scanner. Skills are the instruction scaffolds that frame the task for the model, from cold, which hands it almost nothing and makes it discover, to warm, which spells more of it out. Prompts are the instruction text those skills carry. On top of all three sits the idea of a bundle: a saved target plus model plus skill, a whole configured agent you summon by name instead of rebuilding by hand.

That is a good surface to want, and editing a skill is the same muscle as the per-run prompt knob from two posts ago. So the question was never whether to build the editor. It was whether the editor would be telling the truth the day I shipped it.

II. It was all hardcoded Go

Because at the start, the skills were hardcoded Go. A skill was a package-level variable. A browser cannot edit a Go variable; that is a recompile, not a form submission. So the honest first step toward authoring is not the surface at all. It is making the things data: moving skills out of code and into the datastore, so that there is something a form could actually write to. Only once that exists, and once the run path actually reads it, does an editor mean anything.

This is the step that is easy to skip. You can throw an editor on the page, wire its save button to a handler, and demo it, and it will feel finished while being a prop.

III. The discipline: read-only until it was wired

So the first thing I moved was skills into a table, seeded from the built-in catalog on startup, never clobbering a row that is already there:

// SeedSkills inserts the built-in skills if absent. It never overwrites an existing
// row, so a user-edited skill survives re-seeding.
func (s *Store) SeedSkills(skills []core.Skill) error {
	for _, sk := range skills {
		if _, err := s.db.Exec(
			`INSERT OR IGNORE INTO skills (name, temp, instruction, builtin) VALUES (?,?,?,1)`,
			sk.Name, string(sk.Temp), sk.Instruction,
		); err != nil {
			return err
		}
	}
	return nil
}

Now the skills were data. And here is the part where most tools would have shipped the editor, and I did not, because of one fact: the runs still read the compiled skill, not the row. The table and the runs agreed only because the table was seeded from the same code the runs used, so they happened to match. The moment you edited a row, they would stop matching, and the run would keep using the version baked into the binary.

Put an editor on top of that and watch what happens. You open Studio, sharpen a skill, save, the row updates, the page shows your new text. Then you launch a run with that skill and get the old behaviour, with no error and no warning. A control that accepted your change, displayed your change, and did not apply your change. That is exactly the failure this whole project exists to refuse, only wearing a user interface. In the attack-driven work it is a test whose before and after are both hand-written, so it passes no matter what the real system does. An edit button over a value the run ignores is the same lie in a different costume. So Studio listed, and did not save, and said so.

IV. The payoff: now the edit drives the run

Then the missing half landed, and it is small, which is the nice part. When EVA sets up a run, it no longer reaches straight for the compiled skill. It asks the registry first, and the edited instruction wins:

func (s *Server) effectiveInstruction(stage string) string {
	st := appsweep.Stages()[stage]
	def := st.Skill.Instruction                 // the in-code default
	if row, found, _ := s.store.GetSkill(st.Skill.Name); found && row.Instruction != "" {
		return row.Instruction                  // your edited skill, from the registry
	}
	return def
}

That is the whole wiring. The run resolves the skill it will actually use by reading the registry, falling back to the in-code default only when there is no edit. Which means the table and the runs are no longer agreeing by coincidence; the runs genuinely consume the data. And the instant that was true, the editor stopped being a lie, so I shipped it.

Studio, now with the save button attached. Each skill is editable, with a dry-run to try an edit before you keep it and a revert back to the in-code default. The edit is honest because the run reads the registry, so saving a skill changes the next sortie.

It ships with two guardrails that matter. There is a dry-run, so you can try an edited instruction before you commit it, and a revert, which restores a skill to its pristine in-code default when an experiment goes nowhere. Built-ins stay marked as built-ins, so you can always tell the shipped scaffold from something you wrote. The button means something now, which is the only reason it is allowed to exist.

Conclusion

So the discipline was never about refusing to ship the editor. It was about refusing to ship one that lied. For a while that meant a read-only list and an honest note that authoring was a code change. Now it means a save button that actually reaches the next run, because the run reads the registry it writes to. A control earns its place the day it tells the truth, and not one day before, which is the same standard I hold every green light in this project to.

Next is the mode this whole cockpit was secretly built for: point it at a few of my own apps, turn the autonomy all the way up, walk away, and read a report in the morning of what it tried, what it found, and how the findings chain.

← Back to the blog