t0ul

What the patch loop would take

Part of a short series on the patch loop. The second post found that every product owns a piece of the loop and none owns the whole thing. This post is what the whole thing would look like, built on the open-source parts that already exist.

Intro

When I laid the products side by side, the missing pieces were not the exciting ones. Nobody is lacking a scanner, or a patcher, or a way to open a pull request. Those all exist, several of them are free, and they work. What is missing is the connective tissue: the step that takes two scanners’ disagreeing output and turns it into one list of real findings, the step that decides the smallest safe fix for each, and the step that proves the result after the fact.

That connective tissue is unglamorous, which is probably why nobody sells it, and it is the entire product. This post is what it would be, which open-source parts you bolt it to, and the one limit at the end that no amount of engineering removes.

I. Five layers, two of which exist

The loop is five layers, plus memory and a voice underneath them.

  • Observe: the SBOM and findings for the exact digest, from more than one scanner. Exists: Trivy, Grype, Syft.
  • Translate: one canonical finding per vulnerability across the scanners’ different vocabularies, backport-aware, severity-reconciled. Partly inside each scanner, never across them.
  • Decide: one disposition per finding, with evidence, and the smallest safe fix. Does not exist.
  • Act: OS packages in place, dependencies and tags by PR, one author, tests, rollback. Exists in pieces: Copa, Renovate, Dependabot.
  • Prove: rescan the result with the same scanners, attach the before-and-after to the PR, re-emit the machine-readable statement. Does not exist.

Observe and the Act arms exist. Translate, Decide and Prove are the gap. Say that out loud and the shape of the product is already clear: it is the three middle steps, wired to the two that exist.

II. The open-source stack: the hands

Before writing anything, it is worth being precise about how much is already free, because the answer is “almost all of the mechanical work.” This stack exists, is open source, and self-hosts:

  • {Trivy, Grype, Syft} observe the image, two opinions and an SBOM.
  • {Copa} patches OS packages in place.
  • {Renovate} bumps application dependencies and tags by pull request.
  • {the OSV API, the distro trackers} carry identity: aliases, affected ranges, the commits that fixed each advisory.
  • {govulncheck, Kubescape relevancy} provide evidence that a vulnerable package is never actually loaded.
  • {Dependency-Track} is the memory: findings, dispositions, VEX in and out, KEV and EPSS.
  • {vexctl, cosign} publish the machine-readable statement and attach it to the image.

That covers Observe, both arms of Act, the raw material for Translate, most of Memory, and the VEX format. None of it needs a language model. If you have ever wondered whether you could assemble the loop from parts on a weekend, the honest answer is that you can assemble the hands. What you cannot download is the part that decides.

III. What no component has: the brain

Two layers have no open-source implementation, and they are the ones that make the output trustworthy.

The decider. Every finding needs exactly one disposition, each with required evidence: the distro ships a fix, so patch in place; a dependency release closes it, so open a PR with the smallest fixing version; it is a standard-library CVE, so bump the build-image tag (on my test image, 120 of 593 findings were exactly this, fixable by a single toolchain bump nobody had made); it is fixable only by a newer base, so move the base with the measured delta attached; or nobody has shipped a fix, so track it and re-check on the next database refresh. A dozen dispositions, a rule for each, the smallest fixing version computed rather than the latest grabbed, one pull request per image split by how risky the change is, and the OS patch sequenced after the dependency PRs so they do not fight. No open-source component has any of this. It is about forty lines over the OSV API to compute the smallest fixing version, and a few hundred more to route and shape the PRs, and that few hundred lines is the product.

The prover. After the patch, rescan the new digest with both scanners, write the before-and-after table into the pull request, attach the VEX statement, and record the lineage: this change took digest X to digest Y and closed 185 findings. None of the ledgers has an actions row like that; none keys on digest lineage. The proof is what lets a developer merge the PR without re-deriving the whole analysis, and today it has to be assembled by hand.

There is a third smaller gap, the translator: nobody merges Trivy’s and Grype’s views of one digest into a single finding identity. DefectDojo deduplicates inside one scanner’s output, not across two. That is more glue, keyed on the canonical id plus the package.

I wrote a thin spike to see how much glue it really is: ingest, plan, record, report, and a backport judge, about 850 lines of Go with tests, and it ran the loop on one image end to end with the proofs saved. A real version is several times that. But the point stands: it is a product, not a weekend, and it is also not a scanner, a patcher or a bot, because those exist and are free. It is the decider and the prover in the middle.

IV. Determinism, concretely: the backport judge

The reason I am sure the brain can be deterministic is that I built the hardest-looking piece deterministically. When a fix exists upstream but only twenty-three minor releases away, teams reach for a cherry-pick: backport just the security commit onto the pinned version. That sounds like judgement a human has to make. It is not, or not all of it.

The judge fetches the fix commits from OSV, dry-runs the cherry-pick onto the pinned tag in a throwaway worktree, counts files and hunks, splits any conflict into code versus changelog, and checks whether each function the fix touches existed at the fix’s parent and still exists at the pin. On three real pins from my sample it said three different, checkable things:

  • urllib3, CVE-2023-43804: the fix applies with no code conflict, but it is five files and fifteen hunks, so the call is “review this as a change, not a cherry-pick.”
  • requests, CVE-2023-32681: a code conflict inside rebuild_proxies, a function that was rewritten between the pin and the fix. The call is “a rewrite wearing a backport’s name; upgrade or mitigate instead.” The function the fix touches is not the function the pin has.
  • Go x/net, CVE-2024-45338: three files, four hunks, no conflict, all four touched parser functions present at the pin. The call is “candidate,” the one case of the three where a cherry-pick is reasonable.

One of three qualified. The value of the check is that the other two do not get argued into it on a deadline. Every input is a commit and a tree; every output is a count and a yes or no. There is nothing in there for a model to hallucinate.

V. The one limit nobody engineers away

Honesty check, because the loop does not end at zero. On my image, after the dependency PRs and Copa, 339 findings remained with no fix available anywhere. A customer will ask about those, and some are marked critical.

Here is what that number actually is. It is Debian’s number, not the application’s. Those 339 are 146 distinct CVEs, 2 critical and 70 high, and every one carries a written reason from Debian’s own security team: “minor issue, no advisory,” or “the vulnerable code is not built into this package,” or “fixed upstream, not yet backported to stable.” None is on CISA’s exploited list. The highest exploit-probability score among them is a 2011 TLS issue rated low. And the container’s only process is a static Go binary that loads zero shared libraries, so most of those packages are never executed at all.

So the reasonable answer is three deterministic steps, in order. Remove the packages the service does not use: the same service on a static distroless runtime scans at a single finding, and that one is fixable. Classify what remains by the distro’s stated reason, with the evidence stored per finding, so the scanner’s “critical” becomes “not affected, vulnerable code not present, here is Debian’s note.” Escalate the few that survive both to a base change or an accepted exception with an expiry date. “No fix available” is not the end of the analysis; it is where the analysis gets specific, and all of it is checkable.

Conclusion

The loop is not waiting on a breakthrough. Every mechanical step is already open source and free. What is missing is the judgement in the middle, the decider that picks the smallest safe fix and the prover that shows it worked, and that judgement is deterministic end to end, from the smallest fixing version to the cherry-pick check to the disposition of the unfixable. No language model required, and in a thing you are asking people to merge without reading every line, no language model wanted.

Deliver the result as one pull request, with the smallest fixing versions, the before-and-after table from both scanners, and the OS layer already patched in place, and you are back to the first post: a patch in the shape people merge in a day instead of ignore for a year. The scanners found the problem a long time ago. The missing product is the one that does something about it, the same way, every time, with the receipt attached.

← Back to the blog