https://t0ul.com/tags/testing/Recent content in Testing on t0ulHugoen-usTue, 19 May 2026 00:00:00 +0000https://t0ul.com/blog/the-one-command-scorecard/Tue, 28 Apr 2026 00:00:00 +0000https://t0ul.com/blog/the-one-command-scorecard/<p><em>Part of my <a href="https://t0ul.com/blog/the-plan/">AI Security Engineering</a> series, a local email-to-calendar agent I build, then attack, then harden. If you have not set up the lab yet, start with <a href="https://t0ul.com/blog/setting-up-the-lab/">Set up your lab</a>. The lessons are <a href="https://github.com/t0ul/ai-security-engineering">on GitHub</a>, and the agent itself lives at <a href="https://github.com/t0ul/emaildrop">t0ul/emaildrop</a>.</em></p> <h2 id="from-a-pile-of-tests-to-one-command">From a pile of tests to one command</h2> <p>Five parts in, the agent is covered by dozens of before/after tests, each one an attack that lands without a control and is blocked with it. That is a lot of green, and green that nobody runs is just a feeling again. Part 6 is about the flywheel: the machinery that keeps all of this true as the agent grows, starting with the simplest piece, a single command that scores every control at once and fails loudly the moment one regresses.</p>https://t0ul.com/blog/from-pattern-to-library/Tue, 05 May 2026 00:00:00 +0000https://t0ul.com/blog/from-pattern-to-library/<p><em>Part of my <a href="https://t0ul.com/blog/the-plan/">AI Security Engineering</a> series, a local email-to-calendar agent I build, then attack, then harden. If you have not set up the lab yet, start with <a href="https://t0ul.com/blog/setting-up-the-lab/">Set up your lab</a>. The lessons are <a href="https://github.com/t0ul/ai-security-engineering">on GitHub</a>, and the agent itself lives at <a href="https://github.com/t0ul/emaildrop">t0ul/emaildrop</a>.</em></p> <h2 id="the-same-shape-written-out-too-many-times">The same shape, written out too many times</h2> <p>By now you have read the same test two dozen times. Fire an attack at the control removed, confirm it lands. Fire it at the real control, confirm it is blocked. Read the result with an oracle. The only part that ever changes is the control and the attack; everything else is ceremony, and ceremony repeated by hand drifts, which is how you end up with a test that quietly stopped being able to fail.</p>https://t0ul.com/blog/a-live-red-team-against-the-real-model/Tue, 19 May 2026 00:00:00 +0000https://t0ul.com/blog/a-live-red-team-against-the-real-model/<p><em>Part of my <a href="https://t0ul.com/blog/the-plan/">AI Security Engineering</a> series, a local email-to-calendar agent I build, then attack, then harden. If you have not set up the lab yet, start with <a href="https://t0ul.com/blog/setting-up-the-lab/">Set up your lab</a>. The lessons are <a href="https://github.com/t0ul/ai-security-engineering">on GitHub</a>, and the agent itself lives at <a href="https://github.com/t0ul/emaildrop">t0ul/emaildrop</a>.</em></p> <h2 id="the-suite-that-tests-nothing">The suite that tests nothing</h2> <p>Every live-tagged test in this book has had the same comment at the top: run against the local model, fail if it is down, never skip. This post is why. A security suite for an LLM agent has a tempting shortcut built into every testing framework, and taking it quietly turns the whole scorecard into decoration.</p>