The Risk Rancher Field Manual, Ch. 5: The Deep Dive
Part of The Risk Rancher Field Manual, a hands-on guide to starting, scaling, and running a vulnerability management program.
A scanner casts a wide, shallow net. A pentester dives deep in one spot. Both have their place, but they are not substitutes, and the single most common mistake in this whole field is paying for the second when you have not done the first.
Do you even need one yet?
Here is the honest gate, the one no pentest firm will put on their sales page: if you are not already patching the obvious, a pentest is a waste of money. A tester will find the same out-of-date software your scanner found, add a few things, and hand you a nicer-looking PDF. You will have spent fifteen thousand dollars to confirm the backlog you already had.
The value of a pentest is not in finding the easy stuff. It is in finding the things automation physically cannot: the business-logic flaw where a user can skip the payment step, the chain where three low-severity issues combine into one critical, the creative human move a scanner has no concept of. You only get that value if the easy stuff is already handled, because otherwise the tester spends the engagement, and your money, rediscovering it.
So earn the pentest. Do your scanning and your patching first. Then bring in a human to find what the machines miss.
Two different nets. One covers the estate; the other goes all the way down on one target. Neither replaces the other.
When is it actually time? When a customer or regulator requires one and a deal depends on it. When you are shipping something genuinely new and high-stakes. Or when you have wrung out everything the scanners can find and you want a skilled adversary to tell you what you cannot see about your own system.
The types
“Pentest” covers a range, and buying the wrong one is its own kind of waste.
- Web application and API tests are what most companies actually need: a skilled human attacking your app the way a real attacker would.
- Network and infrastructure tests probe your perimeter and internal segmentation.
- Cloud tests go after your cloud configuration and identity, the modern soft underbelly.
- Red team engagements are goal-based and stealthy: can we get to the crown jewels without being caught? These are powerful and expensive, and you do not need one to check a compliance box.
Buy the test that matches your real risk, not the most impressive-sounding one. A payments company needs a hard look at its payments app far more than it needs a cinematic red team.
How to scope it so you don’t waste the money
The difference between a useful pentest and an expensive scan is the scope and the tester. Control both.
Scope it to a real target with a real question. “Test everything” produces shallow coverage of a lot; “can an unauthenticated user reach another customer’s data in this app?” produces a sharp, useful answer. Give the testers access, documentation, and a test account, because a blind test of a complex system burns half the budget on reconnaissance you could have handed them on day one. Ask for a sample report before you sign; if their idea of a finding is a scanner’s output with a severity score pasted on top, walk away. You are paying for narrative, reproduction steps, and honest business impact. And insist on a retest after you fix, because a finding you never verified closed is a finding you never closed, which is the whole subject of chapter seven.
Try it. Grade a report before you pay for the next one. Run any pentest report you can get (your last one, or a vendor’s sample) against this checklist, one point each: (1) every finding has reproduction steps a developer could actually follow; (2) impact is written in business terms, not just a CVSS number; (3) at least one finding is something a scanner could not have found, a logic flaw, a chain, an abuse of intended behavior; (4) findings are specific to your system, not boilerplate; (5) there is a concrete, prioritized remediation per finding; (6) a retest after you fix is included in the price. Four or less out of six and you are looking at an expensive scan. Read the list the other direction and it is your scoping worksheet: demand these six in the statement of work and you will get a report worth feeding to the herd.
What you do with the report
A pentest report is not a separate universe, and it does not belong in a drawer. Those findings go into the exact same engine as everything else: triaged, prioritized, routed to an owner, fixed, and verified. The report is just another source of findings, a slower and more expensive one that happens to be much higher quality. Feed it to the herd like the rest.
Straight talk. If you are the pentester reading this, Risk Rancher was built with you in mind as much as the in-house team, and it does the thing most trackers cannot: it takes the report in. Upload the engagement report, parse it, and promote the findings through a draft queue into real tickets, each with a CVSS vector, a CWE, a business-impact field, and screenshots attached, so your work lands in the same engine the client runs instead of dying in a Word document. It runs the other direction too: pick the findings and generate a branded engagement report as Markdown or a self-contained HTML you print to PDF, with a library of reusable finding templates so you stop retyping the same SQL injection write-up for the fortieth time. Free and air-gapped, which matters when the findings are a client’s. A consultancy can run every engagement in it and hand the client a live tracker instead of a PDF they will lose. It is free and on GitHub.
Now you have the nets cast and, where it is warranted, a human in the water. Findings are coming in from every direction: scanners, pentests, bug reports, that one engineer who keeps emailing you things. Which brings us to the real work, the part the whole book has been walking toward and the part your spreadsheet cannot survive. Part 3: the vulnerability management engine. We start where everyone starts, drowning, and we learn to sort the herd.