The 30% wasn't a testing win

The cleanest stat on my resume is also the one I'm most careful about.

Somewhere in my FPT years — between 2017 and 2019, on a workflow project for a foreign client that I won't name — we cut production issues by roughly 30%. I list it because it's true. I'm careful about it because every time I say it out loud, someone in the room hears "he wrote great tests." That's not what happened.

The 30% didn't come from the test suite#

I'd been a manual tester before, then moved to automation, and the easy story would be that the automation harness caught everything before release. It didn't. The automation was fine. It found regressions, like it's supposed to. But the regression rate wasn't really the problem. The problem was new-feature defects — bugs that shipped because the spec was already wrong by the time anyone wrote a test against it.

The 30% came from moving the conversation earlier. Not from more tests, harder tests, or smarter tests. From writing the test cases during spec review, before development started.

It sounds boring written out. It was boring to do. It also worked.

Spec review used to be where the BA talked and we nodded#

The pattern before the shift was familiar to anyone who has worked QA on a waterfall-ish project. The business analyst would present the spec. Developers would ask a few clarifying questions about the data model or the API. Testers would mostly listen, because the "real" testing artifact — the test plan — was something we wrote later, after seeing the build.

So the meeting produced a signed-off spec, and three weeks later we'd discover that the spec was ambiguous in five places, contradictory in two, and silent on the edge cases that actually mattered. By then a developer had picked one interpretation, written code against it, and the cost of changing the interpretation was a sprint.

What we changed was small. Testers started bringing a working draft of test cases into spec review. Not exhaustive — just the obvious paths, plus the questions a test plan forces you to ask. What happens if the applicant submits twice in five seconds. What happens if the document upload fails halfway. What does the system do at month-end when the cron job and the user collide on the same record.

Half of those questions had no answer in the spec.

The bugs we prevented were the ones we never wrote#

This is the part that's hard to communicate to someone who didn't sit in those rooms.

You can't count bugs that didn't happen. The 30% came from comparing release windows, not from any clean experiment. What I can say is that the questions getting asked in spec review were questions that, in the old workflow, became defects two months later. A field that should reject negative numbers. A workflow that should warn before letting two users edit the same record. A status transition that was illegal but undocumented as such.

Each one was a five-minute conversation in spec review. Each one would have been a half-day defect investigation, a hot-fix, a customer-facing apology, and a permanent regression test in the old flow.

What I'd tell past me

The cheapest bug to fix is the one you talk out before the code exists. The second cheapest is the one you catch in code review. Everything after that gets exponentially more expensive — and the cost isn't engineering hours, it's the trust your client loses each time a defect reaches them.

The awkwardness moved, it didn't disappear#

The thing nobody mentions about moving testing earlier is that it makes the early meetings worse before it makes the late meetings better.

A BA who is used to presenting a spec to a nodding room does not enjoy having a tester ask, in front of the client's PM, what happens when the file upload times out. The first month was uncomfortable. I remember one review where the BA — senior, sharp, well-respected on the project — pulled me aside afterwards and said, in essentially these words, that I was making her look unprepared.

She was right. I was. The spec genuinely wasn't ready, and the meeting had been working as a polite ritual to pretend that it was.

We talked. The team lead backed the change. Within a quarter, the BAs were writing their own edge-case sections into the spec before the review — partly to get ahead of the tester questions, and partly because once it was clear the questions were going to come, it was easier to think about them in a quiet room than under client gaze.

The shift wasn't QA getting smarter. It was the team agreeing to be uncomfortable earlier in exchange for being calm later.

What the 30% was actually measuring#

I think about this number now whenever someone asks me to "improve test coverage" as a goal in itself.

Coverage is a lagging indicator of something that already happened upstream. If the spec is sharp, if developers understand the edge cases before they write the code, if the test cases got argued out before the keyboard came out — coverage tends to take care of itself, and the bugs that remain are the genuinely surprising ones rather than the ones a careful reading of the spec would have caught.

The 30% wasn't a measurement of how good our tests had gotten. It was a measurement of how much earlier the awkward questions were being asked. The test suite was the visible artifact. The actual mechanism was a meeting culture change.

This is why I'm careful about the number. I want it on my resume because it's the cleanest thing I can point to from those years. I don't want it on my resume framed as "improved test automation" or "led QA initiatives," because that's not what bought the 30%. What bought it was a tester being willing to ask a stupid question in front of a client, and a team willing to let her.

What I carry forward from that number#

Years later, building autonomous commerce systems where the "spec" is half English and half model behavior, the lesson holds in a strange way. The cheapest bug is still the one you talked out before the code existed. The format has changed — a test case has become a behavior trace, a spec review has become a tool-call review — but the underlying move is the same. Move the awkwardness earlier. Trade the comfortable meeting now for the calm release later.

I wrote about a related instinct in why I still think like a tester after four years as a fullstack dev — the impulse to ask "what breaks this" doesn't leave when you switch roles. The 30% is the cleanest receipt I have that the instinct, applied at the right moment, is worth more than any specific tooling.

That's the version of the story I trust. The shorter version — we reduced production issues by 30% — is true, but it's missing the part that actually matters.