Does shipping code to production leave you waking up in the middle of the night?
It should. Every release is a risk exposure event. AI-assisted development accelerates the rate of exposure without necessarily strengthening the control environment around it. Security hardening sits at the top of that gap - and it doesn't go away just because there's only one of you, or you try your best to ignore it.
I'd been using AI to scan and review my code for security issues. Creating tickets, closing them, and moving on to the next one. It feels like progress. It also gives you the theatre of a rigorous process and the appearance of a clean bill of health - what risk advisors call false assurance. But how could I be sure?
How could I be sure my code had actually been reviewed against what NIST recommends? The frameworks, the tooling categories, the defence-in-depth model? I found freely available OWASP ZAP, ran it against the same codebase I'd just “reviewed”, and found a class of issues the LLM had missed entirely.
That's the lesson I'd suspected was true: defence in depth cannot be delegated to an LLM. No single tool, however capable, covers every angle. Enterprise teams understand this because they have specialisations and independent people running separate tools. Solo devs don't have that luxury. That's where Zapper began.
Why I Built It
The output from security scanners is dense and assumes the reader is already a security specialist. ZAP produces hundreds of findings. Many are duplicates. Many are false positives. Reading and digesting them carefully enough to know which is which takes dedicated specialist time.
So I used an independent AI to review and recommend improvements. The model takes each raw finding, assesses context and false-positive probability, and writes a plain-English explanation. I wanted reports I could actually act on.
That grew into a broader pipeline. Five scanner groups feed into a single LLM triage stage, which feeds the reports:
Zapper pipeline. One entry point, five scanner groups, one triage stage, unified reports.
Each capability:
| Capability | What It Covers |
|---|---|
| Web application scanning (DAST) | Runs OWASP ZAP against a live target. Triages findings, maps to OWASP Top 10 2025 and ASVS 4.0. |
| Dependency scanning (SCA) | OWASP Dependency-Check plus pip-audit, consolidated. CVEs flagged by both sources are marked verified. |
| Container security | Trivy for image CVEs, Hadolint for Dockerfile lint, Docker Bench for CIS Docker Benchmark compliance. |
| Host hardening | Lynis audit for SSH config, file permissions, kernel parameters, and authentication posture. |
| LLM endpoint probing | NVIDIA Garak adversarial tests, mapped to OWASP LLM Top 10, NIST AI 600-1, and MITRE ATLAS. |
| AI triage | Each finding analysed by a local LLM for context, likelihood, and false-positive probability with written reasoning. |
| Consensus mode | A second model independently assesses each finding. Disagreements stay flagged as DISPUTED. |
| Workflow reports | HTML, Markdown, and JSON per scanner. Executive summary, Agent Prompt button, copy-as-markdown per finding. |
| Trend tracking | Compare any two scans. See whether things are improving or regressing over time. |
| Web UI and CLI | FastAPI app with a scan wizard and live WebSocket progress, or a single-command CLI for the same pipeline. |
Illustrative only. A Zapper DAST report run against http://example.com, not a real production scan.
How It Grew
The first version was just a cleaner OWASP ZAP report. Same scan, better HTML. Severity colours, deduplication, OWASP Top 10 mappings, trends over time.
I have several personal projects running in different contexts: some internet-facing, some using cloud services, one group running on local machines redirected through Cloudflare to the internet. That last one demanded care. A misconfigured local machine exposed via Cloudflare is a real attack surface.
None of this is reinventing the wheel. It's built on the work of open-source expert communities: ZAP, Trivy, Garak, Lynis, and others. Zapper makes their tools accessible to me in one place. But by using AI, I can make a security pipeline customised to my ways of working.
Security as a Fitness Function
A DAST scan against a real application produces hundreds of findings. Some are critical. Many are duplicates. Some are false positives generated by the scan itself. Manually triaging all of them, every time, for every project, is not viable.
The point of the pipeline is to turn security into a fitness function - the Building Evolutionary Architectures concept of an automated, objective measure of an architectural characteristic, run continuously rather than audited periodically. Each scanner is one such function. The triage and consensus layer turns raw scanner output into an objective assessment.
Zapper's triage pipeline compresses the output, chunks it for LLM context limits, and sends each finding to a local model for analysis. The model assesses context, likelihood, and false-positive probability, and writes a plain-English explanation of its reasoning.
compress → chunk → analyse → consensus → executive summary
Then consensus mode: a second model independently assesses the same finding. If they agree it's a false positive, it's flagged with high confidence. If they disagree, it stays flagged as DISPUTED with both opinions shown. That transparency matters. The system surfaces uncertainty rather than hiding it.
The result: 200 raw findings become 30 that actually warrant attention.
Cost is managed by design. A small local model (Gemma via MLX) handles triage at zero marginal cost. A frontier model handles consensus verification - in my case Claude Sonnet, but the pattern works with any sufficiently capable model. Token budgets and batching keep costs reasonable, and I'm leaning on subscriptions I already have rather than burning pay-per-token API credit.
LLM Security Probes
If you're running an AI application, the attack surface includes the model itself. Guardrails are good, but red teaming techniques help secure your prompts as well.
Zapper integrates NVIDIA Garak to run adversarial tests against LLM endpoints. The results map to three taxonomies: OWASP LLM Top 10 v2.0, NIST AI 600-1, and MITRE ATLAS v5.4.0. Every finding gets all three mappings, which matters if you need to speak the language of a specific framework for compliance or reporting.
These mappings should also give you a useful head start on the Australian assurance frameworks now gaining traction - the Voluntary AI Safety Standard, the Australian Government AI Assurance Framework, and ISO 42001 - which share many of the same underlying concepts.
The probe presets I've configured in Zapper:
| Preset | Attack Vector | Good For |
|---|---|---|
| prompt-injection | Direct and indirect injection attacks | Any LLM that accepts user input |
| jailbreak | DAN variants, Grandma exploits, encoding bypass | Models with safety guardrails you need to verify hold |
| data-leakage | Training data extraction attempts | Models fine-tuned on proprietary or sensitive data |
| output-safety | Malware generation, XSS output, harmful content | Models used for code generation or content output |
| hallucination | Package hallucination (supply chain risk) | AI coding assistants that suggest packages |
| content-safety | Toxicity and refusal bypass | Consumer-facing models with moderation requirements |
| quick | Highest-risk categories combined (default) | First run against any new LLM endpoint |
| all | Complete coverage across all categories | Thorough pre-production audit |
Running Garak this way, with Zapper handling orchestration and report generation, means the adversarial test results land in the same visual format as everything else: same severity colours, same executive summary, same Agent Prompt button to copy findings into a remediation session.
“The reports are designed as inputs to action, not archives.”
The Reports as a Workflow Tool
Every Zapper report has three things:
Agent Prompt button.
One click generates a structured prompt with the full scan context and findings. Paste it into Claude Code and ask it to address the fixes. The report isn't the end of the process; it's the input to the next step.
Copy-as-markdown per finding.
Each individual finding has its own copy button, separate from exporting the full report. Pull out one specific vulnerability, paste it into a session, deal with it in isolation. Useful when you want to work through findings one at a time without being overwhelmed by the full report.
LLM triage reasoning.
Each finding shows the AI's written justification for its classification. It explains what context made it assess it that way, and what would change the assessment. Transparent and auditable.
Together these make the reports feel different from documentation. They help me understand the security concerns better and educate me at the same time. They're designed as inputs to action, not archives.
Shift-Left Governance, Automated
Step back and the pattern Zapper implements looks structurally identical to an Architecture Review Board:
- Multiple independent reviewers. Each scanner specialises in a different layer - application, dependencies, container, host, LLM - and each finding is then reviewed by two independent AI models. Same separation-of-concerns logic that drives ARBs to bring in security, integration, and data architects separately.
- Transparent dispute. When the two reviewers disagree, the finding stays flagged as DISPUTED with both opinions shown. Mirrors how a well-run governance forum handles split decisions: escalate visibly, don't paper over.
- Evidence-mapped findings. Every result maps to recognised frameworks (OWASP, NIST AI 600-1, MITRE ATLAS, CIS). ADRs do this for architecture decisions; Zapper does it for security findings.
- Action-oriented outputs. Agent Prompt buttons and per-finding markdown drive remediation, not archives. Governance exists to improve delivery, not produce shelfware.
The difference worth naming clearly is that Zapper automates what most enterprises still do manually. Architecture governance today still leans heavily on humans reading deliverables, marking them up, and writing reasoned opinions. That model is bounded by reviewer capacity - which is why ARBs become bottlenecks the moment delivery scales. Zapper applies the same governance philosophy but pushes it down to where the code lives, runs it on every change, and produces evidence the Three Lines of Defence can all consume - first-line developers acting on Agent Prompt remediation, second-line control functions reading framework-mapped findings, third-line audit tracing decisions back through transparent triage reasoning.
I wrote about lightweight, risk-based governance earlier in Dare to Shift Left on Architecture Governance. Zapper is what happens when you take that philosophy and ask: what if the reviewers were also AI?
What I Learned Building It
Touching this many security frameworks teaches you the landscape by necessity. You end up gaining a better appreciation of OWASP Top 10 2025, ASVS 4.0, CIS Docker Benchmarks, NIST AI 600-1, and MITRE ATLAS not because you studied them but because you had to wire them into report templates and taxonomy mappings.
The bigger lesson is to start from what already exists. The hard problems in security scanning - parsing CVE feeds, fingerprinting CWEs, running adversarial probes - have been solved by people who specialise in them. The leverage is in the integration, not the invention.
And LLMs have changed the economics of building for yourself. A pipeline like this would have been a product team's roadmap a few years ago. With local models doing the routine work and a frontier model doing the verification, one developer can build something shaped to their context rather than buying a SaaS product designed for someone else's.
Oh, and it helps you sleep a tiny bit better at night.
If you're thinking about AI security governance at enterprise scale, I'd be interested to talk.
Vinod Ralh is an Enterprise Architect specialising in AI governance and responsible AI delivery.