How I Built a Secure AI Relay for an iOS App

The Architecture

This is where I took advantage of a deliberate move by the platform companies. Over the last couple of years, Cloudflare and Vercel have reshaped their edge networks to support AI workloads directly: serverless functions, key-value stores, SQLite at the edge, and gateways that sit in front of the model providers. Work that used to need a backend team is now a handful of managed primitives a solo developer can wire together.

The relay runs on Cloudflare Workers, serverless functions deployed to the edge. No server to maintain, no containers to patch, no uptime to watch. It scales to zero when nobody is using it and absorbs bursts without configuration.

Cloudflare was the deliberate default. I had already built Futurescapes on the same stack, so Workers, KV and D1 were known quantities, which meant the novelty budget for this project could go into the eval pipeline and the safety policy, not into learning a new platform. The trade-off is committing further to one vendor's edge ecosystem, but for a solo developer shipping a one-time purchase, reduced cognitive load and a single bill beat best-of-breed every time.

Defence in Depth

The relay trusts nothing. Every request passes through nine sequential checks before it reaches the AI provider. Fail any one and the request is rejected before a cent is spent.

I did not want to take my own word for any of this. I ran the relay through Zapper, my own security pipeline, which puts deterministic scanners over the code, DAST, dependency and static analysis among them, maps every finding to recognised frameworks, and proposes concrete defence-in-depth fixes I can work through. On top of that I had Claude Opus carry out independent security reviews of the same code. The deterministic scanners catch what a model overlooks, and the model reasons about what a scanner cannot express. Running both, rather than trusting either alone, is what built my confidence in the security of the relay.

That is the lesson Zapper exists to enforce: no single control, and no single reviewer, covers every angle. I wrote about building it in Defence in Depth Can't Be Delegated to an LLM.

The Guardrails Problem

This is the part I thought about most carefully. Slam poetry regularly addresses racial violence, police brutality, mental health crises, sexual identity and war. Block that content and the app is useless for its audience. Moderate nothing and you cannot pass App Store review, with no defence against genuine abuse.

The answer is flag-and-log, not block. Almost everything passes through to the AI provider, but every request runs through a safety classifier via Cloudflare AI Gateway, and the classification is logged. Each hazard category gets one of three dispositions.

At the Cloudflare layer, a flagged poem is analysed normally and the poet sees nothing of it. The flag exists only in the Cloudflare dashboard for monitoring, an audit trail I can show Apple if asked. The classifier costs fractions of a cent per evaluation, effectively free.

That is the policy I set. It is not the only one in the path. The model runs on Microsoft Foundry, and Azure enforces its own default guardrail, Microsoft.DefaultV2, on every prompt and every response. It is stricter than mine, and I do not control it.

On paper that looks like a problem. The Azure default blocks hate, self-harm, sexual content and violence at a medium threshold, the very themes slam poetry lives in. In practice it has not been. When I evaluated sensitive poems across those topics, the model layer handled them without tripping the filter. A poem treats grief or violence differently from the material the threshold is built to catch, and it tends to score below it. The Azure guardrail sits there as a backstop, not a gate.

That is defence in depth working as intended: a permissive policy I set at the edge for a creative tool, and a conservative default the provider enforces at the model, two independent layers from two vendors. On the rare occasion Azure does refuse a request, the poet gets a clear message pointing them to the on-device model, which runs with no network and no content restrictions. Catching the genuinely unacceptable content at the Cloudflare edge earns its keep here too: fewer requests reach the Azure filter, and a request shut down at the edge is one fewer chance of tripping the provider and putting the deployment's standing at risk.

Flag-and-log is also what threads the needle for App Store review. Apple wants to see that you have content moderation. They do not want to see you blocking poems about depression or protest. My edge policy does both, and the audit trail is there if anyone asks.

How the AI Actually Analyses the Poem

The relay does not forward the poem verbatim. It prepends line numbers and wraps the text in a tuned prompt that tells the model exactly what to look for and how to format the response.

1: You memorised every word of the play
2: and never once stepped on the stage
3: stood in the wings for six months
4: holding a costume that fit you perfectly.
5:
6: This poem is for the understudies,
7: the ones who were ready,
8: the ones the audience never needed,
9: the ones who showed up anyway.

Line numbers matter. Without them the model says “in the second stanza” and the app has to guess which lines that means. With them it says “lines 7 to 9” and the highlighting is precise.

Each detection cites its lines and a confidence score. The poet saves or dismisses.

For long poems, over 40 lines, the relay splits the text into overlapping chunks at stanza boundaries, sends them in parallel, and merges the results, de-duplicating detections that appear in the overlap zone. The chunker is a hand-translation from the Python evaluation pipeline that validated the model's accuracy, and a layer of replay-based parity tests ensures the two implementations never drift apart.

The same screen teaches as it analyses. Tap a technique and a short explanation appears, so a poet who has never heard the word “parallelism” can still understand why the model flagged those three lines. The detection is the hook. The definition is the lesson.

There is a line I was careful not to cross. The AI analyses; it does not create. It detects the techniques a poet has already used and explains them. It never writes a line, rewrites a line, or hands the poet words. The craft stays the poet's. To keep that honest in the interface, every AI-detected technique carries a sparkle marker, so it is never mistaken for something the poet wrote or annotated themselves.

The Eval Pipeline

Before shipping any of this I needed to know whether the AI actually finds the right techniques, or whether it hallucinates patterns that are not there. So I built a 64-poem evaluation corpus with hand-annotated ground truth, every technique marked by a human, line by line. Then I ran 45 evaluation runs across 11 models, from frontier cloud models down to small ones that fit on the phone, measuring precision, recall and F1. The shortlist:

The frontier models were the most accurate. Claude Sonnet led on accuracy and recall, but it costs many times what DeepSeek does, too much to sit behind a one-time app and run on every analysis. At the other end, the on-device Gemma models were free but weaker, and, as I found earlier, not fast enough on a typical phone. Several budget options ruled themselves out: GPT-4o-mini only ever ran as a pilot, Phi-4 Mini returned nothing, Mistral emptied out on sensitive poems, one model rejected the prompts outright, and another kept returning malformed JSON.

DeepSeek sat in the sweet spot. It matched Claude Haiku on accuracy at a fraction of the cost, and, the detail that mattered most for an app full of poems about trauma, violence and identity, it never content-filtered a sensitive poem. None blocked across the whole corpus, where Haiku blocked one and the budget models refused or emptied out. Good-enough accuracy, near-free, and no moralising about the poetry. I ran the selection on DeepSeek V3.1 and later moved to V3.2, which the relay uses today.

With the model settled, the rest of the accuracy came from the pipeline. The detector-only setup won on the metric that users actually feel: higher recall, which reads as “the AI found the techniques”. It also costs half as much and runs faster, with no second verification call.

The evaluation also killed an idea that sounded good on paper. A two-stage pipeline, detect then verify each detection with a second call, was a measured no-op across 12 runs: the verifier added latency and cost without improving accuracy. No measurable change.

Nine techniques survived: anaphora, epistrophe, anadiplosis, parallelism, tricolon, asyndeton, antithesis, rhetorical question and volta. Each detection surfaces as “Suggested” in the app, and the poet confirms or dismisses. A noisy precision boundary does not degrade trust when the human has the final say.

And at a fraction of a cent per poem, this is the answer to the question I started with. A small model, kept honest by a human in the loop, can deliver enough value to be worth having, at a price a one-time app can carry. The frontier model was never the hard part. Getting most of the way there for a fraction of the cost was.

Two Lenses on the Same Poem

Rhetoric detection is one lens. The second is critique. Where the rhetoric lens returns tagged techniques on numbered lines, the critique lens returns prose: what lands, what a line is reaching for, where the poem might breathe differently when read aloud. Both run through the same relay, draw from the same pool of 10 free uses, and treat the poet as the editor, not the edited.

Critique on iPhone.

The same critique, in split view on iPad.

The Stack

Lessons Learnt

What I have built is a reusable pattern: a mobile app that reaches large models through a thin, hardened web layer in between, never directly. The device holds no keys. The relay does everything that has to be trusted, authentication, rate limiting, safety, credit accounting, and the routing to whichever provider sits behind it. None of that is specific to Spokenword or to poetry. Any app that needs to put a model behind a button can adopt the same shape, harden it once, and reuse it across builds and deployments.