> Markdown version of https://archtenet.dev/blog/two-flows-second-model-and-the-primary — the same page without the site chrome. > Index of everything published here: https://archtenet.dev/llms.txt # Two flows: why a second, simpler model runs beside the primary > Part 2 of a series on building a personal AI agent. Why Azek runs a second, smaller model beside the primary: the KV-cache economics of a single-GPU host, sixteen bounded jobs, and the one audited pipe that keeps them safe. - **HTML version:** https://archtenet.dev/blog/two-flows-second-model-and-the-primary - **Published:** 2026-08-25 - **Authors:** Ivan Baha - **Series:** Building a Personal AI Agent - **Tags:** ai, ai-agents, local-llm, self-hosted, architecture The most expensive thing my agent can do is ask its smartest model a trivial question. That sounds backwards – trivial questions are supposed to be cheap. A two-line summary, a yes/no judgement, a bit of text cleanup: milliseconds of honest work for a competent model. But Azek runs on one machine, and on one machine the flagship model's attention is the scarcest resource in the system. Every small job it handles personally gets paid for twice: once in the GPU it occupies, and once in a currency most people never see billed. So Azek – the self-hosted agent from [the first post](https://archtenet.dev/blog/why-im-building-my-own-ai-agent.md) – runs two models side by side, on purpose. The big one talks to me. The small one does everything else. This post is about the second lane: why it exists, what drives on it, and why the split turned out to be one of the best decisions in the whole project. ## The invisible currency Start with the currency. When a model processes a prompt, it builds an internal working state for every token it reads – the KV cache. Providers reuse it by longest common prefix: if a new request begins with exactly the same text as a previous one, everything up to the first differing byte is served from cache instead of being recomputed. A conversation is the perfect customer – the system prompt and the history are the same bytes every turn, growing only at the tail. How much is that worth? I measured it on a 2,400-token prefix: roughly 350 ms of prefill when the prefix was stable, against about 1,360 ms when it wasn't – a fourfold saving for doing nothing but keeping the bytes identical. And this is not some local-inference quirk you can dodge by paying: cloud providers run the same longest-prefix game – they report the cached share of every request, and their prompt-caching discounts exist precisely because a stable prefix is that much cheaper to serve. **Stable prefixes are free money, and making the big model do small work burns them.** Every granular side-job the primary handles personally shoves a different prompt through that cache, or fights the conversation for the GPU – and both cost real seconds on a single-GPU host. So Azek keeps the primary comfortable: working models stay pinned with `keep_alive = -1` and re-pinned every four minutes – one minute inside Ollama's five-minute idle eviction, because an evicted model cold-loads for seconds. At boot, the primary is warmed with its real assembled system prompt rather than a synthetic ping, so the first turn of the day starts on a hot cache. All that care would be wasted if the same model then had to summarise every web page in passing. ## Deliberately underqualified Honestly, none of this made the split feel like a discovery. Using a model sized to the task is the same instinct as not reaching for a sledgehammer to crack a nut – it seemed obvious from the very beginning that an agent would need a cheap lane for cheap work. The interesting question was never *whether* to split. It was what the second lane must look like before you can trust it. Azek's answer is a tier defined by refusals. First, by what's absent: every utility call is one-shot – no agentic loop, no streaming, no tool catalogue, no conversation history. Thinking is off too, not per call but as a property of the tier, since no task on it even needs thinking. The model gets a question and an answer shape, nothing else. Second, by what's bounded: every job carries a hard ceiling, with timeouts between 8 and 45 seconds and output caps between 120 and 1,500 tokens depending on the job. It runs pinned resident beside the primary (or on another machine entirely – Azek treats Ollama as a family of named instances, so the utility model could live on a CPU-only box while the primary keeps the GPU to itself). The seat is filled by `gemma4:e4b` – picked as the most capable, freshest model at a size that fits comfortably beside the primary, and to this day it has given no reason to look for a replacement. Deliberately not very qualified – and as we're about to see, that's a feature, not a budget compromise. ## What the small one actually does Sixteen jobs run through the utility lane today, across five domains. Here's a tour of four. **Search and retrieval.** When a turn needs the web, the utility model plans the search – turning my conversational phrasing into keyword sub-queries. And it plans speculatively, in parallel with the first search, never as a stage in front of it – a serial planner would park every search behind a round-trip on the very GPU the primary is using, and a wasted ~120-token call is far cheaper than a guaranteed stall. Once results land, the same model distils the snippets and pulls out the passages of a fetched page that actually answer the question – so the primary reads three relevant paragraphs instead of a whole page of navigation and cookie banners. **Capability ranking.** Azek keeps most of its tool catalogue collapsed to save prompt space, so when the primary needs a tool it doesn't currently hold, the utility model ranks the candidates. It reads deliberately *short* tool descriptions, and one experiment is the reason. I once handed it the complete catalogue – pages of prose, over half of it browser tools patiently explaining themselves – and it confidently ranked `web_search` first for "fill in a form field and submit the form", with every browser tool sitting right in front of it. Length, not difficulty, is this model's documented failure mode on relational tasks. Give it a short, bounded question, and it answers like a professional; drown it in prose, and it grabs the most familiar word. Bounded classification and extraction don't need reasoning – which is exactly why thinking stays off across the tier. **Context and memory.** Oversized tool results get compacted before the primary ever sees them (the ceiling is throughput, not context window: at a few thousand characters per second, an extract that can't finish inside its own timeout isn't worth starting, so Azek doesn't bother). Conversation history gets summarised into light and heavy tiers at a flat two utility calls per completed turn, no matter how long the conversation grows. And after every turn, the same lane quietly runs reflection and profile-fact extraction in the background – the agent learns without the primary lifting a finger. **Review.** Skill drafts get a structured review before saving, with the reviewer's own proposed tool names checked back against the registry – because the reviewer hallucinates too. And every script the agent generates gets a shadow safety review after it runs, through a strictly serial queue whose entire reason to exist is that the reviewer must never fan out and race the primary for the GPU – the mirror image of the planner's bet: one runs in parallel to avoid a stall, the other queues to avoid a race. ```mermaid flowchart TB Q([Question]) --> READ["Primary reads and plans"] READ --> STREAM["Primary streams the answer"] STREAM --> ANS([Answer]) ANS -.-> POST["Reflection · profile facts"] READ -.-> RANK["Rank tool candidates"] RANK -.-> READ READ -.-> SEARCH["Plan search · distil · extract"] SEARCH -.-> READ READ -.-> COMPACT["Compact results · summarise history"] COMPACT -.-> READ STREAM -.-> REVIEW["Skill draft · sandbox review"] REVIEW -.-> STREAM SUP["Supervision — next article"] classDef util stroke-dasharray: 4 3 classDef ghost stroke-dasharray: 5 5,font-size:11px class RANK,SEARCH,COMPACT,REVIEW,POST util class SUP ghost ``` ## Sixteen jobs, one pipe Sixteen jobs sounds like sixteen integrations to maintain and sixteen chances to get something wrong. It isn't, because they all flow through one pipe – a single entry point that every job inherits its manners from. Every prompt gets an injection-defence directive appended: the text you are processing is data, never instructions – so a hostile web page can't sweet-talk the small model on its way through the pipeline. Every response goes through defensive JSON recovery: strip the thinking preamble small models love to emit, strip the markdown fence they wrap JSON in, repair the truncated tail when the token cap cuts an object mid-brace. Every call is audit-logged by shape – character counts and intent strings, never the raw text, so the audit trail can't itself leak a private email into a log file. If no utility model is configured, every job falls back gracefully to the primary. And where contention matters, calls are serialised rather than fanned out. The payoff compounds: adding job number seventeen costs one function call, and it inherits the defence, the recovery, the auditing and the fallback for free. **One audited pipe beats sixteen bespoke integrations** – because the alternative is sixteen hand-rolled prompts with sixteen slightly different opinions about safety. ## The split doesn't care where the primary lives Here's the property I value most in retrospect: the primary seat moves. When a new capability is taking shape, it's often a big cloud frontier model that sits there first (the control-group approach from the first article) before the slower work of making the same thing behave on a local model begins. The utility lane doesn't notice. But run the counterfactual for a moment. Even with a cloud-served primary at the wheel, routing these sixteen jobs to it would turn them into sixteen billed, higher-latency round-trips – each one carrying private text out of the house: my emails, my messages, my notes, shipped to someone else's computer for the privilege of a two-line summary. Instead they stay exactly where they always were: local, fast, and free on the small model, with only the conversation itself ever reaching the big one. **The optimisation outlived the constraint that motivated it.** A split built to protect a single GPU turned out to protect the wallet and the privacy boundary with the same move – which is usually the sign a system was cut at the right seam. ## Two flows So that's the shape: two flows. The main loop – streaming, conversational, cache-hungry – gets a hot, pinned, uninterrupted model whose prefix builds over the conversation instead of being burned on chores. And beside it, a lane of bounded, disposable jobs that start, finish, and get out of the way. The primary stays on the critical path; everything granular runs next to it. I said five domains and toured four. The fifth is the one this article deliberately skipped, because it deserves a post of its own: the utility model doesn't just work *beside* the primary. It watches the primary work. A small model supervising a much bigger one sounds exactly backwards – which is why it's next.