a·v·r·i·z — Engineering Notes

a·v·r·i·z is a voice-native agentic IDE. Instead of a laptop and an editor, each user gets a private Linux machine in the cloud with a coding agent living on it. You describe what you want — out loud, from your phone — and the agent writes the code, runs it, tests it, deploys it, and tells you when it's live.

The whole product is three long-lived services plus one machine per user:

The guiding principle throughout: one isolated machine per person. No shared sandbox, no shared model keys leaking across tenants, nothing you build training someone else's model. Most of the harder design decisions fall out of taking that isolation seriously.

The stack is intentionally boring where reliability matters: AWS for the substrate, Python 3 + FastAPI for the control services, Caddy for the public edge, Stripe for billing, and static HTML/CSS/vanilla JavaScript for the product UI. The interesting part is how those pieces are composed around one private cloud machine per user.

The result is a small surface area: Caddy handles public HTTP and certificates, Python owns product logic, AWS owns machines and deployment plumbing, Stripe owns money movement, and the frontend stays close to the APIs it is operating.

Everything is in one AWS region. The control plane is the only component with broad IAM; boxes are deliberately kept powerless (more on that under Security).

Every box is reachable at <name>.t.avriz.io. The control plane fronts all of these subdomains and proxies each request to the right box.

The subdomain label started as an 8-character hex id equal to the internal tenant id. That works but isn't memorable, so we split the two concepts:

• tenant — a stable internal id. It keys everything durable: the routes table, the box's credentials, the deploy token (an HMAC of the tenant), the DynamoDB memory partition. It never changes.
• slug — a memorable adjective-noun-noun label (e.g. brave-otter-cove) that is only a URL alias. New boxes get one at provision time; existing boxes can regenerate one from the dashboard.

The proxy resolves an incoming subdomain by slug OR tenant, so both the friendly URL and any legacy hex URL keep working. Because the slug is decoupled from the tenant, renaming a box never re-keys anything — memory, tokens, and DynamoDB are all keyed on the immutable tenant. This decoupling is the whole reason "get a new name" is a safe, instant operation instead of a migration.

TLS is issued on demand: when a request arrives for a subdomain, the edge asks the control plane "should I get a certificate for this?" and cp answers yes only if that slug/tenant actually exists in the routes table. Guessable names can't be used to exhaust certificate issuance.

A dedicated always-on VM per user would be expensive and mostly idle. So boxes scale to zero:

• A reaper in the control plane stops any box whose last activity is older than ~15 minutes.
• Visiting a stopped box shows a branded "your box is asleep" page with a Wake button; clicking it starts the instance (~30–60s) and reconnects.
• Every plan includes a pool of active hours — you only burn time while the box is actually running, so an idle project costs nothing.

A subtle rule learned the hard way: only the explicit Wake action starts a stopped box. Earlier, any mutating request could wake it, so a stray background poll from a cached tab would wake the box before the user even asked for it. Now a sleeping box answers navigations with the wake page and everything else with a 503 — nothing but the button spends money.

The box runs the coding agent: the open-source Goose agent from Block, embedded as a first-class engine with its MCP tools, recipes, and skills. The product started with a home-grown tool-loop ("Classic") that drove an LLM through read/write/run/git tools; Goose shipped alongside it as an opt-in, became the default, and has since replaced Classic entirely — one engine, less surface area, and every improvement lands in the agent everyone actually uses. Voice, durable memory, deploys, and PRs all carried over.

Wiring Goose in: ACP over a persistent session

Goose speaks ACP (Agent Client Protocol) over a WebSocket. On the box we run goose serve as a systemd service (goose-acp), and the box app bridges the browser's Server-Sent-Events stream to the Goose WebSocket:

Each turn maps ACP session/update notifications (tool_call, agent_message_chunk, usage_update) into simple SSE events the UI renders. Tool permissions are auto-approved (GOOSE_MODE=auto) so a headless voice session never blocks on a prompt.

Two layers of memory, and why both

A recurring lesson: "remembering" is actually two different problems.

1. Conversation context — does the agent still know what we were just talking about? Goose keeps its session in a store on the box's EBS volume. Since EBS survives stop/start, we persist the session id and session/load it on reconnect, so waking a box (or a deploy that restarts the app) resumes the exact conversation rather than starting blank.
2. The visible chat window — does reopening the tab show the prior messages? That's separate. Each turn is also appended to a durable transcript in DynamoDB (see Memory), and the UI repaints it on load. So the agent remembers (session resume) and the window repopulates (transcript) — together they match a native chat experience.

On top of that there's durable long-term memory: facts the agent chooses to remember (remember/recall/forget). For Goose this is a dependency-free stdio MCP server on the box that calls the control plane's memory API — so Goose gets cross-box, survives-rebuild memory without ever handing the box any database credentials.

We sell a managed model option: the user doesn't bring a key, we route their calls and meter them. The hard constraints:

• Managed keys must never reach the box. A box is the user's machine with a root shell — anything on it is readable. So keys stay exclusively on the control plane.
• Cheap work must not pay frontier prices. Most agent turns are trivial (read a file, run a command). Sending all of them to the biggest model would be wasteful.

The path:

The box always sends a placeholder model name; the control plane's grader looks at each turn, classifies its difficulty with a fast model, and routes to the cheapest capable tier on an auto ladder. Users can pin a specific model instead of auto.

Two economics safeguards live here:

• Native prompt caching. Managed Anthropic traffic is translated to the native Messages API with cache_control on the conversation prefix, so repeated context is billed at the cached rate. This lowers our cost and stretches each plan's bundled allowance. (Gemini caches its prefix implicitly and reports the hit back as cachedContentTokenCount, so cached tokens are metered the same way across providers.)
• A spend gate. When a user has exhausted their bundled allowance and has no credits, the proxy returns 402 and the agent stops — a runaway loop can't run up an unbounded bill.

The grader in detail: one graded turn, memoized

A "turn" is a distinct latest user message. The grader classifies it once — a 16-token judge call on the cheapest capable model (Gemini Flash, ~3× cheaper than Haiku) that must answer with exactly one of trivial / simple / moderate / hard / expert — and that verdict is memoized per tenant (keyed on a hash of the last user text, 10-minute TTL). Every follow-up model call inside the same turn — tool loops, continuations — reuses the verdict instead of re-grading, so a fifty-step agent turn pays for grading once. Each grade maps to a rung on the auto ladder: trivial→gemini, simple→fugu, moderate→sonnet, hard→opus, expert→fugu-ultra. If the judge call fails or times out (4s budget), we fall back to the cheapest rung rather than the most expensive — a grading outage makes us lose margin, never overcharge the user. A per-tenant pin (or admin default) short-circuits the grader entirely.

Two robustness details earned their place the hard way: Gemini Flash will spend its whole 16-token budget on hidden "thoughts" and return no word unless we pass reasoning_effort: none — without it every grade silently failed and mis-routed everything to the cheapest tier. And if the routed upstream is out of money (Anthropic 402, Sakana 429 usage_limit_reached), the proxy climbs to the next funded rung instead of handing the agent an empty turn.

How often we actually grade: once per turn, not per call

Grading is not run on every request that hits the proxy. _resolve_model checks, in order: a per-tenant model pin (skips grading entirely) → an in-memory per-tenant memo keyed on sha256(last_user_text)[:16] → only then a fresh judge call. So a single human message that triggers a fifty-step tool loop is graded exactly once; every follow-up call in that same turn reuses the memoized verdict as long as the message text is unchanged and the memo hasn't expired. A genuinely new message (different text, different hash) always grades fresh — the memo can't mask real turn changes.

The memo TTL is currently 10 minutes, and it's worth being precise about which direction that dial moves cost: the TTL only matters for the rare case where a turn's tool loop runs longer than the memo window and would otherwise re-grade the identical text mid-turn. Raising the TTL means fewer of those redundant re-grades (cheaper); lowering it makes long-running turns more likely to re-grade the same message before they finish (more judge calls, not fewer). It is a cache-freshness knob, not a billing-interval knob — it doesn't change how often new messages get graded, only how long a slow turn can go before paying for a second, wasted judge call on the same text.

The judge prompt: forcing one word out of a chat model

The judge call is deliberately tiny: a system prompt ("You rate how hard a coding-agent request is. Reply with EXACTLY one word: trivial / simple / moderate / hard / expert") plus the raw last-user-message text (truncated to 4000 chars), a max_tokens: 16 cap, and nothing else — no tool schema, no conversation history. That's what makes it cheap: it's a single-turn classification, not a re-run of the agent's context. Parsing the reply is deliberately loose rather than an exact-match: we substring-search the lowercased response for each tier name in order, and if that fails, we also check whether the model just echoed back a raw model id (e.g. it said "sonnet" instead of "moderate") and map it directly. Judge model preference is gemini first, then haiku — Gemini Flash is roughly 3× cheaper per token than Haiku, so grading itself barely dents the margin it's protecting; Haiku is the fallback if the Gemini account isn't configured.

Two different "give up" paths: unconfigured vs. out-of-money

There are two distinct failure modes for a chosen model, and they degrade differently on purpose:

• Not configured (no API key set for that tier, or it was graded but never enabled) resolves via _avail_or_up: walk up the same easy→hard capability ladder from that rung until something with a key is found. The intent is "closest available capability," so a missing fugu tier still gets routed to something in the neighborhood of "simple," not straight to the frontier.
• Out of money at request time (Anthropic 402, Sakana 429 usage_limit_reached) resolves via _fallback_order: this ignores the capability ladder entirely and instead sorts every configured model by wholesale rate (input + output $/token) and tries cheapest-first, skipping any other tier that shares the same exhausted account (so if the Anthropic key is dead, every Claude tier is skipped in one shot, not retried one at a time). The reasoning: an account running out of credit mid-session is a billing event, not a capability gap — the priority shifts from "closest quality" to "cheapest thing that still answers," because the agent mid-turn needs a response more than it needs the originally graded tier.

Both paths funnel through the same non-streaming completion loop: it tries the graded model, and on an exhaustion signal keeps walking the cheapest-first list (marking that provider's account "dead" so sibling tiers on the same key aren't retried), until one answers or every option is exhausted. Only then does the caller see an error — never a silently empty turn.

Metering: the box reports tokens, the control plane prices them

The box can't be trusted to know — or to honestly report — which model actually served a call, because the grader's routing decision lives only on cp. So the split is deliberate: after each call the box posts raw token counts (in, out, cached) to /internal/usage with its tenant deploy token, and cp decides what those tokens cost. It resolves the actually-served model, prices the turn with cached input billed at the lower cached rate, and folds the counts into a per-tenant, per-month usage row (llm_in / llm_out / llm_cached / voice_chars / compute_s). The same call is also stitched onto the routing ledger row the proxy wrote a moment earlier, giving the admin Models tab a per-call trail of model · why-it-was-chosen · tokens · cost — metadata only, never a byte of chat content.

Where the numbers come from

Token counts are not estimated — they're the upstream's own accounting. Every managed call flows through the box's local /v1 proxy, which reads the provider's usage block off the completion (prompt_tokens, completion_tokens, and prompt_tokens_details.cached_tokens) and fires a fire-and-forget POST /internal/usage in a background thread — so metering never sits in the response path or slows a turn. Voice is metered the same way: /api/tts reports the character count actually synthesized (after code blocks are stripped for speech), keyed kind=voice. A report that fails to send is dropped rather than retried — losing a count is a rounding error; blocking the agent on a billing write is not.

Which model gets billed: the routing-row join

The box sends a model-agnostic request and learns nothing about where it was routed, so cp reconstructs the served model at metering time. The proxy writes a routing-ledger row the instant it forwards a call — tenant, chosen model, and why (grader verdict, tenant pin, admin default, cheaper-fallback after an exhausted account, cheapest-rung utility pin, or a mid-turn escalation) — but with zero tokens, because those aren't known yet. When the box's usage report lands a moment later, cp attaches it to the newest zero-token routing row for that tenant within a 900-second window and prices that row's model. This join is what makes the price exact even when the served model differs from the turn's grader verdict: a fallback to a cheaper funded model, or a utility-pinned classification call, is billed at what actually ran — not what the grader first picked. Only if no pending row exists (say cp restarted between forward and report) does it fall back to a best-effort resolution (tenant pin → that turn's grader verdict → account default) and write a standalone row.

Cached input is real money saved

Prompt caching isn't just a latency win — it changes the bill. An agent re-sends its whole conversation prefix every turn; with native prompt caching that prefix is billed at the cached-read rate (~10× cheaper than fresh input) and the box surfaces those cached tokens separately. Pricing splits them out: cost = (in−cached)×ratein + cached×ratecached + out×rateout. The wholesale rates cp meters against (USD per 1M tokens, the direct-provider list/pool prices):

Worked example — a Sonnet turn with 40K input (30K of it a cached prefix), 1.2K output: (10K×$3 + 30K×$0.30 + 1.2K×$15) / 1M = $0.057 wholesale. Bill the same turn without the cache breakpoint and the input alone is 40K×$3 = $0.12 — the cache turns a $0.138 turn into a $0.057 one.

What the box can and can't move

The trust boundary is narrow by design. A tampered box could misreport its own token counts, but it can't touch the two things that set the price — the wholesale rate table and the served-model decision both live only on cp — and it never receives a pricing table it could reverse-engineer or game (the box sees only its live balance and remaining allowance via /internal/usage-get). Under-reporting tokens only shrinks that tenant's own metered usage against its own allowance; it can't shift cost onto anyone else (metering is strictly per-tenant, keyed on the deploy token) and it can't defeat the spend gate, which is checked on cp before a call is forwarded, not from box-reported numbers.

Allowance → credits: a dollar budget, not a token budget

Each managed plan includes a monthly token allowance (defaults 2M in / 800K out, per-user overridable). Rather than tracking two separate token counters against two separate caps, we convert the allowance into a single dollar budget at the plan model's list rate and meter cumulative wholesale cost against it. Usage under budget is free; once the running cost crosses the line, only the incremental slice of each turn that sits beyond the allowance is charged to prepaid credits (1 credit = $0.01), at the LLM margin (4× wholesale ≈ 75% gross margin), billed at the model that actually served it. Metering the marginal overage per turn — instead of re-charging the whole cumulative total — is what keeps the boundary crossing from double-billing. Compute time (box running-hours beyond the included budget) meters onto the same credit balance through a parallel path.

The lifecycle around that is all cp-side: threshold emails fire once each at 80% and 100% of allowance and when credits run low; at 100% usage transparently switches from allowance to credits; and when both are exhausted the proxy's 402 spend-gate stops the agent. The box only ever sees its live balance and remaining allowance via /internal/usage-get — never a pricing table it could game.

Learning the router: RL in shadow mode

The grader is a good hand-built policy, but it is still a hand-built policy — the tier mapping, the two-word judge prompt, the (12, 48) mid-turn escalation thresholds, "grading failure → cheapest rung" are all numbers a human guessed. And it has a failure mode we can name from its own code: the judge sees only the message text, so a terse "fix the login bug" can grade trivial and pin a fifty-step debugging loop to a weak model for the whole turn. That is exactly the shape of problem reinforcement learning is for. Every turn, _resolve_model makes one decision — given this request, which rung? — and the outcome (did it resolve, at what cost, did it have to escalate) is a reward the router could learn from instead of a threshold we tune by hand. Framed that way, routing is a textbook contextual bandit: context = the request, action = the rung, reward = did it go well and what did it cost.

This runs in pure shadow — it does not change what any user gets. On every graded turn, control-plane/rl_router.py records what a candidate policy would have routed, alongside the live judge's actual pick, in a router_shadow ledger. The candidate sees structured features the text-only judge can't — message length, pasted code fences, tracebacks, referenced file paths, hard-vs-easy verbs, the tenant's recent escalation rate, the prior turn's rung. Crucially, no chat content is stored, only the derived feature vector, the same metadata-only discipline the routing ledger already follows.

The reward is backfilled from data we already keep. A turn's llm_calls rows — bounded by the tenant's next decision so back-to-back turns don't bleed together — are aggregated into success − cost − mid-turn-escalation − hard-failure: the soft, self-healing routing events (exhaustion, empty completions, auth rolls) don't count against the turn, but a turn the grader under-routed so badly that _ESCALATE_AT had to bump it mid-flight does. That escalation signal is the headline: it is a direct, non-counterfactual measure of the judge's own mistakes, and "would the candidate have started that turn higher?" is a win we can prove without ever having served the candidate's choice.

Those rewards actually train a model — this is where the RL stops being a metaphor. Each rung is an arm with its own linear model of estimated reward, ra(x) = wa · x, fit online (SGD ridge, pure Python, no numpy) as rewards land in the backfill pass. Because we can only ever observe the reward for the rung the live judge actually served, each arm is trained off-policy — it learns its value only from the turns that genuinely landed on it — and the candidate then picks argmaxa ra(x) over the arms that have enough data. This is the regression / "direct method" contextual bandit (LinUCB without the confidence matrix). The hand-tuned scorer from the previous paragraphs doesn't disappear: it becomes the prior and the cold-start. The learned policy stays dormant until it clears a data bar (≥30 turns on each of ≥2 arms, ≥300 total), so early shadow behaviour is identical to the heuristic and only shifts to the model once real reward has taught it something. The admin report shows exactly which state it's in — warming up (heuristic prior) vs bandit driving — with per-arm sample counts.

An admin-only report (Models tab, /admin/api/router-shadow) backfills rewards on demand and summarizes: judge-vs-candidate agreement and the full confusion matrix, realized reward on the agreement subset (a sanity check that the reward signal is sane before we trust any delta), the escalation-catch rate, and an honest counterfactual cost estimate — labeled an estimate, because on the turns where the picks diverge the candidate's rung was never actually served, so its cost is modeled from that rung's realized mean, not measured. The whole thing is gated by a router_shadow config flag and wrapped best-effort so it can never break serving.

The last piece is serving, and it now exists: a router_bandit_pct knob routes that fraction of graded turns via the trained bandit instead of the judge, with a router_bandit_max ceiling capping how high it may route and the existing cheapest-funded fallback still underneath as the safety net. It ships default-off (0%) and is a hard no-op until the bandit clears its data bar, so a deploy changes nothing until an admin deliberately dials the fraction up in the Models tab. The moment it does, the loop closes on-policy: turns the bandit serves are logged with live_why = 'bandit', so their reward is measured (not estimated) and trains the policy on its own decisions — and the report grows an on-policy A/B block comparing bandit-served vs judge-served reward, success and cost head-to-head. That measured comparison, not a hunch, is what justifies widening the fraction. The design intent throughout: the learned policy earns each increment of real traffic, and every step from observation to full serving is one flag away from being rolled back.

The RL router in the previous section is one feature, but the shape underneath it is the more valuable thing, so it gets its own section. The platform now contains a complete, closed learning loop that runs on production traffic — and the loop is a pattern, not a one-off. Any decision the control plane makes over and over with a measurable outcome can be run through it. Routing was simply the first decision worth the trouble.

The loop, end to end

Each stage was shipped separately, shadow-first, and each is independently reversible — the loop was assembled in production without ever being live in production until deliberately switched on.

One brain across the fleet

The deliberate choice here is that the bandit is fleet-shared, not per-tenant. Every managed box routes through the same control-plane proxy, so every subscriber's turns pool into one policy. This matters for a cold-start reason and a compounding one. Cold start: a single tenant might produce a few dozen graded turns a week — no individual policy would ever clear the activation bar (≥30 turns per arm, ≥300 total). Pooled, the platform clears it quickly, and every tenant benefits from day one. Compounding: a lesson extracted from one tenant's mis-routed turn — "terse messages that mention a traceback spawn fifty-step debugging loops; start them higher" — immediately improves routing for everyone. Each new subscriber makes the router smarter for all existing ones. That is the flywheel: usage → reward data → better routing → better margins and better answers → more usage.

Fleet-shared weights don't mean uniform behaviour. Per-tenant character enters through the features — the tenant's recent escalation rate, the prior turn's rung — so the same shared model still routes a tenant who habitually asks for deep refactors differently from one who mostly chats. Shared parameters, personalised context: the standard trick for learning across users without a model per user.

The honest caveat that comes with pooling: early on, the policy's opinions are dominated by whichever tenants are most active, because they contribute most of the reward data. Until the data broadens, the guardrails carry the risk — the serve fraction starts tiny, the router_bandit_max ceiling bounds how high an over-eager policy can route, and per-plan floors/ceilings keep it inside each plan's entitlement.

Where the learning stops, on purpose

The loop covers exactly the traffic where the routing decision is ours to make — the managed auto ladder — and is structurally excluded everywhere the user or the plan has already decided. A tenant who pins a model short-circuits the resolver before the grader or the bandit is consulted. BYO-key boxes talk to their own provider and never enter the proxy, so they contribute no data and receive no routing opinions. Utility passes stay pinned to the cheapest rung. Mid-turn follow-up calls reuse the turn's decision — the policy decides once per human message, like the judge it augments. These aren't gaps; they're the boundary drawn where user intent outranks learned opinion.

Learning without reading

The whole loop runs on metadata only. What's stored per decision is a feature vector — message length, has-a-code-fence, has-a-traceback, count of file paths, hard-verb and easy-verb counts, question-mark, prior rung, escalation rate — plus the served model and the outcome. Not one byte of chat content, matching the discipline the routing and billing ledgers already follow. The platform learns what its users' work costs and needs without retaining what it says. This is also what makes fleet-pooling defensible: the shared policy carries no tenant's words, only the statistical shape of difficulty.

Why this is the interesting part

Hand-tuned heuristics rot: the (12, 48) escalation thresholds, the judge's five-word taxonomy, "grading failure → cheapest" were all sensible guesses that would have drifted out of date silently as models, prices and usage changed. A learned policy re-fits continuously against the thing that actually matters (realized reward per dollar), and — because the reward folds in wholesale cost — it optimizes the platform's margin and the user's experience as one objective rather than trading them off by hand. And the same ledger → reward → bandit → flag-gated-serve pipeline is sitting there for the platform's other repeated decisions: which memory facts to inject into a turn (reward: did the turn go better?), when to sleep an idle box (reward: compute saved vs cold-wake pain), even which voice to synthesize with. Routing proved the pattern on the hardest constraint — live traffic, real money, zero tolerance for breaking serving. The next learned decision gets the pipeline for free.

📄 The formal treatment — problem formulation, estimator and update rule, activation and serving math, figures, and references — is written up as a technical report: Learning to Route: A Shadow-First Contextual Bandit for Per-Request LLM Model Selection (AVZ-TR-2026-01).

Durable memory is a single DynamoDB table, partitioned per tenant (pk = t#<tenant>, sk = fact#<id> or conv#current).

The important decision is who holds the credentials. A shared box IAM role couldn't isolate one tenant's memory from another's — any box could read any partition. So boxes get no DynamoDB access at all. They call the control plane's /internal/memory API with their tenant token; cp does the DynamoDB I/O scoped to that tenant. Isolation is enforced in the one place that can enforce it.

Because it's in DynamoDB and not on the box, memory survives box rebuilds and is available from any box the tenant is routed to.

Passive learning: mining preferences from every turn

Memory started as an explicit tool — the agent saved a fact only when the user said "remember this" or clearly stated a preference. That misses most of what's worth learning. So after every completed turn the box fires a fire-and-forget background pass (_extract_preferences) that runs one small judge call over just that turn — the user message plus the agent's reply — asking for durable preferences, conventions, decisions, or corrections the user made. It returns a strict JSON array (max 3 facts, [] for most turns), and the box saves anything new.

Three design choices keep it cheap and safe: it's out of the critical path (a daemon thread, so the user's reply never waits on it); it skips trivial turns (very short messages rarely teach anything); and it's deduped against known facts — the current memory is passed to the judge as a "do not repeat" block, and a case-insensitive check drops anything already stored. Like every other model call, it runs through the local metered proxy, so provider keys never touch the box and the pass is priced and graded like any turn. The net effect: the agent quietly gets to know how you work — that you prefer python3, that deploys go to main — without you having to tell it twice.

Auto-learned facts are held to a different standard than ones you saved yourself. Every fact carries its provenance (user vs auto), and the box's Settings → Memory groups the two so you can see exactly what the agent picked up on its own — and edit or delete any of it. Instead of a numeric confidence score there's a simple reinforcement clock: saving a duplicate fact refreshes the original's seen timestamp and bumps its hit count rather than duplicating it, and an auto-learned fact that hasn't come up in ~60 days quietly expires (user-saved facts never do). The renewal loop closes itself: facts nearing expiry are deliberately left out of the mining pass's "already known" list, so a preference that's still true gets re-extracted — which resets its clock — while one that stopped being true fades away. Editing an auto-learned fact promotes it to user-owned, ending its decay.

Speech-to-text turns a mic recording into text; text-to-speech turns the reply into audio (via a pay-as-you-go TTS API). The design rule users asked for: talk only when talked to. A reply is spoken only when the turn was started by voice; typed commands reply silently. The one deliberate exception is the Code menu's git-diff summary, which always reads aloud.

When the agent deploys something, it lands on apphost at <name>.avriz.app. Apphost is intentionally small: it takes a source tarball, builds it, and runs it as a Docker container with per-app CPU/memory limits. Builds run at low priority (nice) so a deploy never starves the apps already serving live traffic.

The build order is: a user's Dockerfile if present → nixpacks auto-detection (Python, Node, Go, …) → a static-site fallback. nixpacks provisions the language runtime inside the container, so apphost stays language-agnostic.

Two failure modes bit us and are now designed out:

• "Builds fine, crash-loops on start." An app whose start command references a binary that was never installed (classic: a Flask app with gunicorn in the Procfile but not in requirements.txt) builds successfully, then exits 127 in a restart loop — surfacing as a silent 502. Fix: before building, if a Python app's start command names a WSGI/ASGI server (gunicorn/uvicorn/…) that isn't in requirements.txt, auto-add it so nixpacks installs it.
• A dead container behind the proxy. As a safety net, if a container isn't staying up shortly after deploy, apphost falls back to serving any index.html as a static site, or reports the crash log — never a bare 502.

Boxes themselves also ship with node, npm, python3, and common Python app servers preinstalled, so the agent can run Python and JS apps directly during development.

When a subscription goes active, the control plane launches one EC2 instance, tags it to the tenant, seeds it via cloud-init, and records the route. The box comes up with the box app, the terminal, the file manager, the agent engines, and the language runtimes.

A guardrail worth calling out: the control plane's IAM role can only launch t3.*/t3a.* instance types — anything else is denied at the policy level. And account recycling dry-runs the launch before destroying the old box, after an early incident where destroying first and failing the relaunch meant permanent data loss (boxes are DeleteOnTermination).

Everything is push-to-deploy. A commit to main triggers GitHub Actions, which ship via an OIDC role through S3 and SSM:

• control plane → the cp host.
• box app → running tenant boxes (rolled out live).
• apphost → ships only the agent, never the app data.

One operational gotcha that shows up repeatedly: box-app deploys roll out to running boxes only. A box that's asleep during a deploy gets the update on its next wake — so verifying a change often means waking the box first, then re-running the rollout. And because SSH is firewalled, all box operations go through SSM. Each artifact is also stamped with a VERSION (build date + short SHA) and a derived memorable codename, so a box can report exactly which revision it's running.

Sleeping boxes catch up on wake

"Gets the update on its next wake" is a real mechanism, not a hope. Every box's voiceide.service carries a systemd drop-in — ExecStartPre=-+/opt/voiceide/self-update.sh — that runs before the app starts, on every start, including a wake from stopped. The hook pulls the latest bundle from the control plane (GET /internal/bundle, authenticated with the box's own deploy token), extracts it over /opt/voiceide, and reinstalls requirements. The template it pulls is the same one the pipeline's update-template job refreshed on the cp host, so a box that slept through ten deploys wakes current in a single step — no rollout needed. Two prefixes make it safe: + runs the pre-step as root (it has to chown and pip install), and - means any failure — a network blip, a truncated tarball — is ignored so a botched update can never keep the box from starting. It's idempotent and strictly best-effort: the box either comes up newer, or comes up on exactly the code it already had.

The isolation principle, made concrete:

• Managed model keys never touch a box. They live only on the control plane, behind the metered proxy.
• Boxes have no database IAM. Tenant memory is isolated by the control plane, not by a role a box could abuse.
• BYO credentials stay scoped. A user's connected GitHub token is referenced from the box's environment, never baked into any config file that a raw-config UI could echo back.
• UI actions bind by data-attribute, not string-built handlers. Values that flow into the DOM (recipe names, file names, extensions) are attached via data-* + delegated listeners rather than inline onclick strings — a JS-string context an HTML escaper doesn't neutralize. This came from a real stored-XSS on a filename, and it's now the house pattern.
• The instance-type guardrail and dry-run-before-destroy protect against both cost surprises and data loss at the IAM layer.

Apphost started single-node in a way that couldn't grow: avriz.app and *.avriz.app DNS pointed straight at one box's elastic IP, and that box's agent looked each subdomain up in its local database and proxied to a local container. A second apphost would receive no traffic and its apps would be invisible. That coupling is now gone, in two phases plus a hardening pass.

Phase 0 — placement registry. The control plane learned where each app runs. An apphost column on the apps table records the host; a small pool abstraction (_apphost_pool/_pick_apphost) does round-robin placement for new apps and reaches an existing app on its recorded host. Apps are sticky — a redeploy returns to the same host, because the container and its per-app Postgres/Redis live there. Destroy, restart, logs, and status all route by the registry. On one host this is a no-op; it's pure groundwork.

Phase 1 — cp becomes the *.avriz.app front door. DNS moved off the apphost's IP to the control plane's. cp now terminates TLS for every app subdomain (on-demand issuance) and reverse-proxies each request to the apphost that runs it, preserving the Host header so the apphost routes to the right container. A key AWS detail shaped this: an instance can't reach its own public elastic IP from inside the VPC (hairpin NAT fails), so cp reaches the apphost over the private network — avriz.app is pinned to the apphost's private address in /etc/hosts, resolved fresh on every deploy so an apphost replacement can never leave a stale pin. Adding a second apphost is now a config change (add its address to the pool), not a rewrite.

Hardening + cleanup. The apphost's public IP was deliberately kept (its subnet routes egress through an internet gateway with no NAT, so that IP is the only outbound path for Docker/nixpacks/npm during builds) but its inbound was locked to the control-plane security group only — no public :80/:443, so apps can only be reached through cp. The internal hop was then simplified from HTTPS-with-an-un-renewable-cert to plain HTTP straight to the agent on the private network, removing a redundant proxy layer, a Sep-2026 certificate time-bomb, and duplicate response headers. Apps serve valid public HTTPS to users throughout.

The box UI carried two overlapping tabs: Code (git-aware, but a flat file list built around an editor) and Files (a real folder-tree browser, but blind to git). They're now one git-aware file manager, and the Code tab is gone — one fewer item in the mobile bottom nav.

The unified tab keeps every file-manager affordance (folder tree, create/rename/delete/move, upload/download, in-place editor, markdown and image preview) and makes git a first-class layer on top:

• Status badges on every entry — modified, added, deleted, renamed, untracked — with folders rolling up "changes inside." Status is computed against each entry's own enclosing repo, so it works even at the workspace root, which isn't itself a repo (the repos are subfolders, each shown with a rollup badge).
• A "changed" filter, a repo/branch header, and Commit / Open PR buttons.
• Per-file actions: view diff, stage, unstage, and discard (confirm-gated).
• An inline diff viewer with an AI summary — it explains the change in plain language and proposes a conventional-commit message you can commit in one tap, or open a pull request prefilled with that title and summary.

Backend-wise this reused what already existed (commit, PR, diff-summarize, branches) and added a small set of endpoints — an enriched directory listing that carries per-entry git status, plus stage / unstage / discard and a per-file diff, all repo-relative to the file's own repository. Three bugs surfaced only by exercising it end-to-end on a live box: empty diffs for untracked files (a status helper that dropped output on git's non-zero "there is a difference" exit code), a discard that didn't remove a staged-new file, and the workspace-root view emptying because the root isn't a repo. All fixed and verified before shipping.

A coding agent that can't see what it built is working blind. So the agent now drives a real headless Chromium on the box — it can open a page, wait for it to render, read the console, click through a flow, and screenshot the result (screenshots land in the box's Files). It's wired as a Playwright MCP server, so "go look at the page" is just another tool call in the same turn as the code that changed it — handy for confirming a deploy renders, or reproducing a visual bug.

Getting there meant chasing the wrong bug for a while. The browser extension was configured in Goose's config.yaml exactly like developer, memory, and the rest — yet the agent kept insisting it had no browser tools, that it was "text-based," that headless wouldn't work. Every one of those replies was the agent honestly reporting what it could see: the tools genuinely weren't loaded. The lesson was to trust the agent's account of its own capabilities and go find why they were empty, instead of arguing with it.

The root cause is a sharp edge in how Goose runs. When Goose is driven over ACP — which is how our box app talks to it — goose serve does not load the stdio extensions from config.yaml at all. Those are a CLI/desktop convenience; in ACP mode the client is expected to declare which MCP servers a session should have, by passing an mcpServers array in session/new (and again in session/load on resume). We were passing an empty array — so every session came up with only Goose's built-ins, regardless of what the config said. Playwright, context7, github, our own memory server: all silently absent.

The fix is a small function, _acp_mcp_servers(), that reads the box's Goose config, expands ${VAR} references, forwards the right environment (HOME, PATH, the GitHub token), and hands the resolved stdio server list to both session/new and session/load — the box app becomes the thing that furnishes the session, because in ACP that's whose job it is. One defensive detail earned from the outage: sessions start with a two-step fallback (_attempts = [_mcp, []]) — try with the full extension set, and if that handshake fails for any reason, retry with none rather than leave the user staring at a dead chat. A missing browser is a degraded turn; a session that won't start is a broken product.

Durable memory (§08) is short, personal, and cross-project — "use python3," "deploys go to main." It's the wrong shape for the other thing an agent needs to be good on a real codebase: project knowledge — a schema, an API contract, a house style, the runbook for a gnarly deploy. That belongs with the repo, in the repo, versioned alongside the code it describes.

So the agent keeps a knowledge base as a folder of markdown files — one concept per file, YAML frontmatter plus a body, cross-linked like a small wiki, with an index.md overview. The format is Google's Open Knowledge Format (OKF): plain, human-editable, portable, and — because it's just files — it travels with the repo through git and reviews like any other change. A house rule plus a Goose skill teach the agent to maintain it: when it learns something durable about the project, it writes or updates knowledge/<area>/<concept>.md in place rather than duplicating, and keeps the index current. The base gets sharper about your codebase the more the agent works in it.

The other half is bringing your own. Upload a coding standard, a spec, a style guide (More → Knowledge) and the agent treats it as authoritative — not a suggestion it might weigh, but a convention it follows. The upload reuses the file manager's existing /api/fs/upload endpoint, drops the document into the knowledge folder, and the same house rule that governs the auto-maintained base points the agent at your standards first. The distinction from memory is deliberate: memory is what the agent picked up about you; knowledge is the ground truth about the project — and now you can hand it that ground truth directly.

The box began with a single running conversation — fine until you have two things going at once and the agent's context for one bleeds into the other. Now the box keeps multiple chats: start a new one any time, switch between them, rename or delete them, each carrying its own thread and its own agent context.

Underneath, this leans on the same two-layer split from §06. The visible transcript of each chat is a durable record in DynamoDB (conv#<id> partitions alongside the tenant's facts, with a conv#current pointer to the active one), so reopening a chat repaints exactly what was said. The agent's memory of a chat is its Goose ACP session; switching chats calls session/load on that chat's session id, so the agent resumes the thread mid-thought instead of starting cold. New chats are auto-titled from their first exchange so the list stays scannable, and a migration on first load folds every existing single-conversation box into the model without losing history — the old conv#current simply becomes chat one.

Two quality-of-life layers sit on top: a search box over the chat list, and — since the same problem exists for files — a recursive filename search in the Files tab that walks the whole workspace, skipping the usual heavy directories and capped so a giant tree can't hang the UI. Both are the same idea: as the box accumulates state, you need a way to jump straight to the thing you mean.

Customer communication is part of the product, so it's built like the rest of it. A single file — CHANGELOG.md — is the source of truth, and it fans out to three destinations with no hand-copying.

As we ship, each user-visible change adds one plain-language bullet under ## Unreleased. From there:

• The public page. avriz.io/changelog renders the customer-facing sections of that same file (the internal "how the email works" and "engineering notes" sections are filtered out). It's kept fresh by adding CHANGELOG.md to the deploy-cp path filter, so editing the changelog is a deploy of the page.
• The weekly email. A scheduled GitHub Action fires Monday 9am (America/New_York), copies the changelog to the cp host over SSM, and runs a small script that builds the update from the Unreleased bullets and sends it through the same SES + branding path as every other transactional mail — so no provider keys leave the box, and no passwords are ever in the message.
• The archive. After a successful send the job rewrites CHANGELOG.md, moving the Unreleased bullets into a dated ## YYYY-MM-DD — emailed section, and commits it back to main. That commit is what stops the same bullets going out twice — and it refreshes the public page in the same motion.

A couple of edges were designed in rather than discovered. GitHub cron is UTC-only and 9am ET drifts with daylight saving, so the schedule fires at both the EDT and EST offsets and a guard step lets only the one that's actually 9am in New York proceed. And a Monday with nothing under Unreleased simply does nothing — no empty email, no failed run — because the alternative (a red workflow every quiet week, or worse, a blank customer email) is exactly the kind of automation that erodes trust in itself.