AI Provider Strategy — Models, Resilience, and Verified Costs
Status: decided — content ratified by AD #19 (Bedrock-only serving, 2026-08-18) and the OQ-15 resolution (PRD Rev 12, 2026-08-19; its processing-location half reopened 2026-09-30, Rev 48, and settled 2026-11-04, Rev 82 — §4b); stands as the AI model/serving/cost reference. Header updated 2026-08-31 — the line had sat at “draft for review” after the decisions that consumed it. Models revised 2026-09-30: Sonnet 5.5 replaces Sonnet 5, Haiku 5.5 will replace Haiku 4.5 once on Bedrock, and §4b now records the inference profile each model is called through (§1 update, §4b, §5 row 9). Models in use, 2026-10-26: AWS keeps Empire Teas’ account at new-account limits, so Haiku 4.5 runs every AI feature in every environment until AWS lifts them, and the backend paces its Bedrock calls (§5 row 10); the model per feature in §2 is the design the code defaults to Author: Tharinda (with Claude), 2026-08-18 Scope: answers three questions raised 18 Aug — (1) which model per AI feature, and is anything above Haiku needed; (2) how the AI layer stays up when a provider is down, following the provider-abstraction mechanism proven on a previous Altrium engagement; (3) an accurate, load-based cost breakdown at verified August-2026 pricing. Inputs:
.forge/client/project-proposal.md § A.3(the committed client-facing numbers) ·architecture.md § AI provenance, governance, resilience and cost· the reference implementation from a previous Altrium engagement (its LLM-provider fallback + model-tuning specs and the provider-abstraction architecture decision) · official provider pricing pages (sources at bottom).
1. Verified pricing, 18 August 2026
Checked against the official Anthropic pricing page and current Gemini pricing on 18 Aug 2026. Per million tokens, input / output:
| Model | Input | Output | Cache hit | Batch (50% off) | Notes |
|---|---|---|---|---|---|
| Claude Haiku 4.5 | $1 | $5 | $0.10 | $0.50 / $2.50 | Current small model; previous-generation tokenizer |
| Claude Sonnet 5 | $2 | $10 | $0.20 | $1 / $5 | The launch “introductory” price is now permanent — the scheduled 1 Sep rise to $3/$15 will not occur |
| Claude Opus 5 | $5 | $25 | — | — | Not used (proposal A.3.2 rules the top tier out) |
| Claude Fable 5 | $10 | $50 | — | — | Not relevant to this workload |
| Gemini 2.5 Flash | $0.30 | $2.50 | — | — | Deprecation scheduled 16 Oct 2026 — comparison-eval candidate only (not in production); use the then-current Flash tier, pinning the concrete model (3.x Flash / Flash-Lite successor) at Phase 2 eval prep |
| Gemini 2.5 Flash-Lite | $0.10 | $0.40 | — | — | Cheapest cross-vendor option — comparison-eval only, not in production |
Update 2026-09-30 — Sonnet 5.5. The lead replaced Sonnet 5 with Claude Sonnet 5.5 (on Bedrock from 28 Sep 2026). Anthropic’s pricing page, read 2026-09-30, lists Sonnet 5.5 at the same $2 / $10 (cache hit $0.20, batch $1 / $5), on the same newer tokenizer, so every Sonnet figure below stands. The same page states that on Bedrock, regional and geo endpoints cost 10% more than the Global endpoint for Claude Sonnet 4.5, Haiku 4.5, Opus 4.5 and every later model. So Haiku 4.5 through the us. geo profile is priced at list + 10%, and Sonnet 5.5 through the Global profile at list (§4b). Haiku 5.5, which replaces Haiku 4.5 once on Bedrock, is not priced yet.
Three pricing facts that matter to us:
- Sonnet 5 is cheaper on paper than the proposal assumed ($2/$10 vs the $3/$15 the proposal quoted) — but Claude models after 4.6 use a new tokenizer that produces ~30% more tokens for the same text. The two effects nearly cancel: effective Sonnet-5 cost per interaction lands ~10–15% below the proposal’s figures. Haiku 4.5 keeps the old tokenizer, so its figures are unchanged. Net: the proposal’s per-interaction cents remain accurate, slightly conservative. Nothing sold to the client needs correcting.
- Prompt caching pays immediately on our shapes — every AI feature has a fixed instruction block (system prompt + tea-domain context) reused across calls. Cache hits cost 10% of input. Architecture already commits to this (
§ AI … cost). - Bedrock batch inference (50% off) halves the one big batch we have — the 200-lead backlog triage is asynchronous by nature; at batch pricing the entire backlog costs well under a dollar.
2. Model per feature — and the honest answer on Sonnet
Thilina’s instinct (default to Haiku) is exactly what the proposal already commits: the cheaper model on every high-volume path by design (A.3.2). But the proposal also assigns Sonnet to six actions, and that split should survive. The workload divides cleanly:
| AI action | Volume/mo (assumed) | Model | Why | Cost per interaction* |
|---|---|---|---|---|
| Lead research + scoring (F11) | ~40 + backlog | Sonnet 5.5 (synthesis) | Revised by PRD Rev 30 (2026-09-16) — every lead is researched before it is ranked (PRD Rev 83: when research cannot run, the lead is ranked from the inquiry and ranked again once researched). Was Haiku classification against fixed criteria at ~0.2–0.4¢; the research pass makes it the same shape as the capture-app row below — synthesise public-source results into a grounded snapshot, then judge fit. Add one web search per lead (~0.7–0.8¢; Exa, provisionally — lead-research-source-options.md §11, §13). Background job, not a synchronous call (Rev 30 NFR). The research half is F11-c’s lead-research feature (spec approved 2026-10-21): Sonnet 5.5’s Global profile by default, overridden to Haiku 4.5 in every environment until AWS lifts the account’s new-account limits (§5 row 10; dev’s task definition sets it today, as for email-draft); a lasting move to Haiku 4.5 waits on its evaluation on Sonnet 5.5 and the lead’s sign-off (.forge/owed.md). The search is Exa on its free credit, one a lead. Scoring is F11-g’s |
~2.0¢ + search |
| Inquiry extraction (F12) | ~87 | Haiku 4.5 | Named fields into a form; human confirms; null-not-guess | ~0.3–0.6¢ |
| Business-card read (capture app) | show-driven | Haiku 4.5 | Mechanical, verifiable at a glance | ~0.5¢ |
| Spam/duplicate detection (inside F11) | with triage | Haiku 4.5 | Binary-ish classification. It runs beside the research pass, not before it: every lead is researched, a spam-tagged one included, because the tag is a suggestion a rep may dismiss (F11-c spec decision 14, 2026-10-21) | included |
| Email / sequence-step drafts (F20/F25) | ~880 | Sonnet 5.5 | Client-facing prose in Empire’s voice — the one place output quality is the product. The email-draft feature (F11-h) defaults to Sonnet 5.5’s Global profile; it is overridden to Haiku 4.5 in every environment until AWS lifts the account’s new-account limits (§5 row 10) (the account refuses it today, cloud-gated-validation.md §C-5 and §B F11-h item 7); the delivery team set the override in forge-et-be .aws/td.dev.json (forge-et-be#280, merged 2026-10-20), and the DevOps team is told of it (.forge/owed.md). The F11-h evaluation run on Haiku 4.5, Haiku 5.5 and Sonnet 5.5 (F11-h spec FR-B6, register §B F11-h item 2) is the evidence §5 row 1 asks for before drafts move off Sonnet |
~1.3–1.9¢ |
| Quote assistant (F14) | ~100 | Sonnet 5.5 | Reasons across readiness + certs + pricing; errors reach a buyer | ~1.5–2.0¢ |
| Account briefing (F13) | ~200 | Sonnet 5.5 | Synthesis across an account’s full history. Made only when a person asks, by pressing Prepare briefing on the account page, under account.manage (F13 spec decisions 2 and 3; AD #32 as its F13 edit reads): one extract call through the account-briefing/v1 prompt, none for an account with no timeline entry. The account-briefing feature defaults to Sonnet 5.5’s Global profile; it is overridden to Haiku 4.5 in every environment until AWS lifts the account’s new-account limits (§5 row 10) (the account refuses it today, cloud-gated-validation.md §C-5 and §B F13 item 4), so refused calls do not open the shared bedrock circuit breaker. The F13 evaluation run on Haiku 4.5 and Sonnet 5.5 (F13 spec FR-B7, register §B F13 item 2) is the evidence for keeping the briefing on Sonnet or moving it |
~1.8–2.3¢ |
| ~ |
— | — | — | |
| Morning dashboard briefing (F15) | 15 users × ~22 days | Sonnet 5.5 | Cross-pipeline synthesis; the first thing the team reads daily | ~2.0–2.6¢ |
| Trade-show research + rank (capture, full scope; F36-c) | show-driven: about 150 captured leads and up to about 500 exhibitors a show | Research: Sonnet 5.5 by design (proposal §A.3.2). Ranking: Haiku 4.5 by design, as the CRM’s lead ranking (F11-g decision 5). Both run on Haiku 4.5 now, in every environment — the research by task-definition override (§4d; F36-c FR-RS2) | The “why this lead matters” judgement: F11-c’s research of the company’s public sources, then a priority shown as Hot, Warm or Cold, a fit score, the reason and a next action. Exhibitors on a show’s plan from F36-c’s wave 1, captured leads from its wave 2 | ~1.5¢ a captured lead or exhibitor on Haiku 4.5, its web search included (one search, one research call, one ranking call — F36-c NFR-4); ~2.0¢ with the research on Sonnet 5.5. A 500-exhibitor show ≈ $7.50 (≈ $10 with the research on Sonnet 5.5) |
* at verified pricing, including the new-tokenizer effect on Sonnet 5 (Sonnet 5.5 has the same price and tokenizer — §1 update); ranges reflect context size variance. Proposal A.3.2’s published figures sit inside or above these ranges — safe.
Do we need anything above Haiku? Yes, for the six writing/synthesis actions — and it costs almost nothing to keep. (Written for Haiku 4.5. Since 2026-11-04, Haiku 5.5 is evaluated first for every feature, these six included, and Sonnet 5.5 is kept only where Haiku 5.5 fails the evaluation — §4d, “Which model to try first”.) The all-Haiku alternative saves roughly $15–18/month (Section 3) and puts the two features that sell the AI layer — buyer-facing drafts in Empire’s voice, and the morning briefing — on the weakest model. That is a bad trade at this price point. Nothing in the workload needs a top-tier model (Claude Opus, Gemini Pro); the proposal already says so and it holds at re-check.
But make the split a config, not a constant. Per-feature {model, enabled} routing under a single provider (Section 4) means the Phase 2 eval can demote drafts to Haiku — or promote extraction to Sonnet — as measured quality dictates, with no code change. That is the reference implementation’s model-tuning discipline: eval first, then per-component model IDs in config.
3. What it costs at expected load
At the proposal’s assumed volumes (15 users, 40 new leads, 87 inquiries, 880 drafts, daily briefings, ~200 account briefings, ~100 quote assists, ~30 product briefs — the last no longer applies since F16 was dropped on 2026-10-11, about $0.6 a month, inside the range below):
| Scenario | Monthly cost | Notes |
|---|---|---|
| Baseline (recommended split, no caching) | ~$26–31 | Matches proposal A.3.2’s “approximately $25”. +~$1/mo from PRD Rev 30 (F11 research): 40 leads × (~2¢ Sonnet synthesis − ~0.3¢ Haiku classification) ≈ $0.70, plus ~40 searches ≈ $0.30. Inside the range the row already carried; the $50 budget and $150 alert are untouched |
| Baseline with prompt caching | ~$18–24 | Instruction blocks cached; hits at 10% of input price |
| All-Haiku (rejected) | ~$8–12 | Saves ~$15/mo; degrades the two flagship features. That rejection concerned Haiku 4.5; since 2026-11-04 Haiku 5.5 is evaluated first for every feature (§4d) |
| Double volume (adoption success case) | ~$50–60 | Still inside the billing-alert threshold |
| One-off: 200-lead backlog triage | < $1 | Bedrock batch inference (50% off), Haiku — ~200 × 0.3¢ × 50% |
| Per trade show (150 leads, full-scope capture) | ~$6 | Three AI calls per lead, per A.3.2 |
| Bedrock (the adopted serving path) | List rates on the $25–30 baseline, with Haiku 4.5’s share about 10% higher for the calls a person waits for, which use the us. geo profile (the background features use its Global profile at list price since F43, §4b); neither pinned model runs in-Region on bedrock-runtime (§4b, as read 2026-09-30) |
The production path — Claude inside Empire’s AWS account, which is also the A.6.5 in-account answer by default (see §4b) |
| Gemini (rejected — comparison-eval candidate only) | ~⅓ of baseline | Third vendor / quality re-validation; comparison candidate only — production stays Bedrock-only |
Yearly: roughly $300–360 at expected volumes (≈$220–290 with prompt caching); budget $600 (the proposal’s $50/month budget line). Controls stay as committed and already architected: per-feature admin toggles, billing alert at $150, a configurable monthly ceiling that disables AI features rather than allowing an unbounded bill, and per-call logging of model/tokens/cost/latency/outcome.
Reconciling the numbers people have seen
Three different figures have circulated; only one is current:
| Figure | Where it came from | Status |
|---|---|---|
| “~$7/year” | June research (ai-in-workflows.md §7) — priced only the extraction/triage trickle, at Haiku rates, before drafts (880/mo) and daily briefings existed in scope |
Obsolete. Do not quote. Corrected in PRD Rev 8 (2026-08-18): the Observability section now carries the committed proposal numbers and explicitly marks the ~$7/yr figure obsolete |
| “~$25/month, budget $50” | Proposal A.3.2 — full workload, per-action pricing | Current and client-committed. Verified here against Aug-2026 pricing; holds (slightly conservative) |
| “costs are much higher (than the report’s yearly figure)” | Thilina, 18 Aug — referring to the proposal’s numbers | Correct: ~$300/yr expected vs the obsolete $7/yr. For the client’s total monthly run cost, quote .forge/client/sow-cost-annexe.md, not a figure copied here — the estate was re-derived on 2026-08-21 after AD #22/#23/AD-O-9 and the numbers this row used to carry are superseded. AI lands on the AWS invoice, so it is inside that total rather than beside it |
4. Resilience and outage posture — a proven Altrium mechanism, extended
What the reference implementation actually built (a previous Altrium engagement): a single LLMProvider interface (completion, streamCompletion, toolCall, ingestMultimodal); one implementation per vendor; a provider factory reading per-component config (LLM_PROVIDER per Lambda group); model IDs in env vars, never hardcoded; llm_cost_usd + token instrumentation on every call with per-session cost telemetry and soft/hard alerts; retry with exponential backoff; a CloudWatch alarm on cumulative LLM errors; and an eval harness gating any model/provider change. Notably, its Claude-fallback spec deliberately kept automatic failover out of scope — that fallback is a config flip.
For Empire Teas we adopt all of that and add the automatic layer, because our users are salespeople mid-workflow, not engineers who can flip an env var:
AIProvider (seam already mandated — AD #6)
├─ per-feature routing table (config, hot-reloadable):
│ feature → { model, enabled } (one top-level provider: bedrock)
├─ resilience pipeline per call:
│ pace: a slot per attempt, per inference profile, per backend task (F42)
│ → timeout (inside the <10s NFR) → bounded retry ×2, exponential backoff + jitter
│ (a Bedrock throttle retried the same way, never counted by the breaker)
│ → per-provider circuit breaker (open on N failures / 60s window;
│ half-open probe; CloudWatch alarm on open)
└─ on open circuit: **degrade to the manual path**
(single provider — Claude on Amazon Bedrock, in-account; no second-provider tier in V1)
Pacing under the account’s request quota (F42, .forge/specs/F42-bedrock-call-pacing-spec.md). Every Bedrock attempt — the first and each retry — takes a slot first from an in-memory sliding 60-second window per inference profile, one window per backend task, at the row et.ai.pacing.limits gives it: the account’s requests-per-minute quota divided by the backend tasks drawing on it, rounded down (10 for Haiku 4.5 in dev, which runs one task). Two slots a minute (background-reserve) are kept for calls a person waits on. Inquiry extraction, email drafts and the account briefing are interactive: each waits at most 3 s for its slot, inside its 9 s deadline, and otherwise answers busy (the next paragraph). Lead triage and lead research are background: each waits at most 60 s for its slot (triage’s wait was 15 s until F43 FR-D1), then ends RATE_LIMITED with no attempt, and the hourly sweep queues the lead again without counting an attempt. To keep a triage message hidden while its call waits, the triage listener asks SQS on every receive for a visibility of its own, et.lead-triage.visibility-timeout (150 s by default), and in a cloud shape startup refuses a triage wait that, added to the 9 s deadline, is not at least 60 s below it (F43 FR-D2, FR-D3). A throttle that still reaches Bedrock is retried like a transient error, ends RATE_LIMITED, never opens the circuit breaker, and counts one lead attempt. The live-evaluation runner keeps its own pace in its own process, so a live run is made while dev is quiet (F42 W-4). The keys and variable names are in the register’s §D and forge-et-be docs/ai/README.md § Configuration (the keys) and § Pacing (how it works); the proof on dev is the register’s §B F42 block. A research run that gets no slot repeats its web searches on retry; F42-WI-2.3’s plan expects about none at the defaults and at most 200 during the first Pipedrive import, inside the 900 backlog searches a month (F42 FR-O2a). The proof on dev found two gaps: in a burst of a few dozen leads about two in five waited for the next hourly sweep (up to an hour) and some for the one after, because each triage job then waited only 15 s and the hourly sweep sends every handed-back lead at once, and a person’s call refused for no slot read as an outage on screen. Both, moving background work to Haiku 4.5’s Global profile (its own quota of 10 a minute), and the design for every source of AI load (the Pipedrive runs, imports, the capture app on a trade-show day, the morning briefing, the mailbox scans, which make no AI call) are F43 (.forge/specs/F43-ai-quota-backlog-and-busy-answer-spec.md). The pacer counts requests, not tokens: no token pacer is built unless production’s quotas need one (the lead, 2026-11-05). A route whose tokens-a-minute quota is lower than its pacing row can use gets a lower pacing row instead, worked out from the largest call it carries; how is in .forge/owed.md, “Scan production’s Bedrock setup”, case 5.
A busy answer for a person (F43, .forge/specs/F43-ai-quota-backlog-and-busy-answer-spec.md FR-A1 – FR-A4). An interactive call whose first attempt gets no slot within its 3 s, or whose last attempt Bedrock throttled, still ends RATE_LIMITED, but inquiry extraction, the email draft and the account briefing do not answer it with their 503 unavailable code: they answer 429 with the code AI_BUSY, a Retry-After header and retryAfterSeconds in the body. The wait is the pacer’s time until a slot frees for a new interactive call on that model, in whole seconds from 1 to 60; after a Bedrock throttle, which gives no wait, it is a fixed 10 s; the stub provider gives 5 s (STUB-FAIL-RATE-LIMITED) and 10 s (STUB-FAIL-THROTTLED). The CRM’s four AI places — the inquiry-extraction dialog, the lead’s follow-up draft panel, the compose dialog’s AI draft button and the account briefing panel — read the wait from the body, because the API is cross-site and exposes no response header to the browser (AD #33), and show “The AI is busy right now. Try again in N seconds.” (“Try again in a moment.” when there is no wait), with the button enabled, what the person typed kept and the manual path still offered. There is no automatic retry and no countdown: the person presses the button again. The answer comes inside the 9 s deadline, so the frontends’ 12 s request timeouts are unchanged. Every other failure — an open circuit, a timeout, a provider error, a disabled feature — is still an outage: it answers the feature’s 503 and shows its unavailable message. Busy means the same request is likely to succeed in a few seconds; unavailable means use the manual path. The wire shape and the stub’s waits are .forge/plans/F43/F43-Wave-2-contract.md; the backend’s own description is forge-et-be docs/api/errors.md (AI_BUSY) and docs/ai/README.md.
Why Bedrock-only, with no fallback provider (final same-day revision — the day’s earlier drafts carried a Bedrock middle tier, then a Gemini fallback; the team settled on the simplest shape):
- One vendor relationship, one bill. Serving Claude through Amazon Bedrock puts AI usage inside the project’s AWS account, on the same AWS bill as the rest of the estate — the client opens and maintains no separate AI provider accounts. For a 15-user internal system, that operational simplicity outweighs a rarely-fired fallback tier.
- In-account processing is the strongest data-governance answer. Buyer content is processed inside the project’s AWS account (Empire Teas’ own account since 2026-09-29, with Altrium working in it as Empire’s processor — SOW ET-002, which would have had Altrium own it, is superseded;
us-east-1— AD-O-9 revised on cost) — theA.6.5in-account posture by default, not as an add-on. The accepted proposal (Appendix A.6) states the arrangements — Anthropic, served via Bedrock, as the production processor; the once-planned separate data-handling document was dropped 2026-08-19 (OQ-15 resolved, PRD Rev 12). - Degrade-to-manual was always the real floor. “An AI outage must never block a rep from working manually” is the architecture’s standing contract — every AI surface has a non-AI path, and drafts queue rather than block. A cross-vendor fallback only ever narrowed the window in which that floor is used; it never removed the need for it.
- The accepted trade-offs, stated: Bedrock can lag brand-new Claude model releases (bites only at upgrade time, since models are pinned) and default quotas need a raise request — made at account creation, not at launch. (26 Oct 2026: on a new account AWS refuses the raise until the account has a few billing cycles of usage and spend — §4b, Quotas row; §5 row 10.) The A.3.1 provider comparison still runs, on anonymised inquiries via Altrium-held evaluation keys; production traffic never leaves Bedrock.
- A second tier stays cheap to add. The
AIProviderseam and per-feature config mean a direct-API or second-vendor tier is a config-level addition later — deliberately not built now. - Provenance is logged on every call — provider/model/tokens/cost/latency/outcome (A.6.2), so any output is attributable, including comparison-eval runs.
Cost instrumentation copies the reference implementation’s cost-telemetry requirement: every call records {provider, model, prompt_tokens, completion_tokens, cost_usd}; a monthly-spend metric drives the $150 alert (CloudWatch alarm → SNS → team email) and the hard ceiling. (2026-10-23, F38 Rev 23: the $150 alert is an AWS Budgets alert belonging to the billing owners since F38 Rev 18, not a CloudWatch alarm the delivery team builds; the hard ceiling is still F38 W-10.)
Eval before lock copies its model-tuning process: build the small eval set from real (anonymised) Empire inquiries as soon as they are available — the Pipedrive exports Empire shared carry no email text, and no Pipedrive access is given (client/dependencies.md §1.3), so the source is still to be agreed; score extraction accuracy, draft quality, and cost per call across {Haiku 4.5 (or Haiku 5.5 once on Bedrock), Sonnet 5.5, Gemini Flash tier}; record the per-feature decision. This is the “side-by-side comparison on real Empire Teas inquiries during Phase 1b” the proposal commits to in A.3.1 — we are contractually running it anyway, so the comparison candidate gets evaluated for free at the same time.
4b. Bedrock or Anthropic direct? — resolved: Bedrock only
Same models, same list prices — the choice was operational, not financial. Comparison kept as the decision record:
| Anthropic API (direct) | Claude on Amazon Bedrock | |
|---|---|---|
| Token price (Haiku 4.5 / Sonnet 5.5) | $1/$5 · $2/$10 | List rates through the Global profile; list + 10% through a regional or geo profile (Anthropic’s pricing page, read 2026-09-30) — so Haiku 4.5 on the us. profile is list + 10%, Sonnet 5.5 on Global is list |
| Billing | Separate Anthropic account (card/invoice) | On the AWS bill — one vendor, one invoice for the client |
| Auth/ops | API key in Secrets Manager | IAM role — no key to rotate |
| Data residency | Global routing by default | Called from inside the project’s AWS account; where the request is processed depends on the inference profile (table below) — neither pinned model is offered in-Region on bedrock-runtime today |
| Feature currency | First-party: Batch API, 1-hour caching, newest models day-one | Days-to-weeks behind on newest features and models; a new model often arrives on the Global profile first and on geo profiles later (Sonnet 5.5, 2026-09-28) |
| Quotas | Standard tiers | Default Bedrock quotas often need a raise request — do it at account creation, not at launch. Found in the dev account, 15–21 Oct 2026: a new account starts far below the defaults (0 tokens a minute for Sonnet 5.5 and Haiku 5.5, 10 requests a minute for Haiku 4.5); Service Quotas accepts a request only at the AWS default, and AWS refused every such request (“unable to approve your request at this time”), so the route is an AWS Support “Service limit increase” case with the use case and the values actually needed (environments-and-identity.md §4 item 4). Request the quotas of the profiles the backend calls — for Haiku 4.5 both, the us. (cross-Region) one for the calls a person waits for and the Global one for the background features (since F43); the Global one for Sonnet 5.5 (the table below) — each model has separate cross-Region and Global quotas. AWS’s answer, 26 Oct 2026: no increase until the account has a few billing cycles of Bedrock usage and spend; AWS lifts the new-account limits on its own review, with no date (environments-and-identity.md §4 item 4, the 26 Oct update; what the team does meanwhile is §5 row 10). So a new account, production’s included, should expect to run for months at these limits. Read 19 Nov 2026: two Service Quotas requests for Haiku 4.5 at 10,000 requests a minute (17 Nov) were closed with no change, and every quota is still at its 26 Oct value (environments-and-identity.md §4 item 4, the 19 Nov update) |
Decision: Bedrock only — no direct API calls in production. The billing column decided it: through Bedrock, AI lands on the AWS invoice and Empire Teas maintains no second vendor relationship — for this client, one bill beats every operational advantage in the direct column. In-account processing also makes the A.6.5 residency posture the default rather than an add-on — within the limits of the inference profile each model is offered on (the table above, as of 2026-09-30). The costs of the choice are managed, not ignored: the newest-model lag matters only at upgrade time (models are pinned); quota raises are requested at account creation; batch and prompt-caching needs use Bedrock’s own equivalents (batch inference; prompt caching) — the 200-lead backlog still triages for under a dollar. Direct API calls remain what the A.3.1 comparison uses (Altrium-held evaluation keys, anonymised inquiries) and what the seam could re-enable later as config, not rework.
How each pinned model is called on Bedrock — AWS’s model cards, read 2026-09-30, for the bedrock-runtime endpoint; Haiku 4.5’s Global route added 2026-11-02 (F43). The same holds from us-east-1 and us-west-2.
| Model | Profile to call | Where a request may be processed | In-Region option |
|---|---|---|---|
| Claude Sonnet 5.5 | global.anthropic.claude-sonnet-5-5; the account also lists a us. profile, us.anthropic.claude-sonnet-5-5 (environments-and-identity.md §4, list-inference-profiles), priced since F43 and granted on dev since forge-et-iaac#16 was applied on 2026-11-04, and used as §4d’s “Which route” says |
Global: any commercial AWS Region worldwide; us.: the US Regions its profile lists |
None on bedrock-runtime; bedrock-mantle does not serve it in any commercial Region |
| Claude Haiku 4.5 | us.anthropic.claude-haiku-4-5-20251001-v1:0 for the calls a person waits for; global.anthropic.claude-haiku-4-5-20251001-v1:0 for the background features (lead triage, lead research and, from F11-j, the summary of a migrated lead’s linked email) where the task definition sets it (F43 FR-R3; register §D) |
us.: us-east-1, us-east-2, us-west-2; Global: any commercial AWS Region worldwide |
Only on the bedrock-mantle endpoint (Messages API) |
| Claude Haiku 5.5 | On Bedrock by 2026-10-13: us.anthropic.claude-haiku-5-5 (called in dev that day and refused; both its quotas are 0 tokens a minute, register §C-5) and a Global profile (Service Quotas lists both a cross-Region and a Global quota for it). Replaces Haiku 4.5; the backend will call it through the Global profile global.anthropic.claude-haiku-5-5 (chosen 2026-10-27: list price, the same route as Sonnet 5.5; .forge/owed.md), and through both profiles while its quotas are tight (§4d, “Which route”, 2026-11-04) |
The us. and Global routes as for the models above — to confirm on its model card when F12 chooses |
To confirm on its model card |
The lead’s direction (2026-09-30): use Sonnet 5.5 now, through the Global profile, and move to a us. profile only if Empire asks for US-only processing (OQ-15); moving is a configuration change (§4c rule 1). Two consequences are recorded:
-
The proposal and the PRD. Proposal §A.3.1 and §A.6.5 promise the models are served “inside Empire Teas’ own AWS account and region”, and the PRD’s OQ-15 resolution (§ Constraints, AI data handling) repeats “inside the project’s AWS account and region”. Every route above is called from inside the account, but neither
bedrock-runtimeroute stays in one Region, and the Global profile does not stay in the US. The proposal divergence is stated in a post-issue note (L-001). The lead reopened OQ-15 for this on 2026-09-30 (PRD Rev 48; client question Q18): whether Empire accepts processing outside the US for buyer data, which includes EU persons, is Empire’s decision. The default proceeded on was that it is acceptable — Global while it is Sonnet 5.5’s only route, and a US-only route only if Empire asks — because a US-only route costs 10% more and new models reach Global first. Settled 2026-11-04 (the lead; PRD Rev 82): processing in any commercial AWS Region is accepted, and OQ-15 is closed. Empire never answered Q18, which is answered with this decision and closed. The AWS Bedrock documentation, read that day, gives the grounds, and they hold on every route:- every call is made from Empire’s own account;
- data between Regions stays on the AWS network, encrypted in transit, with CloudTrail logging every call in the source Region;
- model providers have no access to prompts or answers;
- neither AWS nor the providers use them for training.
A US-only route stays a configuration change if Empire ever asks.
-
A policy refusing other Regions (F38-a Q-2) must allow every destination Region of each profile in use: the
us.profile alone needsus-east-1,us-east-2andus-west-2, and the Global profile routes to Regions worldwide.
The lead’s direction (2026-11-02, F43): Haiku 4.5’s background features use its Global profile. Lead triage, lead research and lead ranking (F11-g, ET_AI_FEATURES_LEADRANKING_MODEL on dev — register §D) call global.anthropic.claude-haiku-4-5-20251001-v1:0, and so does the summary of a migrated lead’s linked email (F11-j, ET_AI_FEATURES_LEADMAILSUMMARY_MODEL), which sends the buyer’s email bodies (F11-j NFR-3); the calls a person waits for (inquiry extraction, the email draft, the account briefing) stay on the us. profile. AWS gives each profile its own request quota (10 a minute each on the development account), and the backend paces each profile separately (§4), so the background features no longer take slots from people’s calls, and they are billed at list price instead of list + 10%. Lead triage sends buyer text (the inquiry, the contact’s email address, the company) on a route that may process it in any commercial AWS Region, which is accepted (OQ-15, settled 2026-11-04, above); moving back to us. is a configuration change (ET_AI_FEATURES_LEADTRIAGE_MODEL, ET_AI_FEATURES_LEADRESEARCH_MODEL; register §D). An account whose Global Haiku 4.5 quota is 0 keeps both features on us.. The proposal records it in a post-issue note of 2026-11-04, an addendum to the 30 September note (L-001). F43’s spec is .forge/specs/F43-ai-quota-backlog-and-busy-answer-spec.md (point 4, FR-R1 – FR-R3).
Models are runtime configuration, not code — one top-level provider: bedrock plus per-feature {model, enabled} keys with env-var overrides (concrete shape in coding-standards.md §5), exactly the reference implementation’s mechanism (an env-selected model ID per component; a provider interface returning {text, prompt_tokens, completion_tokens, cost_usd} on every call) extended per-feature. Not over-engineering: it is one properties block, and the Phase 2 eval + the A.3.2 admin toggles both require it.
4c. Model lifecycle — new versions, retired versions, and prompts that survive a model change
Added 2026-09-29 at the lead’s request. It applies to F12, which builds the AIProvider seam, and to every AI feature built on it; each AI feature’s spec cites this section.
The problem. The pinned models today are Haiku 4.5 and Sonnet 5.5 (Sonnet 5.5 reached Bedrock on 2026-09-28 and replaced Sonnet 5 on 2026-09-30). Haiku 5.5 is expected, but nobody knows in advance when it reaches Bedrock, or on which inference profiles. Haiku 4.5’s model card gives its end of life as no sooner than 16 Oct 2026, followed by a legacy period of at least six months. In the other direction, Bedrock moves older model versions to a legacy state and then ends them on dates AWS publishes; after a model’s end-of-life date, calls to it fail. Either change can happen during V1 or soon after it, and a prompt written and tested against one model is not guaranteed to behave the same way on the next.
What every AI feature is built to, so that a model change is configuration and a test run, not a code change:
-
The model is configuration only. Each feature’s
{model, enabled}key (§4b; the shape is incoding-standards.md §5) holds the full Bedrock identifier, including an inference-profile identifier if the model is only served through one (register “Start here” step 9 checks this on the day). No model identifier appears in feature code; tests use the local stub. Changing a feature’s model means changing that key and restarting the service, with no build. -
One request shape for every model. The
AIProviderimplementation hides model-specific request details, so a feature never depends on a parameter that only one model accepts. F12 decides how, and should consider Bedrock’s model-independent Converse API for exactly this reason. -
A retired or unavailable model fails safe and loudly. A “model not found” or “access denied” answer from Bedrock is treated like an outage: the feature falls back to its manual path (§4) and an alarm fires naming the feature and the model identifier, so the fix — pointing the key at a supported model — is obvious. (2026-10-23, F38 Rev 23: dev carries no alarms, so in dev this failure shows in the backend’s log lines only; production must build this alarm (F38-i;
.forge/owed.md“Decide the UAT and production alarm sets”).) -
Prompts are files with versions, not strings in code. Each feature keeps its system prompt in its own file with an identifier and a version, and every call records the prompt version next to the model and cost it already records (§4, cost instrumentation). This way a change in output can be traced to a model change or to a prompt change.
-
Prompts are written to carry across models. They state the task, the rules and the output format directly; they ask for structured output against a stated schema (JSON or tool use) and never rely on free-text parsing; they give short worked examples. They avoid wording tuned to one model’s habits, and they avoid parameters that newer models handle differently.
-
Each AI feature has a small evaluation set. It is a fixed list of inputs, synthetic or anonymised and never raw client data in a repository, each with the properties a correct answer must have, and it can be run on dev against any configured model. Switching a feature to a new model means: run its evaluation set on the new model, compare the results with the current model’s, and change the key only if the new model does at least as well. The proposal’s A.3.1 comparison is the first use of this.
-
Someone watches the model schedule. Once a month, as part of the monthly reporting, the lifecycle state of each pinned model on Bedrock is checked. AWS sends legacy and end-of-life notices to the AWS account’s contacts, and the account is now Empire Teas’ own, so the delivery team’s address must be on the account as an alternate contact (F38-a). A model that is moving to end-of-life gets an upgrade task with the evaluation run above well before the date. Retirement and the inference profile (read 2026-09-30). A model’s lifecycle is set per model, not per route: each Bedrock model card gives one end-of-life date covering every profile the model is offered on, so choosing Global or
us.does not change when a model retires. AWS’s pages do not say outright that every profile retires on the same day, so the monthly check in rule 7 reads each profile in use. The profiles differ at the start of a model’s life: a new model can reach the Global profile first (Sonnet 5.5 was Global-only at launch), so Global lets a feature move to a new model sooner. -
Prices move with models. A new model can cost more or less than the one it replaces; §1’s pricing table and
sow-cost-annexe.mdare updated in the same change that moves a feature to it.
4d. Running on Haiku 4.5 now, and switching when AWS allows the 5.5 models
Added 2026-10-26 at the lead’s request, after AWS kept the account at new-account limits (§5 row 10).
The approach. Every AI feature runs on Haiku 4.5 until AWS allows Sonnet 5.5 and Haiku 5.5 on Empire Teas’ account. No Phase 1 feature waits on the 5.5 models: each is built, tested and accepted on Haiku 4.5. When the models become callable, each feature is tested on them and moved only if the results justify it, one feature at a time, by configuration (§4c rule 1). Haiku 5.5 alone, with no Sonnet, is a candidate result, and model-choice-evaluation-plan.md is how that comparison is run.
Which model to try first (the lead, 2026-11-04). When the 5.5 models answer, Haiku 5.5 is evaluated first for every feature. That includes the writing and synthesis features §2 puts on Sonnet 5.5: the email draft, the account briefing, lead research and lead ranking.
- A feature moves to Haiku 5.5 when its evaluation shows no material loss of accuracy or quality against the alternative, by
model-choice-evaluation-plan.md§4.3’s rule. - Sonnet 5.5 is used only for a feature whose Haiku 5.5 evaluation fails that rule.
- The reason is cost. Per million input and output tokens, through a Global profile: Haiku 5.5 is $0.10 / $0.50, Haiku 4.5 $1.00 / $5.00, and Sonnet 5.5 $2.00 / $10.00 (
forge-et-beapplication.yml, F43). - The lead signs off each feature’s result (
model-choice-evaluation-plan.mdD-4). A writing or synthesis feature that stays on Haiku refines AD #19 and gets a proposal post-issue note (step 3 of the switch, below).
Which route (the lead, 2026-11-04). Each inference profile of a model has its own request and token quotas (§4b).
- While a model’s quotas are below what the CRM needs at peak, use both its profiles: the calls a person waits for on
us., and background work onglobal.. F43 does this for Haiku 4.5, and the same split applies to Haiku 5.5 and Sonnet 5.5 when a feature moves to them. - Once one profile’s quota covers peak demand with a margin, move every feature on that model to its
global.profile. It is billed at list price, about 10% belowus., and new models reach it first. Aus.profile is worth its extra 10% only for data residency, and processing in any commercial Region is accepted (OQ-15, §4b). - The
us.grants, price rows and pacing rows stay in place, so moving back is configuration. The weekly quota read (below) is when the team checks whether a model has reached that point.
The model per feature: the design, what runs now, and what moves it. This table is the one place that says, for each AI feature, which model the design intends, which model it runs on today, and what has to happen before it moves. Keep it current: a feature built while the limits apply ships on Haiku 4.5 and is added here in its delivering PR, and a switch updates its row in the same change. The model key is et.ai.features.<key>.model in forge-et-be application.yml; the task-definition override is ET_AI_FEATURES_<KEY>_MODEL (the key without its dash).
| Feature (key) | Design model (§2, AD #19) | Runs on now, every environment | Before it can move | Evidence that decides the move |
|---|---|---|---|---|
Inquiry extraction, F12 (inquiry-extract) |
Haiku 4.5; Haiku 5.5 once on Bedrock | Haiku 4.5 (the code’s default) | Haiku 5.5 readiness and the task role’s grant (below) | Its evaluation set on Haiku 5.5 against Haiku 4.5 |
Lead summary and spam / duplicate check, F11-d (lead-triage) |
Haiku 4.5; Haiku 5.5 once on Bedrock | Haiku 4.5 — its us. profile by the code’s default, its Global profile where the task definition sets it (dev since F43; UAT and production from register §D) |
As above | Its evaluation set; the instruction-shaped email domain case (.forge/owed.md) retested |
Summary of a migrated lead from its linked email, F11-j (lead-mail-summary) |
Haiku 4.5 | Haiku 4.5 — its us. profile by the code’s default, its Global profile where the task definition sets it (dev from F11-j’s backend PR; UAT and production from register §D) |
As above | Its FR-A6 evaluation set (register §B F11-j item 3) |
Lead research, F11-c (lead-research) |
Sonnet 5.5 | Haiku 4.5’s Global profile (task-definition override; register §B F11-c item 6, §D; the Global profile since F43) | BUG-248’s live proof (the fix is in forge-et-be#316; register §B F12 row 10) | Its FR-A8 evaluation on Sonnet 5.5 (register §B F11-c item 7); .forge/owed.md “Decide whether lead research moves to Haiku 4.5 in production” |
Email follow-up draft, F11-h (email-draft) |
Sonnet 5.5 | Haiku 4.5 (override; register §B F11-h item 7) | BUG-248’s live proof (the fix is in forge-et-be#316; register §B F12 row 10) | Its FR-B6 evaluation on Haiku 5.5 and Sonnet 5.5 (register §B F11-h items 2 and 5); .forge/owed.md “Decide which model F11-h’s drafts use” |
Account briefing, F13 (account-briefing) |
Sonnet 5.5 | Haiku 4.5 (override; register §B F13 item 4) | BUG-248’s live proof (the fix is in forge-et-be#316; register §B F12 row 10) | Its FR-B7 evaluation on Sonnet 5.5 (register §B F13 items 1 and 2) |
Lead ranking, F11-g (lead-ranking; Phase 1; prompt lead-ranking/v2 from F36-c’s wave 3) |
Haiku 4.5 (F11-g spec decision 5, FR-A1) | Haiku 4.5 — its us. profile by the code’s default, its Global profile where the task definition sets it (register §B F11-g item 3, §D ET_AI_FEATURES_LEADRANKING_MODEL) |
Haiku 5.5 readiness and the task role’s grant (below) | F11-g FR-A8’s set (register §B F11-g item 5). From F36-c’s wave 3 the feature calls lead-ranking/v2, with its lead-ranking-v2.json set and v2 bar (F11-g FR-A8 Rev 6); its live run on Haiku 4.5 on dev is a precondition of the first UAT or production release that carries that wave (register §B F36-c item (k)) |
Card reading, F36-b (card-read; spec approved 2026-11-13; Phase 1) |
Haiku 4.5 (§2’s business-card row; F36-b spec FR-A7, decision D-3) | Haiku 4.5’s us. profile, the code’s default, as a background call that takes that profile’s background slots (F36-b decision D-3). The feature and its configuration are on main from F36-b’s wave 1; cards are read from its wave 2, on dev once the card-read queue exists (register §B F36-b item (a)) |
Haiku 5.5 readiness and the task role’s grant (below); a move to Haiku 4.5’s Global profile is a configuration change | Its FR-H1 evaluation set (register §B F36-b items (b) to (d)) |
Plan research, F36-c (plan-research; exhibitors on a show’s plan, from F36-c’s wave 1) |
Sonnet 5.5 (§2’s trade-show row; F36-c FR-RS2, decision D-3) | Haiku 4.5’s us. profile (task-definition override in every environment, beside card reading’s background share; register §B F36-c item ©, §D) |
BUG-248’s live proof (the fix is in forge-et-be#316; register §B F12 row 10) | F11-c FR-A8’s evaluation set — the research prompt is F11-c’s — on Haiku 5.5 and Sonnet 5.5 (register §B F36-c item (g)) |
Plan ranking, F36-c (plan-ranking; from F36-c’s wave 1) |
Haiku 4.5 (F36-c FR-RS2; as F11-g’s lead ranking) | Haiku 4.5’s us. profile, the code’s default |
Haiku 5.5 readiness and the task role’s grant (below) | F36-c FR-RK6’s capture-ranking set on Haiku 5.5, and on Sonnet 5.5 for the comparison (register §B F36-c item (g)). From F36-c’s wave 3 the feature calls capture-ranking/v2, with its capture-ranking-v2.json set and v2 bar (FR-RK6 Rev 6); its live run on Haiku 4.5 on dev is a precondition of the first UAT or production release that carries that wave (register §B F36-c item (k)) |
Capture research, F36-c (capture-research; captured leads — shipped with F36-c’s wave 2) |
Sonnet 5.5 (as plan-research) |
Haiku 4.5’s Global profile (task-definition override in every environment, beside lead triage, research and ranking; the us. profile where an account has no Global Haiku 4.5 quota; register §B F36-c item ©, §D) |
BUG-248’s live proof (as above) | F11-c FR-A8’s set on Haiku 5.5 and Sonnet 5.5 (register §B F36-c item (g)) |
Capture ranking, F36-c (capture-ranking; shipped with F36-c’s wave 2) |
Haiku 4.5 (as plan-ranking) |
Haiku 4.5’s Global profile on dev (task-definition override); UAT and production from register §D; the us. profile, the code’s default, elsewhere |
Haiku 5.5 readiness and the task role’s grant (below) | F36-c FR-RK6’s set (register §B F36-c item (g)). From F36-c’s wave 3 the feature calls capture-ranking/v2, with its capture-ranking-v2.json set and v2 bar (FR-RK6 Rev 6); its live run on Haiku 4.5 on dev is a precondition of the first UAT or production release that carries that wave (register §B F36-c item (k)) |
| Not built, Phase 2: morning briefing (F15-a), quote assistant (F14), feedback-chase draft (F6-c), sequence steps (F25-a, F25-b) | Sonnet 5.5 | — | Each spec names Haiku 4.5 while the limits apply, and runs the comparison of model-choice-evaluation-plan.md §5 |
Its own evaluation set |
What else waits on the 5.5 models, so nothing is forgotten when they arrive: the register rows above; the register’s §B F38-f row (Sonnet 5.5 from the task role, parent F38 AC-29) and §B F38-g’s model row (the retest of features on their configured model); §C-5’s row “Sonnet 5.5 and Haiku 5.5 answer one call each in dev” (F38-a’s former AC-4 calls, moved by F38-a Rev 4); BUG-248’s live proof; and model-choice-evaluation-plan.md (the Haiku-only comparison). None of them holds a Phase 1 feature open.
What must be in the code before a switch is only a configuration change (each tracked in .forge/owed.md, “The AI layer under Bedrock’s new-account limits”):
- BUG-248 — fixed in forge-et-be#316 (26 Oct): the backend forced a named tool, which Sonnet 5.5 rejects, so every Sonnet 5.5 call would have fallen back to the manual path. Every request now uses tool choice
auto. One live call proves it once Sonnet 5.5 answers (register §B F12 row 10). - Haiku 5.5’s price row and its differences (
.forge/owed.md, “Haiku 5.5 readiness in the backend”;model-choice-evaluation-plan.mdP-3, P-4): the backend refuses at startup a feature model with noet.ai.pricesrow or noet.ai.pacing.limitsrow (F42) — since F43application.ymlcarries both rows for both Haiku 5.5 profiles (F43-WI-4.1) — and Haiku 5.5 can answer with a refusal. - The backend task role’s Bedrock grant names both inference profiles,
us.andglobal., of Haiku 4.5, Sonnet 5.5 and Haiku 5.5 in dev, and each model’s foundation model in every Region (arn:aws:bedrock:*::foundation-model/<model id>); Haiku 5.5’s Global profile came with forge-et-iaac#13 (applied 2026-10-28) and the other three missing routes with forge-et-iaac#16 (F43), applied 2026-11-04. UAT’s overlay carries the same six on forge-et-iaac#11; production’s come with F38-i’s overlay (.forge/owed.md, “Allow the backend’s task role to call Haiku 5.5”).
How the team learns the limits have lifted. AWS reviews the account automatically and does not promise to say when it lifts the limits, so the team checks: the quotas are read once a week with one read-only command, aws service-quotas list-service-quotas --service-code bedrock as svc-harness (the quota codes are in environments-and-identity.md §4 AC-4). A rise is recorded in §4 item 4 and AC-4, and the DevOps team then requests the quotas the backend uses at the values of the lead’s 22 Oct reply.
The switch, per feature, once a model answers on dev:
- Request the quota of the profile the backend will call (§4b), and add the task role’s grant if missing.
- Run the feature’s evaluation set on dev against the new model and the current one (
./gradlew liveEvaluation, §4c rule 6;model-choice-evaluation-plan.md§4 for the fuller comparison). Every run is Bedrock calls, so the lead approves it first. - The lead decides on the results. A move that changes §2’s split (for example a synthesis feature staying on a Haiku model) refines AD #19 and is stated in a proposal post-issue note (L-001), as
model-choice-evaluation-plan.md§7 sets out. - Change the feature’s model key in each environment’s task definition (dev first, then UAT and production), and update §2, §3 and
sow-cost-annexe.mdin the same change (§4c rule 8). - Watch the feature for a month after the switch (
model-choice-evaluation-plan.md§8); moving back is the same one setting.
Testing before the limits lift: not done. AD #19 allows the A.3.1 comparison on Anthropic’s own API with Altrium-held evaluation keys (model-choice-evaluation-plan.md P-6, decision D-2), but the lead declined it on 26 Oct: no API account billed per token, for cost. The team’s Claude plan seats are for people using Claude and give the backend no API key, so the backend’s evaluation runner cannot use them. The comparison therefore runs on Bedrock once the models answer there; the work that needs no AWS (the code changes above, the runner’s repeated runs, the larger evaluation sets) can be done before.
5. Decisions requested / settle-by
| # | Decision | Recommendation | Settle by |
|---|---|---|---|
| 1 | Per-feature model split | Haiku 4.5 high-volume / Sonnet 5.5 writing+synthesis, as §2 (Sonnet 5.5 replaced Sonnet 5 on 2026-09-30; Haiku 5.5 replaces Haiku 4.5 once on Bedrock); revisit only on Phase 2 eval evidence. Revised 2026-11-04 (the lead): once the 5.5 models answer, Haiku 5.5 is evaluated first for every feature, the writing and synthesis ones included, and Sonnet 5.5 stays only where Haiku 5.5 fails the evaluation (§4d, “Which model to try first”). Until then §5 row 10 applies | Now (this doc) |
| 2 | Provider + outage posture | Decided 2026-08-18 (final same-day revision): Claude via Amazon Bedrock only, in-account (us-east-1) — one AWS bill, no separate AI accounts; no fallback provider — the circuit breaker degrades to the manual path (see §4 and §4b) |
Build in Phase 2 |
| 3 | Disclose Google’s comparison-eval role | Decided 2026-08-18: yes; route revised 2026-08-19 — no separate data-handling document exists (OQ-15 resolved); the disclosure stands via the accepted proposal’s A.3.1 description — anonymised inquiries via Altrium-held evaluation keys, never production traffic | Closed |
| 4 | Gemini concrete model (comparison eval only — not in production) | Pin when the A.3.1 eval is prepared (2.5 Flash deprecates 16 Oct 2026 — use the then-current Flash-tier model) | Phase 2 eval prep |
| 5 | PRD “$7/yr” line | Done — corrected by PRD Rev 8 (2026-08-18); Observability now cites proposal A.3.2 | Closed |
| 6 | Streaming for drafts | Decided by the F12 spec (context decision 9, 2026-10-07): not in F12’s interface. AIProvider has one operation, a structured extraction (F12 spec FR-B1); the first feature that wants streamed drafts (F11-h or F13) adds a streaming operation in FR-B1’s shape — a provider-neutral request, a typed result or failure, every call recorded — rather than F12 carrying an operation nobody calls. F11-h added draft with no streaming (F11-h spec decision 3, FR-B2): a follow-up email is a few hundred tokens, is checked for prices and shown whole, and answers through one structured tool, so AIProvider has two operations, extract and draft, sharing one call path. F13 needed no streaming either (F13 spec decisions 5 and 9): the account brief is short, capped by its output schema and useful only once whole, so it is one synchronous extract call inside the shared 9 s deadline; if Sonnet 5.5 cannot answer it inside the deadline on dev, streaming or a background job is reconsidered by an F13 spec revision (register §B F13 item 1). A later feature that needs streaming adds it |
Closed (F12; F11-h and F13 kept it closed) |
| 7 | Real model calls before the AWS accounts exist | Decided 2026-09-23 (Tharinda): none. No direct Anthropic API for local development either (this does not affect the A.3.1 comparison, which AD #19 allows on Altrium-held evaluation keys); every AI feature is built and tested against the AIProvider local/test stub until Bedrock exists. Real prompt quality, latency and cost are proved on dev as part of F38 (.forge/cloud-gated-validation.md §C-5 and the “Start here” checklist) |
Closed |
| 8 | How the AI layer copes with new and retired models | Rules recorded 2026-09-29 (lead request) — §4c: model identifier in configuration only, one request shape for every model, a retired model fails to the manual path with an alarm, versioned prompt files, prompts written to carry across models, an evaluation set per feature, a monthly check of Bedrock’s model lifecycle, prices updated with a model change. Mechanism decided by the F12 spec (context decision 5, FR-B2): every model is called through Bedrock’s Converse API with one tool whose input schema is the prompt’s output schema and toolChoice auto, the prompt saying to answer only through that tool (a toolChoice naming the tool until BUG-248, forge-et-be#316, because Sonnet 5.5 refuses a forced tool), so the answer is structured data checked against the schema, never parsed free text; the model id and the price row are configuration (et.ai), prompts are versioned resource files, and an unknown model id fails as MODEL_UNAVAILABLE to the manual path (forge-et-be/docs/ai/README.md) |
Closed (F12) |
| 9 | Which inference profile each model is called through, and worldwide processing | Lead’s direction 2026-09-30: Sonnet 5.5 through the Global profile; Haiku 4.5 through the us. profile (§4b). OQ-15 reopened (PRD Rev 48, client Q18): whether buyer data may be processed outside the US is Empire’s decision; the default is that it may, with a US-only route only if Empire asks. Stated in the proposal post-issue note of 2026-09-30. Lead’s direction 2026-11-02 (F43): Haiku 4.5’s background features — lead triage, lead research and later lead ranking — through its Global profile, so they have a request quota of their own and are billed at list; the calls a person waits for stay on us. (§4b). This sends buyer text on a route that may process it outside the US; moving back is configuration (register §D). Settled 2026-11-04 (the lead, PRD Rev 82): processing in any commercial AWS Region is accepted, so OQ-15 is closed and Q18 is answered and closed. The grounds, which hold on every route, are in §4b. Stated in the proposal post-issue note of 2026-11-04, an addendum to the 30 September note. The route rule from here on (the lead, 2026-11-04): both profiles of a model while its quotas are tight; every feature on its global. profile once one profile’s quota covers peak demand (§4d, “Which route”) |
Closed (2026-11-04) |
| 10 | Which model runs while the account has Bedrock’s new-account limits | Decided 2026-10-26 (the lead): AWS keeps the account at 0 tokens a minute for Sonnet 5.5 and Haiku 5.5 and 10 requests a minute for Haiku 4.5 until it has a few billing cycles of usage and spend (§4b, Quotas row). Until AWS lifts the limits, Haiku 4.5 runs every AI feature where Sonnet 5.5 or Haiku 5.5 is planned, in every environment — dev, the UAT instance and production — set by each feature’s model key in the environment’s task definition (§4c rule 1), so §2’s split stays the code’s default and moving back is configuration plus the feature’s evaluation run (§4c rule 6). The quality §2 gives as the reason for Sonnet on drafts and briefings is accepted as lower meanwhile; the cost falls, because Haiku 4.5 is priced below Sonnet 5.5 (§1). The backend paces its Bedrock calls to stay under the account’s requests-per-minute quota, so a burst waits instead of being throttled and opening the circuit breaker (§4). The pacing is built by F42 (.forge/specs/F42-bedrock-call-pacing-spec.md; §4, “Pacing under the account’s request quota”), and its proof on dev is the register’s §B F42 block; production’s model overrides and pacing rows are F38-i’s (.forge/owed.md, “The AI layer under Bedrock’s new-account limits”). AD #19’s pinned models do not change; because §A.3.2 of the proposal puts the writing and synthesis actions on Sonnet, the interim model is stated in the proposal header’s post-issue note of 26 Oct (L-001). How the team learns the models are available, and switches a feature once they are, is §4d |
Closed (the pacing built by F42; its proof on dev is register §B F42) |
Sources
- Anthropic official pricing — https://platform.claude.com/docs/en/about-claude/pricing (model table incl. the Sonnet 5 $2/$10 permanence note, cache multipliers, Batch API, Bedrock/CCU mechanics, new-tokenizer note)
- Bedrock model cards, read 2026-09-30 (inference profiles and Regions, §4b): Claude Sonnet 5.5 — https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-sonnet-5-5.html · Claude Haiku 4.5 — https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-haiku-4-5.html. The same day’s Anthropic pricing page gives Sonnet 5.5’s price and the Bedrock regional/geo 10% premium (§1 update)
- Gemini pricing snapshots (Aug 2026): https://www.cloudzero.com/blog/gemini-pricing/ · https://pricepertoken.com/pricing-page/model/google-gemini-2.5-flash-lite · https://benchlm.ai/google/api-pricing (2.5 Flash deprecation 16 Oct 2026)
- Claude pricing corroboration: https://www.cloudzero.com/blog/claude-pricing/ · https://benchlm.ai/anthropic/api-pricing
.forge/client/project-proposal.mdAppendix A.3 (client-committed figures) · the previous engagement’s fallback + model-tuning specs ·architecture.md § AI provenance, governance, resilience and cost