Three buckets (do not merge them on one slide)

Build covers architecture, prompt design, tool wiring (CRM, ticketing, internal APIs), evaluation harnesses, guardrails, and human-in-the-loop workflows when automation stops.

Inference burn is tokens × traffic — plus retrieval if you are grounding answers in your docs.

Ops is everything after launch: eval suites, regression checks when OpenAI ships a new mini-model, incident response, and the dashboard your PM actually looks at.

• Build: one-time + finite retainer through stabilization.
• Inference: variable with usage — model choice matters more than hosting.
• Ops: surprisingly flat monthly if you plan for it — painful if you do not.

Build phase — what we line-item (CAD ranges)

These assume you already have product definition for what “good” looks like — not a science project to “see what GPT can do.” Science projects belong in a time-boxed spike, not a production roadmap.

Inference burn — worked example (ballpark)

Picture a B2B SaaS support agent: four hundred conversations a day, eight turns average, ~750 tokens in and ~450 tokens out per turn (rough — your prompts vary). That is about 3,200 turns/day × 1,200 tokens ≈ 3.8M tokens/day → ~115M tokens/month.

Pricing moves whenever providers ship new models — treat numbers below as order-of-magnitude, not accounting truth. At illustrative blended rates of roughly $4 per million tokens for a frontier-class model (mix of input/output), raw API spend lands near hundreds of dollars a month at this scale — before retrieval, before redundancy, before human review of edge cases.

Now double your estimate for retries, evaluation runs, and staging environments. Then add 30% ego margin because marketing will ask for “smarter answers” (longer outputs) after launch.

Retrieval & vector infra

pgvector inside Postgres is attractive when you already run Postgres — ops stays boring. Pinecone or Weaviate shine when you need managed scaling or hybrid search across embeddings — dollars trade for engineering hours.

We have seen teams burn weeks tuning chunk sizes while ignoring evaluation — do not be that team. Better chunks with worse embeddings lose to mediocre chunks with a weekly eval script your PM trusts.

Observability — the line item everyone deletes

Langfuse, Helicone, or CloudWatch + structured logs — pick something your engineers will actually query when a customer says “it lied last Tuesday.” Without traces tied to prompt versions, you are debugging by vibes.

We bundle a minimum observability package in every agent proposal. Removing it is like shipping payments without reconciliation — technically possible, professionally negligent.

Self-host an OSS model vs API-only

API-only wins until inference spend crosses pain thresholds or until data residency demands it. Self-hosting Llama-class models comes with GPU bills, fine-tuning pipelines, and on-call engineers who understand CUDA grief.

Hybrid is common: API for dev velocity, bring-your-own-model later once economics and legal clarity demand it.

Frequently asked questions