The headline: AI margins sit 20–30 points below SaaS
The commonly cited gross margin benchmark for traditional SaaS is 75–85%. In our modelled sample of 12 early-stage AI application companies, the median gross margin is 51%, and eight of the twelve sit between 44% and 61%. Only one clears 65%. That is not a rounding difference: it changes how much a pound of revenue is worth, how long customer acquisition takes to pay back, and what a sensible valuation multiple looks like.
The reason is structural. In classic SaaS, the marginal cost of an extra user is a sliver of database and hosting capacity. In an AI application, every unit of value delivered triggers model inference, and inference is cost of goods sold. Four mechanisms push it up:
- Cost scales with usage, not seats. A seat-priced contract earns the same whether the user runs ten tasks a month or six hundred; the inference bill does not.
- Agents multiply calls. A single user request in an agentic product can fan out into five to fifteen model calls (planning, tool use, re-reading, checking).
- Context is expensive. Legal, compliance and health products feed long documents into every call, and input tokens dominate the bill.
- Humans remain in the loop. Review queues, escalations and onboarding support still sit in cost of revenue at this stage.
An AI start-up’s gross margin is not a property of its market. It is the residue of a dozen engineering and pricing decisions, most of which can be reversed.
The master table
Inference cost is measured per active user per month, where an active user is a paying seat that triggered at least one model call in the month. CAC payback is margin-adjusted: customer acquisition cost divided by monthly gross profit per customer, not monthly revenue.
| Company (archetype) | Region | Stage | ARR ($m) | Gross margin | Inference cost per active user per month ($) | GPU/cloud commitment | CAC payback, margin-adjusted (months) |
|---|---|---|---|---|---|---|---|
| A: legal copilot | UK | Seed | 1.2 | 52% | 38 | None | 21 |
| B: agentic customer support | US | Series A | 6.5 | 48% | 52 | Committed spend | 26 |
| C: code review tool | EU | Seed | 2.1 | 61% | 9 | None | 17 |
| D: voice agent | UK | Series A | 4.8 | 41% | 44 | Reserved | 31 |
| E: vertical health scribe | US | Series A | 9.2 | 58% | 27 | Committed spend | 19 |
| F: fintech compliance agent | EU | Seed | 1.6 | 50% | 31 | None | 24 |
| G: sales research agent | US | Seed | 2.4 | 46% | 19 | None | 23 |
| H: insurance document extraction | US | Series A | 7.8 | 67% | 6 | Reserved | 15 |
| I: recruitment screening agent | UK | Pre-seed | 0.6 | 38% | 16 | None | 34 |
| J: manufacturing knowledge assistant | EU | Series A | 5.1 | 56% | 12 | Committed spend | 22 |
| K: contract review platform | US | Series A | 11.4 | 64% | 24 | Committed spend | 16 |
| L: marketing content generation | EU | Seed | 3.0 | 44% | 8 | None | 28 |
| Median | Mixed | Mixed | 3.9 | 51% | 21.5 | 6 none, 2 reserved, 4 committed | 22.5 |
Two things stand out. First, absolute inference cost is a poor predictor of margin on its own: Company C spends $9 per user and earns 61%, while Company L spends $8 and earns 44%. What matters is inference as a share of revenue per user. Implied monthly revenue per active user in the model ranges from $20 (L) to $140 (B), so L’s $8 is 40% of revenue while C’s $9 is 30% of a $30 seat. Across the sample, the median inference share of revenue is 34%; in the top three by margin it is 15–30%.
Second, the residual cost of revenue (hosting, third-party data, telephony, human review, support) runs at 6–21 percentage points of revenue. Founders who model only token costs understate cost of goods by a fifth or more.

Inference cost per active user: what actually drives it
Four levers explain most of the variation in the inference column:
- Context length. Input tokens usually outnumber output tokens by 10 to 50 times in document-heavy products. Stuffing whole contracts or policy manuals into every call, rather than retrieving the relevant passages, is the single most common source of waste we see in the archetypes.
- Agent steps. Each extra step in a plan re-sends a growing context. A five-step agent does not cost five times a single call; with accumulating context it can cost eight to twelve times.
- Model mix. Sending every call to a frontier model is the default in early prototypes. Routing classification, extraction and formatting steps to smaller models typically handles a majority of calls at a fraction of the price. Research on cascades and routing, such as FrugalGPT, shows large savings at comparable quality on suitable tasks.
- Caching. System prompts, clause libraries and shared reference documents repeat across calls. Prompt caching, offered at the time of writing by several major model providers at a discount to standard input pricing, attacks exactly the largest line in most bills.
The decomposition below takes Company A, the UK legal copilot, at $38 per active user per month. The modelled user runs 220 tasks a month (about ten per working day), averaging 3.2 model calls and roughly 45,000 input tokens per task, at a blended input price of $2 per million tokens.
| Cost driver | Current ($) | After routing and caching ($) | Change |
|---|---|---|---|
| Retrieval and embeddings | 1.90 | 1.90 | Unchanged |
| Long-context input (documents, clause library) | 19.80 | 9.10 | Shared clause library and system prompt cached |
| Agent steps (re-reads, multi-step drafting) | 8.40 | 4.20 | Planning and checks routed to a smaller model |
| Output tokens | 4.60 | 3.90 | Routine drafts routed to a smaller model |
| Guardrail and evaluation passes | 2.10 | 2.10 | Unchanged (already on a small model) |
| Retries and failed runs | 1.20 | 0.60 | Better output validation |
| Total | 38.00 | 21.80 | −43% |
At an implied $120 per seat per month, Company A’s cost of revenue is $57.60, of which $38.00 is inference. Removing $16.20 takes cost of revenue to $41.40 and gross margin from 52% to about 65%, without a price rise. The trade-off is engineering time (a routing layer needs an evaluation set, or quality silently degrades) and some cache-invalidation complexity when the clause library changes.
GPU cloud commitments and the utilisation trap
Four companies in the sample hold committed-spend contracts and two hold reserved capacity. The two instruments carry different risks.
- Reserved capacity buys a fixed number of GPUs for a term (often one to three years) at a discount to on-demand rates. You pay for every hour, used or not.
- Committed-spend contracts promise a minimum spend with a cloud or model provider over a term in exchange for a discount on consumption. Any shortfall is still billed.
Both convert a variable cost into a fixed one, which is precisely the wrong shape for a start-up whose usage forecast is a guess. The trap is utilisation: a discounted GPU-hour that sits idle costs more per unit of work than an expensive one that is busy.
Worked example: Company D at 40% and 80% utilisation
Company D, the UK voice agent, self-hosts speech and language models on eight reserved data-centre GPUs at an illustrative $2.50 per GPU-hour on a 12-month term.
- Monthly cost: 8 GPUs × 730 hours × $2.50 = $14,600, fixed.
- Assumed throughput: one fully loaded GPU-hour serves 600 call minutes, so full capacity is 5,840 GPU-hours × 600 = 3.5m minutes a month.
- At 80% utilisation: 2.8m minutes served, cost $5.21 per 1,000 minutes.
- At 40% utilisation: 1.4m minutes served, cost $10.42 per 1,000 minutes.
If an equivalent managed API costs an illustrative $8 per 1,000 minutes, the reservation breaks even at about 52% utilisation. Voice traffic for UK business customers is concentrated in roughly ten weekday hours, so a 24-hour reservation struggles to average above 40–45% without overnight batch work to fill it. That is why D, despite a lower headline unit price, carries the second-lowest margin in the sample.
Committed spend fails differently. Suppose Company B commits $1.2m over 12 months for a 20% discount and consumes only $850,000 at the discounted rate. That usage would have cost about $1.06m at list price, yet B pays the full $1.2m: an effective premium of roughly 13% over list, not a 20% saving. The rule we apply: commit to no more than about 80% of trailing six-month consumption, and only reserve capacity you can keep above 70% busy.
Locking in GPU capacity early is the only way to protect margins as you scale.
Reserved capacity only beats on-demand above a break-even utilisation, which in our voice example is about 52%. Below it, the discount becomes a premium.
CAC payback: the honest measure
Many pitch decks report CAC payback on revenue: acquisition cost divided by monthly revenue per customer. That convention was tolerable when gross margins were 80% and every SaaS company used it. At 50% margins it flatters AI businesses by roughly a factor of two, because only gross profit can repay acquisition spend.
Take Company B, the US agentic customer support company. A typical mid-market customer buys 40 seats at $140 per seat per month: $5,600 monthly revenue, $67,200 in annual contract value. Fully loaded CAC (sales, marketing and solutions engineering) is modelled at $70,000.
- Revenue-based payback: $70,000 ÷ $5,600 = 12.5 months.
- Margin-adjusted payback: $70,000 ÷ ($5,600 × 48%) = $70,000 ÷ $2,688 = 26 months.
- For reference, the same CAC at an 80% SaaS margin pays back in 15.6 months.
The headline 12.5 months looks top quartile. The honest 26 months means B needs customers to stay well beyond two years before acquisition spend is even recovered, and churn in a category where buyers are still experimenting makes that uncertain. The sample median is 22.5 months margin-adjusted; on the revenue convention it would read as about 11.
Patterns: what separates the top and bottom quartiles
The top quartile by margin is H (67%), K (64%) and C (61%). The bottom quartile is I (38%), D (41%) and L (44%). With twelve modelled companies these are patterns, not statistically robust findings, but the mechanisms are consistent.
What the top quartile does differently:
- Routing by default. All three send the majority of calls to small or fine-tuned models and reserve frontier models for the hard minority. H runs a fine-tuned extraction model on reserved GPUs that it keeps busy with batch claims processing around the clock, the inverse of D’s problem.
- Aggressive caching and retrieval. Short, retrieved context instead of whole documents.
- Outcome or unit pricing. H prices per document processed and K per contract reviewed, so revenue rises with the same usage that drives cost. Heavy users become profitable rather than dilutive.
- Owning the workflow. K and E sit inside the system of record (the contract repository, the clinical note), which supports higher prices and lowers churn, improving payback from both directions.
What the bottom quartile shares:
- Flat seat or subscription pricing with no usage caps, so the heaviest decile of users consumes a disproportionate share of inference.
- A single frontier model for every call, chosen during prototyping and never revisited.
- Low price points (L at $20 per seat, I at $35) that leave no room for inference at 40–46% of revenue.
- Fixed infrastructure sized for peak rather than average load (D).
- Thin product differentiation, which makes it hard to raise prices as model costs fall for everyone.
Transatlantic differences
The modelled regional medians are 58% gross margin for the five US companies, 53% for the four EU companies and 41% for the three UK companies. The UK figure rests on three rows and should be read as directional at most, but two structural factors push in the same direction.
Contract shape. UK and EU buyers, particularly in legal, financial services and the public sector, prefer fixed annual contracts with predictable budgets. In our model six of the seven UK and EU companies sell on fixed annual terms, against two of the five US companies. A fixed price transfers usage risk to the vendor: when a customer’s staff adopt the product enthusiastically, the vendor’s margin falls. US buyers more readily accept usage-based or hybrid pricing (a platform fee plus metered tasks), which keeps margin stable as usage grows. UK and EU founders can recover some of this with tiered seat bundles, fair-use caps written into the contract, and annual true-ups.
Data residency. EU customers frequently require processing and storage inside the EU, driven by the GDPR, sector regulation and procurement policy, with the EU AI Act adding obligations for some use cases. In-region inference endpoints can carry a price premium or offer fewer model choices, and self-hosting in-region removes access to the cheapest global capacity. Our model applies a 10% uplift to inference for EU rows, worth roughly two to four points of gross margin. UK companies selling into the EU face the same requirement, alongside the ICO’s guidance under UK GDPR at home. US health products bear their own compliance cost (Company E’s hosting line includes HIPAA-eligible infrastructure). Rules and provider pricing change; at the time of writing, check the current guidance and contracts.
Benchmarks to target by stage
These targets are desk estimates calibrated to the modelled sample, not industry-wide statistics. They are intended as negotiating and planning anchors.
| Stage | Gross margin | Inference as share of revenue | CAC payback, margin-adjusted (months) | Commitment posture |
|---|---|---|---|---|
| Pre-seed / seed | 45%+ with a modelled path to 60% | ≤30% | ≤24 | None; on-demand and pay-as-you-go APIs only |
| Series A | 55–65% | ≤22% | ≤20 | Reserve only above 70% sustained utilisation |
| Series B and later | 65–75% | ≤15% | ≤18 | Commit up to 80% of trailing six-month spend |
| Traditional SaaS (reference) | 75–85% | n/a | 12–18 | Not applicable |
The aim is not to reach SaaS margins at seed. It is to show that margin is a managed variable: a decomposition like Company A’s, a pricing model whose revenue tracks cost, and no fixed commitments sized to a forecast rather than to observed load. A company at 50% that can explain how it gets to 65% is a better investment than one at 60% that cannot explain why.



