Service
LLM infrastructure and cost
LLM infrastructure is the layer that decides where model calls go, what they cost and what happens when a provider fails. Zero-Shot builds that layer: the provider stays swappable, the cost of every request is visible, and the whole system can run on your own servers when it needs to.
What we build
- 01A gateway that abstracts the model provider, without vendor lock-in
- 02Per-request cost, latency and error tracking, reported by team and product
- 03Caching and routing — cheap models for routine work, strong models for hard work
- 04Open-weight models running and scaling on your own hardware
- 05Quotas, rate limits and circuit breakers so an outage does not take you down
How it runs
01
Break down current spend
Which call costs what. Most of the cost usually turns out to sit somewhere nobody expected.
02
Put a gateway in front
Every call starts going through one door. After that, measurement, routing and provider changes become possible.
03
Bring the cost down
Caching, prompt trimming and model routing are applied, and each change is measured against the quality evaluation set.
04
Make it resilient
Provider outages, rate limits and price changes are rehearsed. A fallback that has never run is not a fallback.
What we measure
- Cost per request and the monthly cost curve
- p50 and p95 latency — averages are misleading here
- Cache hit rate
- Time survived during a provider outage
- Movement on the quality evaluation set as cost comes down
When this is the wrong fit
At low monthly volume this layer adds complexity you do not need; calling the provider directly is enough. Self-hosting is not automatically cheaper either — at low or spiky volume the cloud usually wins, and we calculate that before recommending anything.
By sector
The same work runs differently in every sector. The detail is here.
Common questions
Is running our own models cheaper?
It depends on volume. At high, steady volume your own hardware can be markedly cheaper; at low or spiky volume the cloud stays cheaper because idle hardware still costs money. We calculate the answer from your real traffic rather than asserting one.
What if a provider raises prices?
That is exactly why the gateway exists. You can move to another provider or your own model without touching application code, and we verify on the same evaluation set that the move did not damage quality.
What changes for data privacy?
Routing can be written as a rule: requests carrying sensitive data go to a local model, the rest to the cloud. The split is enforced at the gateway and logged.
Other services
What we've written on this
- Why AI agent projects failAgents that impress in a demo stall in production. Four observations on tool design, error handling, handing off to a human, and the cost ceiling.
- Is it safe to send company data to AIWhat happens when company data reaches a language model: provider terms, training use, data protection law, cross-border transfer and self-hosting.
- Making LLM costs predictableWhy AI spend surprises you at month end, and how to control it: model tiering, prompt caching, output length, and per-request ceilings.
Last updated: 2026-09-05
Let's talk about this one
Tell us what you want to build and within two days we'll tell you who takes it and how we'd measure it.