November 11, 2026 / 9 min read
A coding agent estimated that a run cost $0.044. The model was free.
That contradiction points at a bigger problem. A software factory runs many AI agents on tickets, and each brings its own credentials, its own way of calling models and tools, and its own report of what happened. You lose control over what they may do and visibility into what they did.
A gateway is meant to fix that. It sits between the agents and everything they call, and routes each request while it applies access rules, limits and logging. Think of it as a company card for each agent: a spending limit, a list of what it may use, and a statement of every charge. How much does it actually give you? We ran three coding agents through four gateways to find out.
A coding agent works in a loop: it asks a model what to do, runs a command or calls a tool, and feeds the result back. Each step can cost money or have side effects. Agents need the same foundations as any application, such as access control, logging and cost attribution, but they decide for themselves which models and tools to use.
An LLM gateway sits between agents and model providers, and an MCP gateway does the same for tools. MCP, the Model Context Protocol, is a standard way for agents to reach tools such as Jira. OpenTelemetry (OTel) is a standard way to collect telemetry, and its protocol, OTLP, carries that data between systems.
Think of this layer as a front door. Agents hold a limited key, the gateway holds the real credentials, and every request passes through one place that can apply rules and record what happened. That door has three jobs, and this post follows them:
We recorded 61 runs, nine of them ending in an error, some on purpose in budget and model-block tests. Nearly all used one small task: read a file and explain a behaviour, which kept the workload comparable.
Scope: a small sample, and not every agent ran on every gateway. We tested the control layer, not model output, and four of many gateways. Others, such as LiteLLM, Portkey and Kong, may behave differently.
A gateway earns its place when something changes, so we compared the changes a platform team actually makes.
| Change you need to make | Bifrost | agentgateway | Cloudflare | Vercel |
|---|---|---|---|---|
| Add Claude Code or Codex | Works | Works | Works | Paid credits only |
| Trust the cost | Exact on Claude | Exact on Claude | Verify first | Verify first |
| Cap a team's spend | Stops with 402, can overshoot | Stops with 429, can overshoot | Docs: best-effort limits by model, provider or metadata | Docs: budgets per team, project, key or member |
| Restrict a key's models | Blocked with 400, unclear error | Blocked with 403, clear error | Not found | Docs: team-level allowlist only |
| Swap to an open-weight model | Works | Works | Partly, agent failed on call 2 | Works |
| Control one MCP tool | Hidden and refused | Hidden and refused | Docs: MCP portals can turn off single tools | Docs: not in AI Gateway |
| Self-host it | Yes | Yes | No | No |
Cells starting with "Docs:" come from vendor documentation, which we did not run. "Not found" means we found nothing there. Every other cell is a result from our runs or a product fact.
The gateways we tested checked recorded spend before a call and charged it afterwards, so the call that crossed the limit still ran. Cloudflare's and Vercel's docs describe the same pattern, which we did not test. A gateway could instead reserve the worst case up front, since input tokens are known and max_tokens caps the output, then release the rest, like a card hold. The price is caution: a key near its limit may refuse calls it could have afforded.
A software factory makes this worse, because it runs many agents at once and all of them pass the check before any finishes. We did not run that case, but Cloudflare's docs describe it, and a reservation would close it too.
A factory also needs budgets in layers:
| Cap per | What it limits | Why |
|---|---|---|
| Agent run | One ticket's work | One stuck ticket cannot drain the rest |
| Project | All runs for one project | Keeps each project inside its share |
| Organization | All agent spend | A hard ceiling on the total |
A gateway can enforce these only if every request says which run and project it belongs to, for example through a separate key or metadata per run.
Test your own clients before you rely on a budget. They react differently to a refusal:
402 five times in about seven seconds, then gave up.429 until our five-minute timeout, in both of two runs.Agents also act on external systems through tools, and that is a separate security concern: a bad model call wastes money, a bad tool call can delete a project. An MCP gateway extends the front door to tools. The agent holds a limited key, the gateway decides which tools that key can see and call, and the real tool credential stays at the gateway.
To test it, we gave two keys different rights on a fake Jira server behind each gateway. The server rejects any call without its own token, and the client never holds it.
Below is the real output through Bifrost, trimmed. The same test through agentgateway ended the same way.
$ npm run mcp -- bifrost
no key: 401
dev
tools visible: fake_jira-jira_create_issue, fake_jira-jira_search_issues
search: ALLOWED: jira_search_issues ran on fake Jira
direct delete: REFUSED (200): Tool execution failed: MCP tool
'fake_jira-jira_delete_project' is not allowed for virtual key
admin
tools visible: fake_jira-jira_create_issue, fake_jira-jira_delete_project,
fake_jira-jira_search_issues
direct delete: ALLOWED: jira_delete_project ran on fake Jira
What the run shows:
REFUSED (200), the HTTP request succeeded, and the gateway rejected the tool call inside the MCP response.If you self-host, keep the admin port private. In our setup one gateway's admin interface answered without authentication, and its configuration listed the keys the agents use.
Budgets only help if you know what agents did and the numbers are right. Gateways show the same traffic differently: some lead with tokens and cost, and one stored the request content.
| Source | What it reported | Verdict |
|---|---|---|
| Claude Code on Claude | Matched both self-hosted gateways to the last digit | Accurate |
| Claude Code on free Nemotron | Estimated $0.044 to $0.084 per run, against a $0 price on Vercel | Wrong price |
| Codex | Token counts, but no dollar figure in any signal we captured | Calculate it yourself |
| A hosted gateway on Claude | About 4.7× the price the other sources applied, on mostly cached runs. Fits cache reads billed near the full input price | Differed |
| Bifrost, agentgateway | Per-call cost including cache. Exact on Claude, and close to each other on GPT | Accurate |
Three numbers are easy to confuse, and they need not match:
What we saw:
modelPricing, which we did not test.input_tokens leaves out cache reads: one call showed 2 in Claude Code and 17,268 at the gateway. Normalize fields before you mix sources.Agents resend long prompts on every call, and providers charge much less for input they have seen recently. That cache lives at Anthropic or OpenAI, not in the gateway: Bifrost and Cloudflare, using one key, shared it seconds apart. Both misses we saw came from the client.
Claude Code changes its request with the base URL. Moving from direct to a gateway gave a cold first call. A proxy that forwards requests untouched went cold too, on the same key, so the request changed on the client side, not in the gateway.
In our tests, Codex started every session cold. It sends a new prompt_cache_key per session, a request field that OpenAI's prompt caching docs use to group requests for caching. When our proxy pinned that key to one value, the next two runs read 10,594 tokens on their first call.
Gateway telemetry is per call, but a factory needs answers per ticket: what a run cost and which project it was for. Give every call a correlation ID, such as run, ticket and project, and pass the trace context from agent to gateway. In our lab a run ID and traceparent were enough to show one run as a single trace.
The third job is knowing where data goes. Prompts can contain source code and customer data, so many teams must keep them inside a region, which is called data residency. A request can pass through several services, and each may process or store part of it. A gateway in Frankfurt does not keep prompts in Europe: the gateway, the model host, the telemetry collector and any observability backend that receives prompt text all count.
Keep prompts out of telemetry. Make sure spans do not carry full prompts and responses. In our setup one gateway exported them by default, and tool arguments still slipped past our filter, so filter on an allowlist.
The model host is usually the deciding hop, and options differ by vendor. This table reflects vendor documentation as we checked it on 5 November 2026. Regional options change, so check them again before you rely on one.
| Route | How you choose the region | Watch for |
|---|---|---|
| Direct API | ||
| Claude API, direct | inference_geo: global or US | No EU value documented |
| OpenAI API | EU project and eu.api.openai.com | New projects only, check eligibility |
| Cloud infrastructure | ||
| Claude on Bedrock | eu. inference profile | Can process in several EU Regions |
| Claude on Vertex AI | EU endpoints | The eu endpoint excludes the UK and Switzerland |
| Open-weight hosting | ||
| Open-weight on Scaleway | api.scaleway.ai | Stated as Paris. The only EU route we tested |
A gateway makes switching easy. We added two Scaleway models to agentgateway as aliases, and OpenCode finished the task on both with no change. That shows the route works, not where the model ran.
Start from your requirements. These apply even if you pick none of the products we tested:
| Requirement | What to prioritize |
|---|---|
| Centralized credentials | Virtual keys, with provider credentials held at the gateway |
| Fine-grained model access | Per-key model lists, or rules on role |
| MCP security | Per-tool authorization enforced by the gateway |
| Cost management | Per-call cost that counts cache, and budgets you understand |
| Observability | OTLP support, trace propagation and configurable filtering |
| Data residency | Control over every service that processes or stores prompts |
| Infrastructure independence | Self-hosting and configurable routing |
A software factory runs many agents on tickets nobody watches live. A gateway gives you one place to control what they access, see what each run cost and decide where their data goes. In our tests it did not guarantee accurate cost, firm budgets, clean telemetry or data residency on its own, so verify those. The agent ecosystem will keep changing. Your front door shouldn't have to change with it.
Blog
A coding agent can turn a ticket into a green pull request in minutes. We split the merge decision into narrow checks, let fixed rules decide, set the strictness by the blast radius of a change, and bring in a person only where the agent gets stuck.
We compared a stronger model, a different agent, answers from the author and a read-only investigation before any code. On 19 fresh tickets, the investigation went ahead on 9 of 10 wanted changes and held back 7 of 9 requests an author had already declined.
We ran OpenSpec, Spec Kit, Superpowers and Claude Code plan mode on a real ticket, then planted drift and contradictions. For one feature we couldn't separate the code, and plan mode cost about half. What differs most is the record each leaves for the next agent.
With the team behind the first AI for code.
Backed by frontier coding agent research with ETH Zurich.
Cookie settings
By clicking 'Accept all cookies', you agree to storing cookies on your device to enhance site navigation, analyze site usage and assist in our marketing efforts as outlined in our cookie policy.