Token Cost Attribution by Team and Project

Token cost attribution means knowing exactly which team, project, or feature is responsible for every dollar an LLM call generates, at the moment it fires, not weeks later when the invoice lands. Most engineering orgs still can't do this. They can tell you the total. They can't tell you why, and that's the more expensive gap of the two.

The default setup explains why so many teams are stuck here. One or two API keys, shared across the whole engineering org, sometimes across every team the company has. Every service calls the same key. The provider dashboard sums it all into one number per billing period, and that number is accurate. It's just useless for anything beyond "here's what we spent."

No per-feature breakdown, no per-team split. If costs spike 40% in a week, nobody can point to the workflow, the agent, or the prompt change that caused it, because the API call that generated the cost never carried any information about who owned it. That's a design flaw baked into the request itself, not a dashboard problem, and no amount of better reporting fixes a request that was never structured to answer the question.

Add multiple providers to the mix (OpenAI, Anthropic, Mistral, Google, Groq, Cohere, Perplexity, whatever combination a team has settled on) and the problem multiplies. Separate invoices, separate formats, no shared identifier, reconciled by hand if it gets reconciled at all. Agentic workflows make it worse still: one user action might trigger five model calls across three providers, and without a shared trace linking them, there's no way to say what that single action actually cost.

Enterprise LLM spend hit multibillion-dollar scale by 2025, per Menlo Ventures, and many AI teams report costs blowing past forecast as they scale. The shared-key model was never built for this job. It was built for a handful of engineers hitting an API in a side project, not an org running production traffic across seven providers. Stretching a side-project pattern to cover enterprise spend is where the whole attribution problem starts, and no amount of tooling on top fixes a foundation that was never designed to carry it.

What attribution actually means, and how it differs from tracking and allocation

Three different jobs get lumped together under "cost visibility," and treating them as interchangeable is where most of the confusion comes from.

Tracking answers "how much did we spend?" It's a total, maybe broken down by model or provider. Allocation takes that pooled total and splits it across owners after the fact, usually with some formula or estimate, which makes it a modeling exercise dressed up as an answer. Attribution is the hard one: it answers "which call belongs to which owner?" and it can only answer that if ownership information was attached to the request when it fired, not reconstructed afterward.

Attribution requires instrumentation upstream of the invoice. Tracking and allocation both work backward from a document neither of them had any hand in shaping, the provider's bill, after the fact. Attribution doesn't wait around for that. It captures ownership before the call ever leaves the building, which is the whole difference and worth repeating: everything downstream depends on that one design choice.

AI spend attribution is a different animal from cloud cost attribution too, and treating them as the same problem is a mistake. Cloud teams map spend to infrastructure that already has owners and resource tags: a compute instance belongs to someone, a bucket belongs to a team. AI spend attaches to activity instead, a session, a task, an agent run, and there's no resource sitting there with a tag on it already. Outcomes don't map cleanly either. A cloud bill doesn't need to account for whether an output was good enough to ship, or how much rework a bad response caused. An AI bill kind of does, and there's no clean equivalent for that anywhere in a billing dashboard.

A majority of organizations can't accurately state their total AI spend. That's an attribution gap, not a tracking gap. They know the total. They don't know the shape of it, and the shape is the part that actually informs a decision.

Some tools offer a shortcut here: tag spend to teams or features after the fact, without touching any infrastructure. Fine for a first pass at showback, a rough sense of who's spending what. But call it what it actually is: allocation wearing attribution's clothes. Real attribution means the ownership signal travels with the request itself, solved in the request layer, not reconstructed in a report someone runs after the month closes.

The tagging schema that makes attribution possible at the request level

Attribution starts with structured metadata, attached to every LLM call before it ever leaves the application. A handful of tag dimensions show up again and again as the backbone of a working schema:

• team_id: the squad or business unit responsible for the feature making the call
• project_id: the product or workstream the call is serving
• feature: the specific capability inside that project, summarization, classification, response generation, whatever it is
• customer_id: when AI sits inside a customer-facing product and per-customer cost actually matters
• environment: keeps production traffic separate from staging and dev noise

That last one gets skipped constantly, and skipping it causes real damage. Without environment tags, staging runs and testing traffic get folded into the same numbers as production, and the per-feature unit economics come out wrong, sometimes badly wrong, because a chatty test suite is padding the "real" cost of a feature that hasn't even shipped yet.

Think of the tagging schema as a contract between product and engineering. Product decides which dimensions actually matter for accountability, who needs to answer for what. Engineering makes sure every call carries them. The failure mode shows up when that contract isn't enforced: some services tag their calls, some don't, and the result is partial attribution. That's worse than no attribution at all, because it looks trustworthy and isn't.

Agentic pipelines need one more layer. Tagging the top-level user request isn't enough if that request spins off five separate model calls under the hood. Each of those calls needs to carry a shared trace identifier, so the cost of the whole chain rolls up to one place instead of scattering across five untraceable line items.

None of this works without somewhere to enforce it, though. Tags are necessary, but they're not sufficient on their own. Someone has to capture them, store them, and make them queryable, which means there has to be a single point that sees every request going out.

Why the gateway is the right enforcement point for attribution, not the application and not the provider

Three candidates for where attribution logic could live: the application, the provider, or the gateway sitting between them. Only one of them actually works, and the other two fail for reasons worth spelling out plainly.

Application-level logging fails because it depends on every single service remembering to do it, consistently, with the same schema. That almost never happens across a real engineering org with a dozen teams shipping independently. One team logs cost per request. Another logs cost per session. A third doesn't log it at all. There's no single place to query across all of it, because there was never a single standard to begin with.

Provider dashboards aren't built for this either, and expecting them to be misreads what they're for. OpenAI's usage dashboard supports attribution on dimensions like user and project, which is genuinely useful, but it has no concept of custom tags, no environment labels, no workflow identifiers. If cost spikes, tracing it back to a specific feature means leaving the dashboard entirely and cross-referencing application logs by hand. Anthropic's Enterprise Analytics API, launched in 2026, goes further: per-user attribution, named user consumption, individual token and cost data, ninety days of history. Useful, but it comes with a three-day delay on engagement data, it's enterprise-tier only, and there's still no per-request granularity underneath it.

The gateway closes both gaps at once. It sits between every application and every provider, so it sees every request regardless of which service sent it, and it enforces one consistent tagging schema and one logging standard across the entire org, instead of five different standards depending on who built what.

The observability has to be native to the gateway, not bolted on afterward. If request data has to get forwarded somewhere else to be analyzed, traces fragment across tools and something always gets lost in the handoff. Kept in the same pipeline, cost data sits right alongside latency and error data, so a cost spike and a latency spike from the same bad deploy show up together instead of in two dashboards nobody cross-references.

There's a second reason the gateway wins, and it's arguably the bigger one: it's the one place holding actual provider credentials centrally. That means it can issue scoped virtual keys to teams that carry identity into every request automatically, without depending on a developer remembering to tag a call by hand.

Industry analysts project that by 2028, most software engineering teams building multimodel applications will run an AI gateway, up from roughly a quarter in 2025. That jump reflects other factors beyond performance gains. It's a governance need: teams are hitting the wall described above, and gateways are the answer that's actually holding up under real production traffic.

How virtual keys translate team identity into per-request cost records

A virtual key is a scoped API key the gateway issues to a team, a project, or a service. The gateway holds the real provider credentials behind it; the application never sees them, never touches them, never gets the chance to leak them into a repo by accident.

Every request that hits the gateway carries whichever virtual key sent it. That alone gives the gateway team identity on every call, without asking the application to tag anything by hand. Combine that with the metadata tags from the schema above, and here's what a gateway can log for every single request:

• Which virtual key fired it
• Which provider and model handled it
• Input and output token counts, tracked separately
• Latency at each step of the request
• Cost in dollars, at published provider rates
• Whether a fallback triggered, and which provider it fell back to
• Any metadata tags attached to the call

Put those together, who sent it, what it was for, where it went, how much it cost, and there's a complete cost record for every request, sliceable by any dimension, with nothing to reconstruct after the fact.

Virtual keys double as a control surface, too. A key can be locked to specific models, capped at a token budget, rate-limited, or restricted to one environment, so a staging key literally cannot burn through production budget even if someone wires it up wrong. Revocation is instant: a compromised or dead key gets shut off at the gateway, no code changes, no rotating credentials at the provider.

For customer-facing products, keys can be issued per customer or per tenant. That turns raw provider cost into per-customer unit economics without adding a single line of extra instrumentation, since the gateway is already logging everything needed to answer what a given customer actually costs.

Real-time aggregation versus end-of-month reconciliation, and what each one enables

End-of-month reconciliation is the default for any team without a gateway, and it's a bad default, not a neutral one. Finance gets the provider invoice, asks engineering to explain the number, and engineering doesn't have request-level data to answer accurately. Everyone's guessing, just with more confidence than the situation warrants.

Per CloudZero's 2026 survey, 30% of finance leaders still reconcile AI spend by hand. That's a process measured in weeks, and by the time the explanation arrives, the spend that caused it happened a month ago and can't be undone.

Real-time aggregation flips that. The gateway emits cost data as requests complete, so a dashboard reflects current spend against budget without waiting for a billing cycle to close. That difference isn't cosmetic:

• Catching a runaway agent or a misconfigured prompt before it burns through a month's budget in a day
• Comparing cost-per-feature day over day as code ships, instead of guessing which release caused the spike
• Spotting which team's usage is driving a spend increase while there's still time to act on it
• Walking into a budget review with a live number instead of a reconstructed one

Worth separating two related ideas here: showback and chargeback. Showback surfaces cost data to teams as an awareness tool, without actually charging them for it. It's a good starting point, since it builds cost-conscious habits without the political friction of moving money around internally. Chargeback goes further and enforces actual internal transfers based on usage. Both need the same underlying data. Chargeback just adds a policy layer on top of it.

The FinOps Foundation's State of FinOps 2026 report identified pre-deployment guidance and granular monitoring of tokens, LLM requests, and GPU utilization among the most requested tooling capabilities. The same report noted that managing AI spend went from a minority practice to near-universal scope in just two years. That tracks with what shows up in per-request records once teams actually start looking at them: a meaningful chunk of token spend goes to capability nobody ends up using. Invisible in a monthly invoice. Obvious the moment there's a per-request line to check it against.

Budget enforcement at the gateway: rate limits, spend caps, and hierarchical controls

Attribution without enforcement is just watching the number climb. Enforcement without attribution is a blunt instrument, a hard stop with no idea which team or feature actually tripped it. Neither one is worth much alone, and teams that pick just one are solving half a problem.

Enforcement has to run on tokens, not request counts. Getting this wrong is a common mistake, and it's worth being direct about why: a single request can cost a hundred times more than another depending on how long the prompt and response run, so capping "requests per minute" misses the actual unit that shows up on the bill. Budgets need to track what's actually being billed, not a proxy for it.

A working gateway supports enforcement at several nested levels:

• Per-virtual-key limits, the tightest scope, covering a single service or feature
• Per-team limits, budget pools assigned to a squad or business unit
• Per-project limits, spending caps tied to a specific use case or deployment
• Organization-wide ceilings, the hard backstop underneath everything else

When a limit gets hit, the gateway should return a clear, structured error, typically a 429 with a retry-after header, rather than letting the request fail unpredictably downstream or letting spend quietly keep climbing past the cap. Enforcement policy itself is a choice, not a fixed behavior: some teams want a hard stop the instant a budget's hit, others want an alert with a grace window before anything actually blocks. A gateway worth using supports both, because risk tolerance genuinely differs by team.

Agentic pipelines need special handling here. A multi-step agent can burn through a real budget across dozens of small calls, none of which individually crosses a per-request limit. Enforcement has to happen at the session or trace level, cumulative across the whole chain, or the limit means nothing.

The clearest illustration of what happens without this: one widely discussed rollout of AI coding tools to thousands of engineers, without per-team budget enforcement in place, burned through a full year's AI budget in roughly four months. Attribution paired with enforcement would have flagged that trajectory weeks in, while there was still time to adjust course instead of just watching the number blow past the ceiling.

What the attribution data reveals about model selection and where routing compounds the savings

Attribution surfaces cost per feature, not just cost per model, and those two numbers frequently tell very different stories. This is where most teams are optimizing the wrong thing: they tune which model to call, when the bigger lever is which feature deserves a frontier model in the first place.

Take a typical enterprise pipeline: intent classification, entity extraction, document summarization, response generation. The first three of those don't need frontier-model capability at all, full stop. Routing them to a smaller, cheaper model cuts cost without touching quality, because the task never demanded that much model in the first place. The price gap between frontier models and capable mid-tier models is large enough in 2026 that this routing decision is worth real engineering time, but only if attribution data actually shows which tasks are driving the spend. Without that data, the routing work is a guess dressed up as an optimization.

Four routing strategies show up consistently, each suited to a different kind of workload:

• Static: the task type decides the model, fixed at configuration time
• Dynamic: a complexity score at runtime picks the small model or the large one
• Semantic: the incoming query gets embedded and matched to a domain cluster, useful when workloads split cleanly into specializations like code, medical, legal, or customer support
• Cascade: try the cheap model first, escalate to the stronger one only if the output fails a quality check

Routing isn't free, though. The embedding lookup or the scoring pass adds its own latency, and if that overhead creeps past roughly 50 to 100 milliseconds, it can eat the exact savings the routing was supposed to deliver by picking a faster downstream model. Attribution data is what tells a team whether that tradeoff is actually paying off, instead of assuming it is.

Caching is the second lever attribution exposes, and it's the one teams notice last. If the same or near-identical queries keep getting sent at full inference cost, semantic caching can return a stored response instead of re-running the model. AWS research found meaningful cost reduction for chatbot query workloads using this approach, at the right similarity threshold. Without per-request attribution, nobody notices the repetition in the first place. With it, routing and caching stop being guesses and start being experiments with a number attached to the outcome.

How to evaluate whether a gateway's attribution capabilities are production-ready

The market splits into open-source and managed options, and the choice comes down to more than preference. It's about whether a team has the operational bandwidth to run and patch a gateway itself. If it doesn't, managed is the honest answer, not a compromise.

LiteLLM is the clearest open-source entry point: supports 100+ providers and a wide range of models, with cost tracking built in from the start. Worth knowing going in that some capabilities, SSO past five users, audit logs, the more advanced UI, sit behind an enterprise tier, so the depth of attribution actually available depends heavily on which tier a team is running.

Whatever gateway gets evaluated, the attribution-specific checklist doesn't change. Does it log token counts separately for input and output, not just a combined number? Does it support the full tag schema, team, project, feature, customer, environment, without forcing a workaround? Does it enforce budgets at the token level, hierarchically, from virtual key up through org-wide ceiling? Can it trace a multi-call agent chain back to a single session instead of scattering it across five untraceable entries? And does the observability live natively inside the gateway, or does it require piping data somewhere else and hoping nothing gets lost on the way?

A gateway that answers yes to all of that is a full-fledged control plane sitting between an application and a provider. It's the system of record for what AI spend actually costs, broken down by the people and the work responsible for it. That's the gap between a team that controls its LLM bill and a team that finds out what it spent a month after the fact, with no way to explain why.