LiteLLM AI Gateway
Learn how to bring LiteLLM gateway consumption data into Ternary — natively through the AI Gateway integration, or through Bring Your Own Data (BYOD) — to allocate AI and LLM spend by team, model, and provider.
The Ternary LiteLLM integration brings AI/LLM consumption data from a self-hosted LiteLLM proxy into Ternary, so you can analyze AI spend by team, model, and provider alongside your other cloud costs — and allocate your invoiced AI spend across teams and cost centers.
AI provider invoices (Anthropic, OpenAI, and others) arrive as opaque line items with no team or project attribution. Your LiteLLM gateway, by contrast, sees every request — priced and tagged with team, key, and model. This integration sends that consumption data to Ternary as FOCUS-formatted cost data.
How allocation worksThe invoice is the source of truth for dollars; the gateway supplies the proportions. LiteLLM's per-request cost is an estimate, so it is used to weight how invoiced AI dollars are split — never reported as billed truth. See Using the data below.
Choose a method
There are two ways to connect LiteLLM. Use the AI Gateway integration if it is available to you; use BYOD if you run an older LiteLLM version or want the export to land in storage you own.
| AI Gateway integration (recommended) | Bring Your Own Data (BYOD) | |
|---|---|---|
| LiteLLM setup | callbacks: ["ternary"] and three environment variables | callbacks: ["focus"] plus a GCS or S3 destination |
| Storage and access | None — LiteLLM sends data straight to Ternary | Your own GCS or S3 bucket, with read access granted to Ternary |
| Ternary setup | Create an AI Gateway integration — no schema mapping | Create a Bring Your Own integration and map the schema |
| Token counts | Input, output, cache-read and cache-write tokens | Request counts only |
| AI provider and model | Populated automatically | Map the FOCUS columns yourself |
| LiteLLM version | v1.108.0 or later | 1.98.0 or later |
Method 1: AI Gateway integration
LiteLLM's built-in ternary callback uploads your gateway's spend to Ternary on a schedule. There is no bucket to create and no schema to map.
Prerequisites
- A self-hosted LiteLLM proxy, version v1.108.0 or later, running in database (Postgres) mode.
- A Ternary user with the Tenant Admin role.
Step 1: Create the AI Gateway integration in Ternary
- In Ternary, go to Admin → Integrations, click New Integration, and select AI Gateway.
- Enter an Integration Name, then click Create Integration.
- The AI Gateway connection created dialog shows three values and a ready-to-paste LiteLLM snippet:
- API key (
TERNARY_API_KEY) - Connection ID (
TERNARY_CONNECTION_ID) - Base URL (
TERNARY_BASE_URL)
- API key (
Copy the API key nowThe API key is shown only once and cannot be retrieved later. Copy it, and store it as a secret, before closing the dialog. If you lose it, create a new AI Gateway integration.
Step 2: Configure LiteLLM
Add the ternary callback and the three values from Step 1 to your LiteLLM proxy config, then restart the proxy:
litellm_settings:
callbacks: ["ternary"]
environment_variables:
TERNARY_API_KEY: <API key from Step 1>
TERNARY_CONNECTION_ID: <Connection ID from Step 1>
TERNARY_BASE_URL: <Base URL from Step 1>You can also set these as environment variables on the proxy instead of in environment_variables. Keep the API key in your secret store rather than in source control.
For team attribution, assign a team_id to your LiteLLM virtual keys (see LiteLLM's Virtual Keys & Teams docs). That team arrives in Ternary as the SubAccountId / SubAccountName fields; keys without a team_id still export, but their spend is unattributed.
LiteLLM's own Ternary destination docs cover the remaining options.
How exports work
- Schedule. By default LiteLLM exports once a day, shortly after midnight UTC. The proxy must be running at that time; a missed export is not retried until the next day.
- Whole days, re-sent. Each export sends the current day so far plus the two previous full UTC days. Each day replaces the previous upload for that day, so re-sent days never double-count, and spend LiteLLM records late is picked up on the next export.
- Ingestion. Ternary loads newly uploaded days on its standard schedule (every four hours). Use Request Billing Update to trigger ingestion immediately.
- Privacy. Only cost and usage metadata is sent — spend, token counts, model, provider, and team/key identifiers. Prompt and completion content never leaves your gateway.
Verify the connection
To see data without waiting for the daily export, temporarily set an interval schedule on the proxy, restart it, and check the integration in Ternary after the next ingestion:
environment_variables:
TERNARY_EXPORT_FREQUENCY: interval
TERNARY_EXPORT_INTERVAL_SECONDS: "300"The first export runs one interval after the proxy starts. Remove both settings once you have confirmed data is arriving — the daily default is all LiteLLM's daily spend data needs.
If nothing arrives, check the LiteLLM proxy logs for Ternary export warnings: they name the day that failed and the HTTP status returned. A 401 or 404 means the API key or connection ID is wrong; confirm TERNARY_BASE_URL matches the value shown when you created the integration.
Method 2: Bring Your Own Data (BYOD)
This method uses LiteLLM's generic FOCUS export to write consumption data to object storage you control, which Ternary ingests through Bring Your Own Data (BYOD).
Prerequisites
- A self-hosted LiteLLM proxy, version 1.98.0 or later, running in database (Postgres) mode. Earlier versions do not reliably emit the scheduled bucket export.
- An object storage bucket you control: Google Cloud Storage or Amazon S3 (S3-compatible endpoints are also supported).
- A Ternary user with the Tenant Admin role.
- The Bring Your Own Data feature enabled on your tenant. Contact Ternary support if it is not available.
Step 1: Enable the FOCUS export in LiteLLM
This step happens entirely in LiteLLM. Enable the built-in FOCUS export by adding the focus callback to your LiteLLM config and pointing it at a Google Cloud Storage or Amazon S3 bucket you control (Azure Blob is not yet supported on the LiteLLM side). LiteLLM then writes FOCUS-formatted files to that bucket on a schedule.
Follow LiteLLM's own documentation for the exact configuration — it is the authoritative source and covers all destination, format, and scheduling options:
- FOCUS Export — enabling the
focuscallback and configuring the GCS/S3 destination. - Virtual Keys & Teams — defining teams and assigning a
team_idto keys.
For Ternary, two choices in that configuration matter:
- Destination. Export to a GCS or S3 bucket (S3-compatible endpoints included) that you can grant Ternary read access to in Step 2.
- Team attribution. Assign a
team_idto your LiteLLM virtual keys so the export attributes each row to a team. That team arrives in Ternary as theSubAccountId/SubAccountNamefields; keys without ateam_idstill export, but their spend is unattributed.
Verify the export
After the first scheduled run, confirm that files appear under your prefix, for example:
gs://my-litellm-focus-bucket/focus_exports/date=YYYY-MM-DD/usage_<start>_<end>.csv
Each row is a daily aggregate per team, model, and provider, conforming to the FinOps FOCUS v1.2 specification.
Export scheduleFiles appear on the export frequency you configured in LiteLLM (see their FOCUS Export docs). Allow one interval to pass before expecting the first files.
Step 2: Create the BYOD integration in Ternary
In Ternary, go to Admin → Integrations, click New Integration, and select Bring Your Own. For the full walkthrough and IAM details, see Bring Your Own Data (BYOD). The LiteLLM-specific choices are:
- Data source: Google Cloud Storage or Amazon S3, matching where LiteLLM writes. LiteLLM's FOCUS export currently supports only GCS and S3 (S3-compatible endpoints included) as destinations — Azure Blob Storage is not yet available on the LiteLLM side. If your LiteLLM proxy runs in Azure, write the export to a GCS or S3 bucket, or an S3-compatible endpoint.
- Storage URI: the export prefix — for example
gs://my-litellm-focus-bucket/focus_exports(ors3://…). All files under the prefix are ingested. - Access:
- GCS — grant your tenant's Service Account Email (shown on the Admin page) read access to the bucket (
storage.objects.get,storage.objects.list). - S3 — provide the Role ARN and ensure your tenant's Service Account Unique ID is trusted by that role, exactly as for an AWS bill. See Amazon Web Services.
- GCS — grant your tenant's Service Account Email (shown on the Admin page) read access to the bucket (
- Integration name: cannot contain spaces — use letters, numbers, hyphens, or underscores, for example
LiteLLM_Gateway.
Click Preview to validate the data source before continuing.
Step 3: Map the schema
On the schema assignment step, set the Charge Period Start column to ChargePeriodStart (required), then map the LiteLLM FOCUS columns to Ternary's existing Multi-Cloud fields. Because LiteLLM already emits FOCUS column names, this is a direct, same-name mapping:
| LiteLLM column | Map to (Ternary field) | Represents |
|---|---|---|
ChargePeriodStart | ChargePeriodStart (required) | Usage day |
ChargePeriodEnd | ChargePeriodEnd | Usage day end |
BilledCost | BilledCost (measure, currency) | Estimated cost — used as the allocation weight |
ServiceName | ServiceName | Model group |
ChargeDescription | ChargeDescription | Model |
ProviderName | ProviderName | anthropic, openai, … |
SubAccountId | SubAccountId | Team ID |
SubAccountName | SubAccountName | Team name |
ConsumedQuantity | ConsumedQuantity (measure) | Request count |
Tags | Tags (JSON) | Per-request dimensions: user, API key, org (see below) |
MapTagsfor richer dimensionsLiteLLM's JSON
Tagscolumn carriesteam_id,team_alias,api_key_alias,user_id,user_email,model,model_group, andcustom_llm_provider. Map it as a JSON column to expose each key as its own report dimension — most usefullyuser_idandapi_key_alias, which have no discrete FOCUS column. Ternary automatically flattens the JSON into individual dimensions.Team, model, and provider are also available directly as
SubAccountName,ServiceName, andProviderName— no special handling is needed on either side. LiteLLM's FOCUS export natively emits these standard FOCUS column names, and Ternary maps them by name, so mappingTagsis optional if those columns cover your needs. (The general caution about large JSON tag columns in the BYOD guide doesn't apply here — LiteLLM's tag set is small and bounded.)
Field-name notesIf you choose to map
InvoiceIssuerNameorServiceSubcategory, map them to Ternary'sInvoiceIssuerandServiceSubCategoryrespectively (note the exact spelling/casing).
Review and Submit. Ternary validates access, then ingests on the standard schedule (every four hours). Use Request Billing Update to trigger ingestion immediately.
Using the data in Ternary
Once ingested, LiteLLM data appears as Custom Data in the Reporting Engine and dashboards. You can:
- Break down AI spend by team (
SubAccountName), model (ServiceName/ChargeDescription), and provider (ProviderName). - With the AI Gateway integration, also break down by AI provider and model, and report input, output, cache-read and cache-write tokens. Every row is tagged with the cost source LiteLLM estimate.
- With BYOD, if you mapped
Tags, also break down by user (user_id) and API key (api_key_alias). - Combine AI spend with your other cloud costs in a single report.
Allocating invoiced AI spend
To allocate your real invoiced AI dollars using gateway proportions:
- Ingest your AI provider invoice as its own source — this provides the dollars.
- Ingest LiteLLM gateway data with this integration — this provides the proportions.
- Create a Reallocate Costs billing rule that splits the invoiced AI spend across teams or cost centers, weighted by the gateway data (by team, model, or provider). The result is zero-sum and reconciles to the invoice.
Limitations and notes
- Estimated cost. LiteLLM's
BilledCostis an estimate. Use it as a weighting basis for allocation, not as billed truth. - Daily granularity. Both methods aggregate to one row per day, per team/model/provider.
- Requests, not tokens (BYOD only). With BYOD,
ConsumedQuantityreflects request counts and per-token quantities are not included. The AI Gateway integration includes token counts. - Missed exports (AI Gateway). Each export re-sends only the current day and the two previous full days. If the proxy is down for longer than that, the older days are not re-sent.
- Full rebuild (BYOD only). BYOD re-reads the bucket each ingestion cycle; there is no incremental mode.
For general BYOD behavior, limits, and troubleshooting, see Bring Your Own Data (BYOD).
Updated 9 days ago
