Documentation

LiteLLM AI Gateway

Learn how to bring LiteLLM gateway consumption data into Ternary using Bring Your Own Data (BYOD) to allocate AI and LLM spend by team, model, and provider.

The Ternary LiteLLM integration brings AI/LLM consumption data from a self-hosted LiteLLM proxy into Ternary, so you can analyze AI spend by team, model, and provider alongside your other cloud costs — and allocate your invoiced AI spend across teams and cost centers.

AI provider invoices (Anthropic, OpenAI, and others) arrive as opaque line items with no team or project attribution. Your LiteLLM gateway, by contrast, sees every request — priced and tagged with team, key, and model. This integration uses LiteLLM's built-in FOCUS export to write that consumption data to object storage, which Ternary ingests through Bring Your Own Data (BYOD).

📘

How allocation works

The invoice is the source of truth for dollars; the gateway supplies the proportions. LiteLLM's per-request cost is an estimate, so it is used to weight how invoiced AI dollars are split — never reported as billed truth. See Using the data below.

Prerequisites

  • A self-hosted LiteLLM proxy, version 1.98.0 or later, running in database (Postgres) mode. Earlier versions do not reliably emit the scheduled bucket export.
  • An object storage bucket you control: Google Cloud Storage or Amazon S3 (S3-compatible endpoints are also supported).
  • A Ternary user with the Tenant Admin role.
  • The Bring Your Own Data feature enabled on your tenant. Contact Ternary support if it is not available.

Step 1: Enable the FOCUS export in LiteLLM

This step happens entirely in LiteLLM. Enable the built-in FOCUS export by adding the focus callback to your LiteLLM config and pointing it at a Google Cloud Storage or Amazon S3 bucket you control (Azure Blob is not yet supported on the LiteLLM side). LiteLLM then writes FOCUS-formatted files to that bucket on a schedule.

Follow LiteLLM's own documentation for the exact configuration — it is the authoritative source and covers all destination, format, and scheduling options:

For Ternary, two choices in that configuration matter:

  • Destination. Export to a GCS or S3 bucket (S3-compatible endpoints included) that you can grant Ternary read access to in Step 2.
  • Team attribution. Assign a team_id to your LiteLLM virtual keys so the export attributes each row to a team. That team arrives in Ternary as the SubAccountId / SubAccountName fields; keys without a team_id still export, but their spend is unattributed.

Verify the export

After the first scheduled run, confirm that files appear under your prefix, for example:

gs://my-litellm-focus-bucket/focus_exports/date=YYYY-MM-DD/usage_<start>_<end>.csv

Each row is a daily aggregate per team, model, and provider, conforming to the FinOps FOCUS v1.2 specification.

📘

Export schedule

Files appear on the export frequency you configured in LiteLLM (see their FOCUS Export docs). Allow one interval to pass before expecting the first files.

Step 2: Create the BYOD integration in Ternary

In Ternary, go to Admin → Integrations, click New Integration, and select Bring Your Own. For the full walkthrough and IAM details, see Bring Your Own Data (BYOD). The LiteLLM-specific choices are:

  • Data source: Google Cloud Storage or Amazon S3, matching where LiteLLM writes. LiteLLM's FOCUS export currently supports only GCS and S3 (S3-compatible endpoints included) as destinations — Azure Blob Storage is not yet available on the LiteLLM side. If your LiteLLM proxy runs in Azure, write the export to a GCS or S3 bucket, or an S3-compatible endpoint.
  • Storage URI: the export prefix — for example gs://my-litellm-focus-bucket/focus_exports (or s3://…). All files under the prefix are ingested.
  • Access:
    • GCS — grant your tenant's Service Account Email (shown on the Admin page) read access to the bucket (storage.objects.get, storage.objects.list).
    • S3 — provide the Role ARN and ensure your tenant's Service Account Unique ID is trusted by that role, exactly as for an AWS bill. See Amazon Web Services.
  • Integration name: cannot contain spaces — use letters, numbers, hyphens, or underscores, for example LiteLLM_Gateway.

Click Preview to validate the data source before continuing.

Step 3: Map the schema

On the schema assignment step, set the Charge Period Start column to ChargePeriodStart (required), then map the LiteLLM FOCUS columns to Ternary's existing Multi-Cloud fields. Because LiteLLM already emits FOCUS column names, this is a direct, same-name mapping:

LiteLLM columnMap to (Ternary field)Represents
ChargePeriodStartChargePeriodStart (required)Usage day
ChargePeriodEndChargePeriodEndUsage day end
BilledCostBilledCost (measure, currency)Estimated cost — used as the allocation weight
ServiceNameServiceNameModel group
ChargeDescriptionChargeDescriptionModel
ProviderNameProviderNameanthropic, openai, …
SubAccountIdSubAccountIdTeam ID
SubAccountNameSubAccountNameTeam name
ConsumedQuantityConsumedQuantity (measure)Request count
TagsTags (JSON)Per-request dimensions: user, API key, org (see below)
📘

Map Tags for richer dimensions

LiteLLM's JSON Tags column carries team_id, team_alias, api_key_alias, user_id, user_email, model, model_group, and custom_llm_provider. Map it as a JSON column to expose each key as its own report dimension — most usefully user_id and api_key_alias, which have no discrete FOCUS column. Ternary automatically flattens the JSON into individual dimensions.

Team, model, and provider are also available directly as SubAccountName, ServiceName, and ProviderName — no special handling is needed on either side. LiteLLM's FOCUS export natively emits these standard FOCUS column names, and Ternary maps them by name, so mapping Tags is optional if those columns cover your needs. (The general caution about large JSON tag columns in the BYOD guide doesn't apply here — LiteLLM's tag set is small and bounded.)

📘

Field-name notes

If you choose to map InvoiceIssuerName or ServiceSubcategory, map them to Ternary's InvoiceIssuer and ServiceSubCategory respectively (note the exact spelling/casing).

Review and Submit. Ternary validates access, then ingests on the standard schedule (every four hours). Use Request Billing Update to trigger ingestion immediately.

Using the data in Ternary

Once ingested, LiteLLM data appears as Custom Data in the Reporting Engine and dashboards. You can:

  • Break down AI spend by team (SubAccountName), model (ServiceName / ChargeDescription), and provider (ProviderName).
  • If you mapped Tags, also break down by user (user_id) and API key (api_key_alias).
  • Combine AI spend with your other cloud costs in a single report.

Allocating invoiced AI spend

To allocate your real invoiced AI dollars using gateway proportions:

  1. Ingest your AI provider invoice as its own source — this provides the dollars.
  2. Ingest LiteLLM gateway data with this integration — this provides the proportions.
  3. Create a Reallocate Costs billing rule that splits the invoiced AI spend across teams or cost centers, weighted by the gateway data (by team, model, or provider). The result is zero-sum and reconciles to the invoice.

Limitations and notes

  • Estimated cost. LiteLLM's BilledCost is an estimate. Use it as a weighting basis for allocation, not as billed truth.
  • Daily granularity. The export aggregates to one row per day, per team/model/provider.
  • Requests, not tokens. This is a limitation of LiteLLM's FOCUS export, not of Ternary: ConsumedQuantity reflects request counts, and per-token quantities are not included in the exported dataset. If LiteLLM adds token-level quantities in a future release, they can be mapped like any other column.
  • Full rebuild. BYOD re-reads the bucket each ingestion cycle; there is no incremental mode.

For general BYOD behavior, limits, and troubleshooting, see Bring Your Own Data (BYOD).

Updated 1 day ago

Did this page help you?