Cost tracking

Routing to the cheapest provider only pays off if you can see what it saved. ai-lcr reports real per-call cost and the savings versus the priciest route — here's how to use it well. Everything below is from 0.3.0+ and is optional; the router works without any of it.

Two ways to observe a call

• onCost(event) — fires once per call with { model, provider, inputTokens, outputTokens, costUsd }. Quick to wire; good for a running total.
• onCall(record) — fires once per settled request with the full failover chain (CallRecord: every attempt, the winner, latency (and streaming time-to-first-token), tokens, cost, and the savings baseline). This is what a dashboard wants.

Prefer onCall for anything real — onCost can't tell you a failover happened.

import { createLCR, formatCallRecord } from "ai-lcr";

const lcr = createLCR({
  autoSort: true,
  models: { /* … */ },
  onCall: (rec) => console.log(formatCallRecord(rec)),
  // ✓ text  tokenmart  840ms (ttft 96ms)  $0.0004  (saved $0.0011)
  // ⚠ text  tokenmart→openrouter  910ms  $0.0004  ⤷ tokenmart 502
});

The headline number: savings = baselineUsd − costUsd

CallRecord.baselineUsd is what the same usage would have cost without routing, and baselineKind (0.6+) says exactly which baseline that is — so a dashboard can qualify the savings number instead of trusting it blindly:

• "last-leg" (text): the last priced provider in the chain — your always-on, list-price fallback. Deliberately not the most expensive leg: prompt caching can make a sticker-cheaper provider cost more on a cache-heavy call, and a max-of-chain baseline would fabricate "savings" on calls the fallback itself served.
• "official" (media): the model maker's first-party API price for the call's actual usage — an 8-second clip is baselined at 8 seconds of the official rate. officialUsd carries the same figure.
• "priciest-route" (media with no official price known): the most expensive route you configured — honest about cross-provider spread, but self-referential, not a market price.

onCall: (rec) => {
  if (rec.baselineUsd !== undefined) {
    metrics.add("ai_spend_usd", rec.costUsd);
    // clamp per call: one mispriced row must not eat the others' savings
    metrics.add("ai_saved_usd", Math.max(0, rec.baselineUsd - rec.costUsd));
  }
},

For text it's set whenever at least one provider in the list carries a cost — so price your providers (the cost field) even if you don't use autoSort, or there's nothing to compare against. Don't want to hand-type prices? Set autoPrice: true and native-maker routes are priced from the bundled MODEL_PRICES table automatically (see How routing works).

Price-table drift: estCostUsd (0.6+)

CallRecord.estCostUsd is what your configured price table predicted the call would cost, on the same usage. When a provider reports its own bill (costUsd came from the API, not an estimate), costUsd − estCostUsd is price-table drift — a stale or mis-entered registry price. A ~100× ratio is almost always a USD-vs-cents unit slip. The router also raises onError when a reported bill is ≥25× off the prediction. Cheapest-first routing is only as good as its price table; watch this number.

Media records additionally carry modality ("image" | "video") and the typed usage behind the bill ({ seconds?, outputs?, megapixels? }), so per-unit economics — $/second of video, $/image — are derivable downstream.

Responsiveness: time to first token

For streaming calls, CallRecord.ttftMs reports time to first token — ms from the winning provider's stream start to its first content token. It's measured against the winner's attempt, so failover overhead (already in latencyMs) doesn't distort it. undefined for non-streaming calls and for calls that failed before producing content.

onCall: (rec) => {
  if (rec.ttftMs != null) metrics.add("ai_ttft_ms", rec.ttftMs);
  // throughput is derivable from the same record:
  const tps = rec.outputTokens / ((rec.latencyMs - rec.ttftMs) / 1000);
},

Pair it with costUsd to compare providers on both price and speed — the cheapest route isn't always the snappiest.

Prompt-cache-aware cost

If you use Anthropic-style prompt caching, cached input tokens bill far cheaper than fresh input. Tell ai-lcr by adding cacheRead (USD per 1M cached input tokens) to a provider's cost:

models: {
  "claude-sonnet-4-6": [
    {
      model: tokenmart("claude-sonnet-4-6"),
      label: "tokenmart",
      cost: { input: 3.0, output: 15.0, cacheRead: 0.3 }, // cached reads at 1/10th
    },
  ],
}

When a call reports usage.inputTokens.cacheRead, those tokens bill at cacheRead instead of input; omit it and they fall back to the full input rate (so adding it can only make the estimate more accurate). CallRecord.cachedInputTokens exposes the cache-hit volume so you can audit why costUsd came in under sticker × tokens. (Accounting only — caching doesn't change routing weights.)

Roll a tool loop up into one cost

An agent doing a multi-step tool loop makes many model calls for one logical request. Stamp the same correlation id on every step via providerOptions.lcr.requestId, and each CallRecord.requestId carries it — so your dashboard can sum the steps into a single request's cost:

import { generateText } from "ai";

const requestId = crypto.randomUUID();
await generateText({
  model: lcr("smart"),
  prompt,
  stopWhen: stepCountIs(10),
  providerOptions: { lcr: { requestId } }, // same id on every step
});
// every CallRecord for this loop now shares rec.requestId → group + sum by it

Don't mistake "free" for "cost unknown"

Some providers serve a call fine but report zero usage tokens. Then costUsd silently reads 0 — which looks free but really means unmeasured. CallRecord.usageMissing flags exactly that case, so credit-metering doesn't under-bill:

onCall: (rec) => {
  if (rec.usageMissing) {
    logger.warn(`no usage from ${rec.winner} on ${rec.model} — cost is unmeasured, not zero`);
  }
},

formatCallRecord renders it as a ⚠no-usage suffix.

Ship records to a dashboard

createHttpSink builds an onCall handler that POSTs each CallRecord as JSON to a collector:

import { createLCR, createHttpSink } from "ai-lcr";
import { after } from "next/server";

const lcr = createLCR({
  models: { /* … */ },
  onCall: createHttpSink({
    url: "https://your-collector.example/ingest",
    headers: { authorization: `Bearer ${process.env.LCR_INGEST_KEY}` },
    project: "my-app",      // merged into each payload as { project, ...record }
    dispatch: after,        // ← see the serverless note below
  }),
});

Serverless gotcha: on Vercel/Lambda an un-awaited POST can be killed when the function returns. Pass dispatch: after (Next.js after from next/server, or any waitUntil-style fn) so the record actually ships. On a long-lived server you can omit it.

The companion dashboard

ai-lcr-dashboard is a self-hostable Next.js + Postgres collector built for exactly these records — point createHttpSink at its /api/ingest and you get saved-vs-spent over time (qualified by baselineKind, clamped per call), per-provider failover health, media $/unit economics, and a price-drift panel that surfaces any model@provider route whose reported bills disagree with your price table by

±20%. One-click Vercel deploy, any Postgres, metadata only (no prompts or outputs stored). The ingest contract is plain CallRecord JSON, so any other log drain works too.