Skip to main content
Every Trace has a cost & latency rollup at the top, and per-step breakdowns beside every timeline entry.

​
The rollup

At the top of every Trace:
  • Total cost — dollar amount, plus token counts by model.
  • Total latency — wall clock start to answer sent.
  • Compute — sum of all model + tool time (excludes waiting).
  • Wait — time spent waiting on external services (APIs, databases, humans in approval queues).
Wall clock = compute + wait + small overhead.

​
Per-step breakdown

Each step in the timeline shows its individual cost and duration. Sorted view: click the cost column header to see the most expensive steps first. Common offenders when cost is high:
  • Long prompts — retrieval returning too many chunks. Reduce top-K in the preset.
  • Expensive model — using a top-tier model when a mid-tier would do. Consider a fallback or a router.
  • Repeated tool calls — Agent looping on the same tool. Fix the tool description or the Agent’s role.
  • Enrichment on the fly — should be pre-computed via a Foundry pipeline, not per-run.

​
Latency waterfall

The waterfall view (click Timing in the top bar) shows latency as parallel bars. Anything not on the critical path can safely be optimized last. Compute-bound steps show as solid bars; wait-bound show as hatched. Wait bars in the middle of the run usually mean serial API calls that could be parallelized in the Agent’s tool config.

​
Comparing runs

Click Compare with… on any Trace and pick another Trace (same Flow, different run). Side-by-side latency and cost view. Useful when:
  • Latency spiked and you want to know which step changed.
  • A new version increased costs — see where.
  • A/B testing two Agents.

​
Cost alerts

Per-Flow cost alerts fire when:
  • A single run exceeds a threshold (e.g., “any run over $1”).
  • Rolling average cost per run exceeds a threshold over N runs.
  • Total daily/weekly/monthly spend exceeds a budget.
Configure in Flow settings → Alerts. Route to Slack, email, or a webhook.

​
Cost per user / per tenant

If your runs carry scope keys, the Insights page can group cost by scope:
  • Cost by user_id — who costs the most.
  • Cost by tenant — which customer is expensive.
  • Cost by feature (via tags on Traces).
Great for pricing decisions, cost attribution, or spotting a runaway integration.

​
Latency budgets

Set a latency budget per Flow. Runs exceeding the budget are flagged in the runs list. Common budgets:
  • Chat: 3 seconds.
  • Background: 30 seconds.
  • Batch: 5 minutes.
Runs over budget aren’t stopped — you’re notified and can decide.

​
Where the money actually goes

Model calls: usually 60-80% of cost. Embeddings and small helper models: 10-25%. Third-party tools with paid APIs: variable. Storage: negligible for most workloads. If you’re surprised where cost lives, the per-step view is where to look.