AI Agents Need Loop-Aware Observability to Explain 40x Cost Spikes

3 articles · Updated · O'Reilly Media · Jul 23

A proposed observability substrate would record each agent run as a turn-by-turn loop trace, letting teams explain why one run cost 40 times the median instead of only seeing total tokens and a bill.
The design shifts monitoring below the agent—at model gateways, tool proxies, API gateways, sandboxes and policy engines—because agents cannot be trusted as their own system of record for cost, authority or blocked actions.
At minimum, the substrate captures run_id, turn_id, token counts, tool events, policy decisions, identity context and stop reasons in a fleet-queryable store, with metadata by default and payload capture only by exception.
That same evidence layer is meant to solve three governance problems at once: cost attribution, delegation and access control, and runaway-action prevention such as repeated tool calls, loop-bound hits or oversized 10 MB-plus results.
The proposal argues existing trace viewers and billing dashboards show single runs or aggregate usage, but governance requires querying thousands of runs to detect patterns like context growth above 5x before invoices, audits or incidents surface them.