AI Experts Warn Token Metrics Misfire as OpenAI Eyes Bank Data and Firms Burn $500 Million
Updated
Updated · O'Reilly Media · Jun 5
AI Experts Warn Token Metrics Misfire as OpenAI Eyes Bank Data and Firms Burn $500 Million
3 articles · Updated · O'Reilly Media · Jun 5
Summary
$500 million in Anthropic tokens burned in one month at one company became a cautionary example in a podcast discussion arguing that token counts and code volume are poor measures of AI productivity.
OpenAI’s move to analyze users’ transaction data with financial institutions was framed as a push to infer consumer intent, combining chat histories with spending records to build profiles that could be highly valuable for advertising.
Doug Shannon and Maya Mikhailov said the real professional edge is metacognition—knowing what to offload to AI, what to question, and when human judgment must stay in the loop to avoid “cognitive surrender.”
Amazon’s scrapped AI leaderboard and GitHub Copilot’s shift toward usage-based pricing were cited as signs that companies are starting to curb incentives that reward token consumption while masking technical debt and governance risks.
Forward-deployed engineers alone cannot fix enterprise AI rollouts, they argued, because deployment failures usually stem from missing organizational context, siloed data, legacy systems and regulatory constraints rather than model capability.