JetBrains Finds Caveman Prompting Cuts AI Tokens 8.5%, Not 65%
Updated
Updated · InfoWorld · Jul 7
JetBrains Finds Caveman Prompting Cuts AI Tokens 8.5%, Not 65%
3 articles · Updated · InfoWorld · Jul 7
Summary
JetBrains’ benchmark across 86 software-engineering tasks found the Caveman prompting style trimmed Claude Code output tokens by about 8.5%, far below the 65% savings promoted by supporters.
An initial 10-task sample showed roughly 30% savings, but the effect shrank as the workload broadened, suggesting terse replies save relatively little in more representative coding sessions.
JetBrains said most agentic coding tokens are spent reading files, reasoning, calling tools and generating code, limiting how much cost can be cut by stripping conversational padding alone.
One dependency-audit task pushed a Caveman run into Claude Code’s long-context pricing tier, making the full benchmark costlier even though per-task costs were generally lower; JetBrains said the outlier reflected the workload, not the prompt style.
The test found no detectable hit to task success, code quality or execution time, reinforcing JetBrains’ view that prompt-engineering cost claims should be validated on production workloads.