ChatGPT inflation forecasts trail Cleveland Fed model in new study
Updated
Updated · MarketWatch · May 8
ChatGPT inflation forecasts trail Cleveland Fed model in new study
4 articles · Updated · MarketWatch · May 8
Researchers in “ChatMacro” found ChatGPT’s error rate was up to 12 times higher than the Federal Reserve Bank of Cleveland’s inflation-expectations model.
The authors said proper out-of-sample testing is crucial because AI may use forward-looking information despite prompts restricting it to earlier public data, creating hindsight bias.
The findings challenge investor enthusiasm for AI macro forecasting; the Cleveland Fed model currently projects US inflation averaging 2.4% annually over the next decade.
If ChatGPT is 'useless' for forecasting, why are central banks developing their own AI to predict inflation?
Is the AI boom, promised to be disinflationary, actually driving up inflation through its massive energy demands?
Why ChatGPT Fails at Inflation Forecasting: A 7x Error Gap Compared to Specialized Econometric Models
Overview
Research by the Federal Reserve Bank of San Francisco found that starting in May 2024, ChatGPT-4 Turbo's inflation forecasts became rigid and unresponsive, failing to adapt to new economic data. This led to forecast errors more than seven times higher than those of the Cleveland Fed's specialized inflation nowcast model, which achieves accuracy through real-time updates and mixed-frequency data integration. The deficiencies in ChatGPT stem from its training on data beyond the cutoff date, ineffective prompt restrictions, and a fundamental mismatch between its pattern recognition design and the demands of econometric forecasting. These issues highlight significant risks in relying on large language models for economic decision-making, while hybrid AI-econometric approaches offer promising paths forward.