AI Advice Cuts Accuracy to 9% as Confidence Jumps to 76%, Study Finds
Updated
Updated · TNW · Jul 19
AI Advice Cuts Accuracy to 9% as Confidence Jumps to 76%, Study Finds
2 articles · Updated · TNW · Jul 19
Summary
Accuracy fell to 9% from 27% when participants could consult AI, while confidence climbed to 76% from 30% and willingness to say “I don’t know” collapsed to 3% from 44%.
Researchers designed the test around questions the model usually gets wrong—such as visual details from films—showing people often overrode correct instincts and adopted false AI answers.
Monetary incentives improved performance only modestly: admitting ignorance rose to 8% and accuracy to 16%, still far below the no-AI baselines.
The French and Italian researchers said the findings sharpen concerns about “cognitive surrender,” especially for children, as AI products increasingly deliver confident answers instead of signaling uncertainty.
If AI is designed to never say 'I don't know,' how can we prevent a generation from losing its critical thinking skills?
As AI makes us dangerously overconfident, are we outsourcing the very ability to think for ourselves?
The Confidence Trap: How Overconfident AI Erodes Human Judgment and Critical Thinking
Overview
AI systems are designed to always provide confident answers, even when they are wrong, which leads users to passively accept information and gradually erode their own critical thinking skills. This constant stream of unwavering confidence encourages a state of 'cognitive surrender,' where people stop questioning and become dependent on AI for information. As a result, users lose the ability to recognize uncertainty and develop durable discernment, making them more vulnerable to misinformation. The report highlights how this dynamic undermines public resilience and stresses the need for AI to support, rather than replace, human judgment and critical engagement.