You ask an AI a question you cannot answer on your own. A polished response appears in seconds. It sounds settled, so you move on.
That jump from uncertainty to confidence is worth noticing.
Researchers recently gave people six difficult questions about details in films. The questions were chosen because the AI in the experiment was likely to answer them incorrectly. Participants could give an answer or say they did not know.
People with access to AI advice answered more questions. According to the paper, they were correct about one-third as often as people without AI advice, while their confidence nearly doubled. The fluent response seemed to change whether they felt ready to answer.
This was a narrow experiment, and the limits matter. The paper is a new preprint that has yet to complete peer review. It tested deliberately unreliable AI on obscure film questions with modest incentives and no real-world consequences. It cannot tell us how often the same thing happens with everyday AI use.
The study points to a useful habit.
When an AI answer matters, try these questions:
- Was I about to say “I don’t know” before this appeared?
- What part of this answer can I confirm myself?
- Can I check the key claim in two minutes?
- What happens if the answer is wrong?
The researchers found that rewards for accuracy helped people resist bad advice, although participants still withheld answers less often than people working without AI.
A confident answer can arrive before you have decided how much confidence it deserves. Keep the pause. Use it to decide whether the answer is good enough for the choice in front of you.
Public source
Marcoccia, Quattrociocchi, and Capraro, “AI advice suppresses people's willingness to say ‘I don't know,’ even when the advice is wrong and accuracy is incentivized,” arXiv:2607.13562, submitted July 15, 2026. New preprint; peer review and independent replication remain open.