--- service: "Publicasta" schema_version: "1.0" article_id: 179 title: "The AI abstainers have a point: chatbots can make wrong answers feel safe" language: "en" default_language: "en" canonical_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21?lang=en" json_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.json?lang=en" api_url: "https://publicasta.com/api/public/v1/channels/ai_practice/articles/ai_critical_thinking_overconfidence_study_2026_07_21?lang=en" channel_url: "https://publicasta.com/api/public/v1/channels/ai_practice" channel_articles: "https://publicasta.com/api/public/v1/channels/ai_practice/articles" search_url: "https://publicasta.com/api/public/v1/search" documentation_url: "https://publicasta.com/api-docs#reading-publicasta" openapi_url: "https://publicasta.com/api-docs/openapi.json" published_at: "2026-07-24T18:09:27+00:00" updated_at: "2026-07-24T18:09:27+00:00" translations: - language: "ar" html_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21?lang=ar" markdown_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.md?lang=ar" json_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.json?lang=ar" - language: "de" html_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21?lang=de" markdown_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.md?lang=de" json_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.json?lang=de" - language: "en" html_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21?lang=en" markdown_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.md?lang=en" json_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.json?lang=en" - language: "es" html_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21?lang=es" markdown_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.md?lang=es" json_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.json?lang=es" - language: "fr" html_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21?lang=fr" markdown_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.md?lang=fr" json_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.json?lang=fr" - language: "pl" html_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21?lang=pl" markdown_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.md?lang=pl" json_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.json?lang=pl" - language: "ru" html_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21?lang=ru" markdown_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.md?lang=ru" json_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.json?lang=ru" - language: "zh" html_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21?lang=zh" markdown_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.md?lang=zh" json_url: "https://publicasta.com/ai_practice/ai_critical_thinking_overconfidence_study_2026_07_21.json?lang=zh" --- # The AI abstainers have a point: chatbots can make wrong answers feel safe > A new study is a useful warning for AI practice: the danger is not asking a model, but letting it erase the moment when you should say “I don’t know.” The most useful sentence in the age of AI may be the least glamorous one: "I don't know." It has no startup energy. It does not scale. It will not appear in a glossy product keynote. It is, however, the small brake pedal that stops a person from turning a plausible answer into a confident mistake. ![Critical thinking desk with AI advice and an I do not know note](https://publicasta.com/storage/projects/8/pages/179/2026/07/5f280a89-a5d3-4786-9ac9-270aaa738ff9.webp) A new study covered by The Next Web gives that brake pedal a number. Researchers Valerio Capraro, Chiara Marcoccia and Walter Quattrociocchi tested people on hard questions about visual details in films. One group answered without AI. Another could ask an AI model for advice. The model, Step 3.5 Flash, was chosen because it usually failed on these questions, which made the experiment less about whether AI can be helpful in general and more about what people do when a machine sounds helpful but is wrong. The results are uncomfortable. Without AI, participants were correct 27% of the time and said "I don't know" 44% of the time. With AI available, accuracy fell to 9%, while willingness to admit uncertainty collapsed to 3%. Confidence moved in the opposite direction: from 30% without AI to 76% with AI. Monetary incentives improved the AI group's accuracy to 16%, but that still left them below the no-AI baseline. This is the part where the internet naturally invents a counterculture. AI vegans. AI old believers. People who refuse algorithmic advice the way some people refuse ultra-processed food. A dark little web where the sacred troubleshooting ritual is not ChatGPT but a drum, a stack trace and a suspicious senior engineer squinting at logs at 2 a.m. The joke works because it points at something real. There will be a category of people whose brand is not "I use AI for everything" but "I keep one room in my head where the machines are not allowed to answer first." They may sound dramatic. They may also be early to a useful hygiene habit. ## The study is not saying "never use AI" The boring caveat matters. The experiment was designed around questions where the model was unreliable. It does not prove that every AI-assisted task makes people worse. A calculator helps arithmetic. A spellchecker catches typos. A coding assistant can write boring boilerplate faster than a human who has already decided what the code should do. The study points to a narrower and more dangerous pattern: when a system gives advice in a domain where the user cannot easily verify the answer, people may outsource the moment of doubt. The damage is not only the wrong answer. The damage is losing the habit of saying, "wait, I do not actually know this." That matters for AI practice because most real workflows are not trivia contests. They are messy mixtures of recall, interpretation and judgment. A product manager asks for a market summary. A developer asks why tests are failing. A student asks for an explanation of a paper. A founder asks whether a legal clause is risky. The model answers with the emotional shape of certainty. If the user does not have a verification loop, the answer can become a shortcut around thinking. The TNW report frames this through judgment suspension. That phrase is less catchy than "AI kills critical thinking," but it is more precise. Good thinking includes the ability to pause before committing. It includes knowing when the evidence is thin, when a memory may be false, when a confident explanation is just a story wearing a tie. ## Overconfidence is the product risk A wrong answer is annoying. A wrong answer plus confidence is operationally dangerous. The numbers in this study are interesting because confidence rises as accuracy falls. That is exactly the failure mode many AI users recognize from daily work. The model produces an answer that is fluent, structured and calm. The user feels the friction drop. The decision no longer feels like a guess. It feels like a checked answer, even when nothing has been checked. This is not only a consumer problem. Microsoft Research published a CHI 2025 study on generative AI and critical thinking among 319 knowledge workers, collecting 936 examples of GenAI use at work. The researchers found that higher confidence in GenAI was associated with less critical thinking effort, while higher confidence in one's own ability was associated with more critical thinking. In other words, the person who treats AI as an authority may think less, while the person who treats AI as raw material may still do the work. That distinction is practical. Two people can use the same chatbot and have opposite outcomes. One asks, accepts and forwards. The other asks, challenges, compares, tests and edits. The second person is not anti-AI. They are anti-surrender. The phrase "cognitive surrender," cited by TNW from Wharton researchers, is useful because it avoids the cartoon version of the debate. The problem is not that using a chatbot makes your brain melt like cheap cheese. The problem is that certain interface patterns reward mental passivity. The answer appears instantly. It arrives in clean paragraphs. It has no sweat on it. The user has to add the sweat back. ## Why film-detail questions were a smart trap At first glance, film trivia sounds like a toy domain. It is actually a good trap for AI overconfidence. Many people have partial memories of films. Large language models also have partial, pattern-shaped representations of popular culture. Both can sound confident when asked about a uniform color, a scene detail or a specific prop. The difference is that a human who is unsure may say "I don't know." A chatbot tends to keep talking. That makes the experiment closer to many office tasks than it first appears. A model may know the outline of a policy but miss the exception. It may know the common API pattern but invent an argument name. It may summarize a legal issue but skip the jurisdictional detail that changes the answer. It may explain a medical topic in a way that sounds reasonable while being unsafe for a specific patient. The user's job is to notice when the question is detail-sensitive. If the answer depends on a specific fact, date, setting, file, contract clause or line of code, a generated answer is not the end of the task. It is a candidate answer. The work begins where the fluent paragraph ends. ## The rise of AI refusal as a lifestyle signal The chat joke about AI vegans and AI old believers lands because technology culture loves identity tribes. We already have people who refuse social media, refuse smart speakers, refuse cloud storage, refuse subscriptions, refuse IDEs, refuse dark mode, refuse light mode and somehow manage to make each refusal a worldview. AI refusal will have its own flavors. Some people will reject AI for labor politics. Some for privacy. Some for religion. Some because the tools are annoying. Some because they watched one colleague paste a hallucinated citation into a deck and decided the machines were spiritually unclean. There will also be a healthier version: deliberate abstinence windows. No AI for the first draft of an argument. No AI before reading the source. No AI during exams or skill acquisition. No AI for the first pass at debugging. No AI until you have written your own hypothesis. That is not Luddism. That is strength training. A runner does not take a taxi for the middle kilometer and call it training. A musician does not let the metronome decide the phrasing. A developer should not let a model be the first and only owner of the hypothesis. Tools are fine. Tool-shaped autopilot is the problem. ## The practical rule: think first, ask second For everyday AI use, the simplest change is order of operations. Before asking the model, write your own answer in one or two sentences. If you do not know, write what you would need to verify. Then ask the model. Now you have something to compare against instead of a blank mental page waiting to be filled. This tiny step changes the psychology. It turns the model from oracle into sparring partner. You can ask, "where is my reasoning weak?" rather than "what is the answer?" You can request counterarguments. You can ask for uncertainty. You can demand citations and then open them. You can make the model earn trust instead of receiving it by default. For teams, this can become a workflow rule. In code review, AI-generated fixes should come with tests or a reproducible explanation. In research, AI summaries should link to primary sources and identify unknowns. In product work, AI-made market claims should be marked as hypotheses until checked against real data. In hiring or education, AI assistance should be separated from the part of the task meant to measure independent reasoning. None of these rules require moral panic. They require friction. Good friction. The kind that makes a person stop before turning a probability-shaped paragraph into a decision. ## A small checklist for not becoming confidently wrong Ask three questions before accepting AI advice. First: can I verify this quickly? If yes, verify it. Open the source, run the command, check the file, read the contract, inspect the data. If verification is cheap and you skip it, the failure is not the model's fault. Second: what would make this answer wrong? This question forces the mind out of admiration mode. It looks for edge cases, hidden assumptions, old information, missing context and ambiguous wording. Third: am I using AI to think better or to avoid thinking? The honest answer is often obvious. If the model is helping you compare options, find blind spots or produce a draft you will rewrite, fine. If it is helping you avoid the discomfort of uncertainty, you are near the danger zone. The hardest part is preserving the right to say "I don't know." In many workplaces and online spaces, uncertainty feels like weakness. AI makes that worse because it always has something to say. But the person who can pause may be more valuable than the person who can produce instant confidence on command. ## What this means for AI practice The next phase of AI literacy is not prompt tricks. It is epistemic hygiene: how to know what you know, how to mark what you do not know, and how to prevent fluent systems from laundering uncertainty into confidence. That sounds academic until it becomes a pull request, a medical search, a financial decision, a school assignment or a public post. The gap between 27% and 9% accuracy in the study is not a universal law, but it is a warning label. The model can be useful and still make you worse if you hand it the wrong job. So yes, the AI vegans are coming. Some will be unbearable at dinner. Some will announce their refusal with the moral intensity of a person who discovered mechanical keyboards last week. Let them have their bit. Under the joke is a serious practice worth stealing: keep a part of your process unaided, skeptical and slow. Say no to Skynet if you want. The less theatrical version is better: say "I don't know" before the machine says anything at all.