Survey of 319 Workers Links Greater Confidence in AI to Less Self-Reported Critical Thinking
Carnegie Mellon University and Microsoft Research surveyed 319 knowledge workers. Their mental effort shifted away from doing the work themselves and toward inspecting what the AI produced.
Researchers at Carnegie Mellon University and Microsoft Research surveyed knowledge workers who use generative AI on the job. In a paper published in the proceedings of the CHI conference in April 2025, they reported that the more confident people were that AI would handle a task well, the less likely they were to say they had engaged in critical thinking.
Does greater confidence in AI mean less critical thinking?
The study surveyed 319 people who said they use generative AI at work at least once a week. The researchers analyzed 936 real-world examples of use that participants supplied. For each example, they asked whether the person had engaged in critical thinking and how using AI changed the mental effort the task required. Rather than testing participants’ abilities directly, the design asked them to reflect on their own experience.
Participants reported engaging in critical thinking in 555 of the 936 cases. The more confident they were that AI would do a particular task well, the fewer such reports there were. Conversely, the more confident they were in their own ability to do the task, or to evaluate the AI’s output, the more often they reported critical thinking. The two kinds of confidence pointed in opposite directions.
Where the thinking went
What people spent their mental effort on also changed. The researchers described a shift in the focus of thinking: away from gathering information and carrying out the work, and toward verifying and editing what the AI had generated. For people who use AI at work, the pressing question becomes what to check and what to fix once the output lands in front of them.
The study looked at real work experiences across a range of occupations. The researchers noted that earlier work had largely focused on educational settings or narrowly scoped tasks. Here, they examined what kinds of thinking people felt they were doing across varied work scenarios, and how that experience related to their confidence.
Does this mean thinking skills declined?
What the survey captured is participants’ own sense of how their mental effort changed. The researchers themselves acknowledged that participants may have conflated a task simply becoming easier with a reduction in the effort devoted to critical thinking. Whether using AI actually erodes thinking skills, and how things change with long-term use, was not addressed in this study.
The scope is also limited. The survey was conducted only in English, among people fluent in English. Whether the same relationships hold in workplaces that operate in Korean remains to be confirmed.
The value of this research lies in examining where people direct their thinking when they work with AI. Notably, confidence in one’s ability to evaluate AI output went hand in hand with reports of critical thinking. The question going forward is whether those self-reports line up with actual verification ability.