[Adobe Stock]
Insights 2024: Attitudes toward AI report—released July 9 and based on survey data gathered from December 2023 to February 2024—81% of respondents worried that GenAI could erode critical thinking skills.
Now, a fresh study out of Microsoft and Carnegie Mellon seems to validate those fears. While it didn’t look at researchers in particular, it did find that knowledge workers (n=319) who lean heavily on large language models often invest less cognitive effort in their work—even when accuracy and sound judgment remain paramount. It also notes that “GenAI shifts the nature of critical thinking toward information verification, response integration, and task stewardship.”
Nevertheless, the study cautions that while prompt refinement can sharpen cognitive skills for some, others slip into a pattern of blind trust. In part, this discrepancy reflects task confidence: those who already feel assured in their own abilities tend to challenge the AI more, whereas those who lack self-confidence are more likely to default to the AI’s outputs without question. This emerging dichotomy underscores the importance of designing GenAI tools that explicitly remind users to verify, contextualize, and improve on machine suggestions. Thus, the antidote to slipping into overreliance on genAI is to continue cultivation of domain-specific critical thinking. Professionals confident in their expertise tend to critically engage with GenAI, while those lacking self-confidence are more prone to passive AI reliance.
The potential for overreliance
As generative AI tools become integrated into knowledge work—assisting with tasks like summarizing research or drafting communications—a subtle shift in cognitive engagement emerges. The Lee et al. study from Microsoft Research and Carnegie Mellon reveals that while GenAI reduces the perceived effort of critical thinking, it simultaneously fosters a potential for over-reliance.
As alluded to earlier, professionals who rely heavily on large language models tend to engage in less cognitive effort, even in high-stakes tasks. “This shift towards passive oversight, potentially contributing to what has been termed “mechanized convergence” (a reduction in output diversity), is observed. The authors caution that overconfidence in AI, especially for routine tasks like grammar checking, can lead to a gradual decline in the practice of critical thinking. This phenomenon echoes Bainbridge’s “ironies of automation,” where automatic systems reduce certain skill sets over time unless deliberate steps are taken to maintain them.
From the paper: The distribution of perceived effort (%) in cognitive activities dips for many participants using a GenAI tool compared to not using one.
3. Critical thinking paradox
Specifically, higher confidence in GenAI is associated with less critical thinking, while higher self-confidence is associated with more critical thinking.
The study also shows that users who have high confidence in GenAI’s capabilities often forego deeper analysis. For example, a sales representative might use GenAI to quickly generate emails—bypassing reflection on messaging or strategy owing to perceived time pressure and faith in the AI’s output. Conversely, individuals with stronger self-confidence in their own expertise remain more vigilant, applying thorough review processes even if it feels more effortful. Some tasks could require inherently more critical thinking owing to a feedback loop impact. A coder who has a clear goal in mind, for instance, might rigorously test and debug AI-generated code snippets, knowing that verification is crucial for ensuring quality and that the software aligns with expectations and context.
The paper doesn’t insinuate that genAI necessarily degrades critical thinking, however. Knowledge workers shift from task execution to what the study terms “task stewardship.” This includes extra effort in verifying information (example: double-checking AI-generated summaries of diabetes management guidelines against hospital protocols) and integrating responses (example: carefully revising AI-generated resumes to match personal experiences or job requirements). This stewardship also involves prompt optimization—asking better questions or clarifying constraints to guide the AI toward more useful outputs.
4. Real-world impact for R&D and knowledge work, write large
When applied to high-impact activities—such as drafting grant proposals, conducting peer reviews, or interpreting experimental data—over-reliance on GenAI poses risks to scientific rigor. A hasty acceptance of AI outputs in scientific papers or hypothesis testing can erode the essential practice of skepticism and detailed analysis.
As knowledge workers in R&D environments grow accustomed to letting AI handle initial drafts or summaries, they risk attenuating their own analytical skills over time. This makes it all the more critical to maintain “sound judgment” in evaluating AI-suggested conclusions or data interpretations.
5. Proposed strategies for dealing with genAI overrelaince
Nudges for verification: The paper suggests building GenAI tools that prompt users to check the AI’s work—citing sources, verifying calculations, or clarifying assumptions.
Human-AI collaboration: By positioning AI as a partner rather than a replacement, organizations can emphasize accountability, metacognition, and domain expertise. This ensures that humans remain active decision-makers rather than passive recipients of AI output.
Training programs: Ongoing education that reinforces the habit of verifying claims and data is crucial. Rather than relying on AI’s first draft, professionals should be encouraged to refine outputs and remain actively engaged with the subject matter.
Ultimately, the authors propose a measured path forward: treat AI as a “challenger” whose outputs, while potentially invaluable, should always be tested against professional judgment and verifiable sources. This mindset ensures that even as AI automates routine steps, human intelligence remains active in the final judgment.
