Understanding Critical Thinking in Generative Artificial Intelligence Use: Development, Validation, and Correlates of the Critical Thinking in AI Use Scale

Generative AI tools are increasingly embedded in everyday work and learning, yet their fluency, opacity, and propensity to hallucinate mean that users must critically evaluate AI outputs rather than accept them at face value.

The present research conceptualises critical thinking in AI use as a dispositional tendency to verify the source and content of AI-generated information, to understand how models work and where they fail, and to reflect on the broader implications of relying on AI.

Across six studies (N = 1341), we developed and validated the 13-item critical thinking in AI use scale and mapped its nomological network.

Study 1 generated and content-validated scale items.

Study 2 supported a three-factor structure (Verification, Motivation, and Reflection).

Studies 3 and 4 confirmed the higher-order model, demonstrated strong factor loadings, internal consistency, sex invariance, convergent and discriminant evidence for validity, and showed that critical thinking in AI use was positively associated with openness, extraversion, positive trait affect, and frequency of AI use.

Study 5 supported the scale's test-retest reliability.

Lastly, Study 6 demonstrated criterion evidence of validity for the scale, with higher critical thinking in AI use scores predicting more frequent and diverse verification strategies, greater veracity judgement accuracy in a novel and naturalistic GPT-powered AI chatbot fact-checking task, and deeper reflection about responsible AI.

The current work clarifies why and how people exercise oversight over generative AI outputs and provides a validated scale and ecologically grounded paradigm to support theory testing, cross-group, and longitudinal research on critical thinking in AI use.