Fostering critical thinking skills in students through prompt engineering training

Abstract

The development of generative artificial intelligence (GenAI) and its integration into the industrial sector is influencing the range of skills in demand in the labor market. To prepare competitive professionals, higher education systems need to develop students’ skills in interacting with large language models (LLMs) as well as soft skills, such as critical thinking, which enable them to adapt to changing requirements. Prompt engineering, as the process of creating and modifying queries to LLMs, is becoming an important tool in education. However, existing research on the potential of prompt engineering for critical thinking development is primarily theoretical. The aim of this empirical study was to identify opportunities for developing specific aspects of critical thinking through prompt engineering training. The target sample included 93 engineering students who collaboratively completed tasks in the discipline “Legal regulation in the field of communications” using generative AI tools. Thirty-eight student dialogues with LLMs were analyzed using qualitative content analysis. Eight interaction patterns were identified, some of which were associated with the use of prompt engineering techniques. The most common pattern was the addition of information from lectures or other sources, indicating students’ efforts to refine and improve LLM responses. The data analysis revealed the manifestation of critical thinking aspects, such as analysis, comparison, and synthesis of information, in student prompts. A comparison of the results from two coding stages demonstrated a connection between three of the eight patterns and the aforementioned critical thinking aspects. The study suggests that prompt engineering training can become a promising tool for developing critical thinking in students, provided certain educational conditions are met. Additionally, the results indicate the potential of using LLM dialogue analysis to assess student skills.