Result: Meta-Prompting as a Solution to Students’ Prompt Engineering Difficulties for an Optimized Use of GenAI LLMs in the Context of Education: A Quasi-Experimental Study using Mistral Model.

The emergence of prompt engineering as a rising field of GenAI has garnered attention with the purpose to resolve ambiguities accompanying its use. In essence, a perfect input prompt performing well on LLM A might not perform well on LLM B as there is a certain disparity in how each model behaves, along with students’ poor prompting competency can make of crafting excellent prompts almost impossible to achieve. To address this problematic issue, the present study sheds light on meta-prompting which can solve students’ countless difficulties encountered with forming appropriate prompts, besides disparities in how different LLMs respond to the same prompt. To this end, the study adopts a within-subjects quasi-experimental design, with a sample involving N=50 undergraduate students of the Higher School of Teachers – Moulay Ismail University. For data analysis, the study uses SPSS version 25 for statistical representation of data, and Python code executed on Google Colab coding environment in which the Wilcoxon Signed-Rank Test for paired samples was conducted. Results demonstrated that there is a strong significant difference between students’ self-crafted prompts and meta-prompts in terms of specificity, comprehensiveness, logical sequence of instructions, and good structure criteria. The direction of the difference is positive suggesting an increase in the overall rating scores between pre-test and post-test results. The present paper has also proven that students’ confidence with prompts significantly increases with the use of meta-prompting technique compared to traditional prompting. Ultimately, the study identifies limitations and offers recommendations orienting future research projects in the field of GenAI and LLMs. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Science & Technology Studies is the property of Al-Kindi Center for Research & Development and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)