Result: GenAI-Powered Text Personalization: Natural Language Processing Validation of Adaptation Capabilities

The authors conducted two experiments to assess the alignment between Generative AI (GenAI) text personalization and hypothetical readers’ profiles. In Experiment 1, four LLMs (i.e., Claude 3.5 Sonnet, Llama, Gemini Pro 1.5, and ChatGPT 4) were prompted to tailor 10 science texts (i.e., biology, chemistry, and physics) to accommodate four different profiles varying in knowledge, reading skills, and learning goals. Natural Language Processing (NLP) was leveraged to evaluate the GenAI-adapted texts using an array of linguistic and semantic features empirically associated with text readability. NLP analyses revealed variations in the degree to which the LLMs successfully adjusted linguistic features to suit reader profiles. Most notably, NLP highlighted inconsistent alignment between potential reader abilities and text complexity. The results pointed toward the need to augment the AI prompts using personification, chain-of-thought, and documents regarding text comprehension, text readability, and individual differences (i.e., leveraging RAG). The resulting text modifications in Experiment 2 were better aligned with readers’ profiles. Augmented prompts resulted in LLM modifications with more appropriate cohesion features tailored to high- and low-knowledge readers for optimal comprehension. This study demonstrates how LLMs can be prompted to modify text and uniquely demonstrates the application of NLP to evaluate theory-driven content personalization using GenAI. NLP offers an efficient, real-time solution to validate personalized content across multiple domains and contexts.