Treffer: Challenges and feasibility of multimodal LLMs in ER diagram evaluation.

This study investigates the capabilities of four multimodal large language models (MMLLMs), namely GPT-4o, Claude-3 Sonnet, Grok-4, and Gemini-2.5-pro, in assessing Entity-Relationship (ER) diagrams created by students. These diagrams are widely used in database design education, yet their evaluation is time-consuming and often subjective. Automated assessment may reduce instructor workload and provide timely, formative feedback. We developed a structured rubric to evaluate entities, attributes, relationships, and cardinalities, and conducted a controlled experiment with forty student diagrams created in Chen notation and Crow's Foot notation Models were evaluated under four conditions that combined two types of input context, case description based and solution comparison, with two prompting strategies, with or without Chain of Thought(CoT) reasoning. The results of the structural analysis showed that the models extracted the main components of the diagrams reasonably well, although cardinalities were the most difficult to interpret. For rubric-based scoring, alignment with human grading improved when CoT reasoning was used with Chen notation, although its effect in Crow's Foot notation was mixed. These findings indicate that notation style, input context, and reasoning play important roles in automated evaluation, and that multimodal models can offer scalable support for formative assessment in database education. [ABSTRACT FROM AUTHOR]

Copyright of Cogent Education is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)