Treffer: Designing Computer Vision Support for Science Practical Work: A Qualitative Investigation into the Noticing Practices and Support Preferences of Science Teachers
1573-1839
Weitere Informationen
With teachers continuing to report challenges in classroom management and difficulties in implementing scientific inquiry, the current manner in which science practical work is conducted in schools suggests the need for added teacher support. In this regard, we can leverage computer vision to provide instructional support by relieving teachers of the need to carry out mundane observations and perform basic interpretations of student activity. However, to our knowledge, little is known about the noticing practices of teachers during practical work, and the support preferences of such a computer vision system have not been studied before. To this end, we recruited 17 science educators with different teaching expertise for a qualitative investigation into the noticing practices and support preferences of science teachers. Results revealed seven major categories and 36 minor categories of student activity that teachers typically observe, which enabled us to derive observation routines that can emulate quality teacher noticing for computer vision input. Our obtained list of observation categories represents a first-of-its-kind list which takes into account concrete noticing practices of science teachers and remains applicable across all types of practical tasks. From participants' ranking of computer vision models, we further understood the type of computer vision output that teachers prefer for instructional support. To our best of knowledge, no prior research has examined the connection between teacher noticing and computer vision in such detail. Using these findings, we can then pursue the development of computer vision for instructional support in science practical work in an informed manner, taking into account the realities of science laboratories and proclivities of science teachers.
As Provided
AN0179144866;4n601oct.24;2024Aug23.06:30;v2.2.500
Designing Computer Vision Support for Science Practical Work: A Qualitative Investigation into the Noticing Practices and Support Preferences of Science Teachers
With teachers continuing to report challenges in classroom management and difficulties in implementing scientific inquiry, the current manner in which science practical work is conducted in schools suggests the need for added teacher support. In this regard, we can leverage computer vision to provide instructional support by relieving teachers of the need to carry out mundane observations and perform basic interpretations of student activity. However, to our knowledge, little is known about the noticing practices of teachers during practical work, and the support preferences of such a computer vision system have not been studied before. To this end, we recruited 17 science educators with different teaching expertise for a qualitative investigation into the noticing practices and support preferences of science teachers. Results revealed seven major categories and 36 minor categories of student activity that teachers typically observe, which enabled us to derive observation routines that can emulate quality teacher noticing for computer vision input. Our obtained list of observation categories represents a first-of-its-kind list which takes into account concrete noticing practices of science teachers and remains applicable across all types of practical tasks. From participants' ranking of computer vision models, we further understood the type of computer vision output that teachers prefer for instructional support. To our best of knowledge, no prior research has examined the connection between teacher noticing and computer vision in such detail. Using these findings, we can then pursue the development of computer vision for instructional support in science practical work in an informed manner, taking into account the realities of science laboratories and proclivities of science teachers.
Keywords: Science education; Practical work; Teacher noticing; Computer vision; Instructional support; Artificial intelligence
Copyright comment Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Introduction
Despite its purported benefits, a few researchers have called into question the existing value of science practical work (Hodson, [8]; Wellington, [19]), and a number of teachers have continued to report less than ideal effects of conducting practical lessons for students (England et al., [3]). In particular, teachers need to know how to design laboratory sessions (Zhu et al., [16]), and challenges with classroom management and difficulties in implementing scientific inquiry approaches are commonly highlighted by teachers (Gallagher & Tobin, [6]; Welch, [15]). When these issues go unaddressed, they lead to further problems such as teacher disengagement or a cookbook recipe style of conducting science experiments (Pine et al., [11]). In a study on the effectiveness of practical work as a teaching and learning method, Abrahams and Millar ([1]) discover that "practical work was generally effective in getting students to do what is intended with physical objects, but much less effective in getting them to use the intended scientific ideas to guide their actions and reflect upon the data they collect" (p. 1945). The researchers then conclude that the key to conducting an effective science experiment lies in the quality of teacher instruction. In another study, Hodson ([9]) observes that there is a tendency to encourage teachers to "use practical work in pursuit of any and every goal of science education" (p. 756), leading to teachers' inability to realize the full potential of practical work. Hodson ([9]) further argues that "significant change will only occur if teachers' concerns are recognized, their problems acknowledged, and their viewpoints taken seriously" (p. 759). In other words, while researchers and practitioners can agree upon the value of practical work in science education, the manner in which practical work is conducted currently suggests the need for added teacher support.
Although technologies such as applications in mobile devices have been used to support practical work (e.g., planning an experiment or sharing and communicating findings), it still requires teachers to monitor students' engagement with the deployed applications closely (Falloon, [4]). In this regard, the use of computer vision can aid with alleviating some of the instructional issues encountered by science teachers. For instance, computer vision can conduct continuous observation of students' execution, highlighting incorrect procedural steps or safety violations. However, given present-day technology, computer vision can only serve to reduce teachers' observational load. The attainment of higher pedagogical goals still falls within the responsibility of teachers. Regardless, computer vision can provide instructional support for teachers by liberating them of the need to carry out mundane observation tasks. When this happens, teachers can focus on getting students "to use the intended scientific ideas to guide their actions and reflect upon the data they collect" as advocated by Abrahams and Millar ([1]), and not have to deal with lower pedagogical goals such as student monitoring and safety management as commented by Hodson ([9]).
With this in mind, the overall aim of this research is to examine the design of a computer vision system that can provide instructional support for teachers during students' conduct of practical work. Currently, little is known about the noticing practices of teachers during practical work for input into computer vision, and the support preferences of the proposed computer vision system have not been studied before. Hence, this work plans to uncover knowledge in these areas by conducting a qualitative investigation into the noticing practices and support preferences of science teachers. For the purpose of this paper, the terms "practical work," "practical task," "science experiments," and "experiments" are used interchangeably to refer to teaching scenarios whereby a set of practical instructions is given to students for them to manipulate apparatus and gather data, so as to investigate a scientific phenomenon.
Research Questions
As a starting point, we seek to obtain a concrete list of observation tasks for our proposed computer vision system. To accomplish this, we ask "(1) What are the noticing practices of science teachers during practical work?" By investigating teacher noticing practices, we hope to derive observation routines that can emulate quality teacher noticing for computer vision input.
Next, we strive to bring educators into the research conversation about the use of computer vision in practical work. To this end, we ask "(2) What are the types of computer vision models that science teachers prefer for instructional support?" Findings from this investigation could also assist future researchers in developing research agendas that are more aligned with teacher preferences.
Teacher Noticing in Science Practical Work
The study of teacher noticing as a construct originates from researching professional vision as a concept. When probing the discursive practices of professionals, Goodwin ([7]) identifies three professional vision components—(1) coding schemes, (2) highlighting, and (3) articulation of graphic representations. In coding schemes, professionals utilize the knowledge they possess internally to process information they receive externally. This is akin to surveying information through the lens of professional vision. In highlighting, professionals focus attention on pertinent details while passing over inconsequential information. This is akin to applying filters of professional vision in situations to single out relevant information. In articulation of graphic representations, professionals employ symbols and materials to express thoughts and communicate with each other. This is akin to exchanging interpretative information using the language of professional vision. With Goodwin's ([7]) framework, teacher noticing can be regarded as the use of professional vision by teachers to code, highlight, and articulate critical teaching and learning aspects that exist in various pedagogical situations.
In education research, Sherin ([13]) dissects teacher noticing into selective attention and knowledge-based reasoning. Selective attention concedes that teachers do not pay attention to every single detail within classrooms. On the contrary, they are only drawn towards classroom events that hold significant pedagogical value (i.e., applying filters of professional vision). Knowledge-based reasoning recognizes that teachers do not operate in a vacuum. They are actively utilizing pedagogical content knowledge (Shulman, [14]) to derive interpretations of significant classroom events (i.e., surveying through the lens of professional vision). With Sherin's ([13]) framework, we can anatomize teacher noticing practices into observations (i.e., what teachers pay attention to selectively) and interpretations (i.e., what teachers conclude from the knowledge-based reasoning of observations).
In practical work, Millar et al. ([17]) distinguish between physical and mental actions taken by students during learning. As pointed out by Tiberghien ([18]), the fundamental purpose of practical work is to help students derive connections between the domain of observables and the domain of ideas. To accomplish this, students have to exert effort in the domain of observables with physical actions, and the domain of ideas with mental actions. Therefore, to evaluate the effectiveness of a practical task, Millar et al. ([17]) conceived of a model that takes into account student actions within domains of observables and ideas (see Fig. 1). Boxes B and C lie within the domain of observables, starting with the practical task that teachers designed for students and ending with physical actions taken by students to complete the experiment. Boxes A and D lie within the domain of ideas, starting with the teachers' intended learning objectives and ending with mental actions taken by students to attain learning. If students' physical actions meet the practical task requirements, then the practical work can be considered effective at the first level. If students' mental actions achieve the learning objectives, then the practical work can be considered effective at the second level. As uncovered by Abrahams and Millar ([1]), the current form of practical work was "generally effective in getting students to do what is intended with physical objects, but much less effective in getting them to use the intended scientific ideas to guide their actions and reflect upon the data they collect" (p. 1945). In this respect, the use of computer vision can support teachers within the domain of observables, relieving them of the need to constantly fixate on whether students are performing the designed practical task as expected (i.e., effectiveness at the first level). With the freed bandwidth received from computer vision support, teachers can then easily focus on the second level of effectiveness for practical work.
Graph: Fig. 1 Millar et al. ([17]) model of the process of design and evaluation of a practical task
Methods
This study adopts a qualitative stance to interpret participants' ideas after watching videos of science practical work in session and examining shared computer vision models. A qualitative description provides an in-depth understanding of teachers' sense-making processes as they facilitate practical sessions and teachers' support preferences when they receive computer vision outputs. To address the stated research questions methodically, we separate this study into two phases. In Phase I, we investigate teacher noticing practices. In Phase II, we uncover the types of computer vision models that science teachers favor for deployment.
Participants
Both study phases involved the same group of participants. In total, 17 science educators (seven females, ten males) from the Singapore education system were recruited for our purpose. The educators were selected based on their science laboratory experience (ranging from less than 2 years to more than 20 years) and science teaching subjects (covering biology, chemistry, physics, and lower secondary science). More crucially, we sought to obtain a mix of experienced educators (who have demonstrated proficiency in their craft and are considered pedagogical leaders) and beginning educators (who may not possess a lot of pedagogical experience but are thoughtful in their craft). This purposeful selection of astute educators with varied experiences and subjects was needed for the crystallization of teacher intelligence into artificial intelligence (i.e., replicate teachers' noticing practices with computer vision), and the codification of proclivities for an instructional tool (i.e., reproduce teachers' support preferences for computer vision). All participants interacted with the primary researcher online via Zoom, with every meeting recorded for later analysis. Figure 2 summarizes the participation experience for each subject and key details for each phase are outlined in the next two subsections.
Graph: Fig. 2 Summary of participant experience for each subject
Phase I
At the beginning of Phase I, a unique link was provided to each participant to complete a demographic survey online. This demographic survey collected participants' background information such as their years of science laboratory experience and main teaching subjects in science.
Thereafter, all participants were tasked to watch videos of a single student carrying out a particular experiment (see Table 1). Before each video was played, the primary researcher gave an explanation of the practical work context, providing participants with information such as practical task instructions and student academic level. Each video lasted 20–35 min and every participant was invited to watch two distinct videos in isolation. The choice of videos for each participant was determined by their teaching subjects in science, with every participant reviewing at least one video in their subject area. It should be noted that a number of participants have more than one teaching subject in science and may review up to two videos of their teaching subjects.
Table 1 Description of student videos
As videos played, participants were instructed to voice their thoughts whenever they noticed instances of significant student activity. The primary researcher would then pause the video whenever participants spoke and invite them to frame their noticing into observations and interpretations (i.e., discern pure observations from processed interpretations as in Sherin's ([13]) framework). For instance, a participant may state an observation as "Student observed to be re-reading the instructions for experiment several times", and the corresponding interpretation might be "Student appears to be uncertain of the next experiment step". Overall, this approach sought to mimic the in-the-moment noticing condition that teachers face in laboratories while taking care not to overload participants' cognitive faculties by requesting them to only focus on a single student.
After reviewing the videos, the primary researcher probed participants further by asking them to come up with suitable categories for their stated observations. In addition, the primary researcher prompted participants to think of other potential categories of observations that might be useful to science teachers and asked them to prioritize observations for feedback to science teachers. Information from these added questions would eventually lay the groundwork for considering observation tasks for our proposed computer vision system.
Phase II
During Phase II, the primary researcher provided participants with background information on computer vision and the proposed instructional tool. This is necessary as not all participants are familiar with current developments in computer vision or its potential value as an instructional tool. At this juncture, we also sought to assure participants that the underlying motivation for deploying computer vision in practical work is for instructional support (and not replacement) of science teachers.
To provide participants with a more tangible idea of computer vision's capabilities, the primary researcher shared details of five different computer vision models—action recognition, object detection, person re-identification, gaze estimation, and emotion detection. These models were chosen based on their potential affordances for science education. Thereafter, participants were asked to rank and evaluate each model for their perceived pedagogical value (for instructional support in practical work).
Analysis
The proposed computer vision system hopes to relieve teachers of mundane observation tasks and perform basic interpretations of student activity for science teachers. Within Sherin's ([13]) framework, this translates to using computer vision to pay selective attention to students' conduct of experiments and processing student observations with rudimentary knowledge-based reasoning. To accomplish this and address the first research question, we analyzed data collected in Phase I to obtain a concrete list of observation tasks that can emulate quality teacher noticing for computer vision input. Specifically, we identified categories of observations through emic coding of participants' verbalized noticing when reviewing student videos. In carrying out thematic analysis emically, we explicitly favored educators' "insider" perspective over our (researchers') perspective as "outsiders" (Parker-Katz & Bay, [10]). This methodological stance is taken in recognition that individuals with professional expertise can better identify items of value when presented with familiar situations.
In unraveling preferences for types of information from computer vision (and to address the second research question), we analyzed data collected in Phase II to understand types of computer vision outputs that teachers favor. We ascertained the type of computer vision output that is of pedagogical value by examining participants' comments on the various computer vision models that were shared. Using participants' order of rank for each model's perceived pedagogical value, we calculated an average ranking score for each computer vision model, with lower scores indicating higher perceived pedagogical value (i.e., first-ranked has score 1, last-ranked has score 5).
Results
RQ1: What Are the Noticing Practices of Science Teachers During Practical Work?
From participants' verbalization of noticed student activity in practical work, we uncover seven major categories and 36 minor categories of student activity that teachers typically observe (refer to Table 2 for details). These observation categories can be further classified into general observations that remain applicable across all types of practical tasks, and task-specific observations that depend on the nature of the practical task. For instance, off-task student behaviors and help-seeking social interactions are commonly witnessed in all practical work (and can be considered as general observations), while identification of critical and non-critical procedural steps is contingent on the practical task (and can be considered as task-specific observations). There exist certain observation categories that may include both general and task-specific observations. For example, the removal of unneeded apparatus from the work area as a housekeeping practice can be considered a general observation whereas the wiping of the work area with a dry cloth to remove fluids as a housekeeping practice can only be considered a task-specific observation (since not every experiment involves fluids).
Table 2 Categories of noticing practices in practical work
During the derivation of observation categories, we became aware of some participant struggle in framing observations and interpretations. When probed for the observation that led to the interpretation that the student appears frustrated with circuit setup, one participant remarked, "I think it is from her body language, her face is covered by a mask so I cannot really tell any facial expressions, I'm not sure but her body language gave me a sense that she is frustrated". In this case, the participant had trouble articulating the subtle body language that resulted in his interpretation. Another participant was more successful in this endeavor when she stated, "I could tell her frustration from her constant pointing at the circuit when talking to her peer, and also from her agitated hand movements". This also illustrates that humans do not instinctively think in terms of observations and interpretations when noticing events. Moreover, the primary researcher noted a tendency for participants to simply verbalize interpretations without stating any observations. A considerable amount of prompting was needed to get participants to pause and reflect upon the observations that led to their interpretations. However, the separation of pure observations from processed interpretations is necessary because current computer vision technology can only deal with very specific observation tasks. It is certainly possible to train a computer vision model for the detection of agitated hand movements, but it is nearly impossible (with current technology) to develop a computer vision model for the recognition of subtle body language.
Additionally, we noticed that it is possible for participants to arrive at different interpretations for the same observation. When the same student is observed to be re-reading practical work instructions, some participants inferred that the student is uncertain about the next procedural steps while others surmised that the student is being meticulous in carrying out procedural steps. Admittedly, with video as the sole source of information, a range of interpretations exist for a number of observations. Consequently, this presents a challenge for our system when extracting valid interpretations of student activity from computer vision outputs. To tackle this challenge, we can first isolate interpretations that hold strong links to observations for immediate processing (e.g., student raising a hand is a definite indicator of a student needing assistance). Thereafter, we can pinpoint observations that possess a narrow range of interpretations for highlighting, including a note on the level of speculation. Wherever appropriate, we should also include some pedagogical understanding of the experiment (i.e., perform rudimentary knowledge-based reasoning) when determining interpretations (e.g., the appearance of bubbles during heating indicates that acid is sufficiently warmed). For observations with a wide range of interpretations or necessitating the use of higher-level knowledge-based reasoning, we can rely on human teachers for better interpretation.
RQ2: What Are the Types of Computer Vision Models That Science Teachers Prefer for Instructio...
Using participants' ranking and comments on shared computer vision models, we found that information from action recognition and object detection models were perceived to be of higher pedagogical value. On the other hand, gaze estimation, emotion detection, and person re-identification models were deemed to only provide secondary pedagogical information.
Action recognition stood out as the most pedagogically valued model with an average ranking score of 1.41. Participants view science practical work as a performance-based task with students having to perform a series of actions for task completion. In that case, action recognition can detect both the procedural steps taken by students and the techniques exhibited during practical work. This information can then assist teachers in determining students' level of proficiency, which is a common blindspot since practical work assessment in many education systems today relies primarily on written submissions. Furthermore, participants agree that safety is an important aspect of practical work. If action recognition could be deployed to detect hazardous actions or safety violations, then it would address a key observational concern of teachers and contribute greatly in terms of instructional support for monitoring large groups of students. Finally, participants expressed more confidence in the outputs of action recognition (as compared to other models) since displayed actions are more objective in nature, making them suitable for computer processing to derive pedagogically valuable interpretations.
Object detection is the next most pedagogically valued model with an average ranking score of 2.18. For most participants, information from object detection is complementary to that of action recognition as students are expected to manipulate scientific apparatus during their experiments. With object detection, teachers can tell if students are selecting the appropriate apparatus for use. When combined with action detection, teachers can ascertain if students are manipulating apparatus with the correct technique. In addition to object identity, object detection models generally provide information about object position too. With known apparatus positioning, teachers can also gather if students have set up apparatus in the right configuration. With regard to safety, more preference is indicated for the use of action recognition models to address concerns. However, some participants convey a desire in deploying object detection models to check whether the apparatus has been placed in a precarious position or whether students are setting up the apparatus in a safe manner. Regardless, the use of object detection appears inseparable from action recognition in participants' minds.
Gaze estimation did not receive as much demand as compared to previous models, obtaining an average ranking score of 3.29. Participants remarked that it would be challenging for teachers to arrive at pedagogical decisions based on gaze information alone. Yet, a few participants managed to raise examples of when gaze information might be of pedagogical use. For example, the direction of student gaze might indicate where students are hoping to receive help from. This could be from the teacher's whiteboard, the practical worksheet, or the experimental setup of another student. The wandering of student gaze might signal off-task behavior, or uncertainty in the next experimental step, or moments when students are in need of assistance. The angle of student gaze can help teachers verify the absence of parallax error during measurement reading. Lastly, the tenuous link between gaze and attention can also inform teachers if students are paying attention to the right details (e.g., monitoring the mixing of liquids in the beaker during titration). Nonetheless, these raised examples refer to very specific instances during a practical work session and require purposeful processing design for detection, which might explain the lower pedagogical demand for gaze estimation models.
Emotion detection did not gain much acceptance from participants either, obtaining an average ranking score of 3.47. Participants voiced concerns about the soundness of examining emotion data for pedagogical decisions as most student emotions are subtle and fleeting. Extreme displays of emotions are rare and would have been noticed by teachers instantly (i.e., reduced need for outsourcing to computer vision). Furthermore, different students possess different levels of expressiveness for underlying emotions. They also react differently to similar situations depending on their state of mind. Such diversity in expressions and reactions increases the subjectivity (and difficulty) in deriving pedagogical decisions based on emotion data. Of the few participants who value emotion data, detecting student confusion and frustration ranks high on their priority. These participants are interested in moments when students are struggling with the experiment so that timely teacher intervention can be provided and common challenges in the experiment can be identified.
Person re-identification did not earn much favor with participants as well, obtaining an average ranking score of 3.76. Participants questioned the need for student identification in a laboratory setting where students work individually in their own space. As such, student identities can be inferred from the positions of cameras that were used to record videos for computer vision input. In other words, person re-identification is only useful if camera frames contain more than one individual. Since, in the present setting, students are expected to conduct experiments alone, participants see little value in processing videos through person re-identification models. Moreover, the identification of students holds little bearing on the identification of student issues. If student issues can be identified by computer vision and aggregated at a class level, then teachers can continue to plan for class-level intervention regardless of student identities. Nonetheless, some participants argued for the use of person re-identification models to determine student social interactions. Knowledge in this area can inform teachers who students normally seek help from. Also, if the laboratory setting is such that students are expected to collaborate on the given practical task, then person re-identification models would certainly be useful in analyzing student social interactions for collaborative states.
Besides the shared computer vision models, there are other models that participants opined to be of pedagogical value and can provide instructional support in other respects. As teachers walk around science laboratories during experiments, they habitually check for written recordings of measurements in the students' reports. A text recognition model can thus help teachers process student writings to check for issues in data recording. In another scenario, color detection models can help teachers communicate color observations to students who possess difficulty in discerning colors or verify recorded color observation matches the actual color obtained during the experiment. The suggestions of these computer vision models by participants (beyond the shared ones) prove that there is much potential in deploying computer vision in practical work to provide instructional support for science teachers.
Discussion
From our engagement with educators, it became clear that their noticed events in practical work stemmed from underlying pedagogical concerns. Therefore, to derive pedagogical value from computer vision, we can adopt a top-down approach in which we start with teachers' pedagogical concerns to obtain a list of required observation tasks before selecting computer vision models with suitable affordances (see Fig. 3). This proposed process for deriving pedagogical value from computer vision is not restricted to science practical work and is undoubtedly applicable to education in general.
Graph: Fig. 3 Process for deriving pedagogical value from computer vision
The critical component in this process lies in the listing of observation tasks. With a list of observation tasks, we can then begin to design computer vision systems that address teachers' pedagogical concerns and provide instructional support. As accomplished in Phase I of this study, Table 2 gives a list of potential observation tasks for our envisioned system. However, Table 2's observation tasks do not possess equal pedagogical value for science teachers. Moreover, if resources are limited, it is essential that we only select observation tasks with high pedagogical value for outsourcing to computer vision.
To this end, we discover that observations of safety, critical procedural steps, and technique proficiency were consistently regarded as having high pedagogical value by participants. Participants view safety as pedagogically valuable because of the desire for real-time intervention to prevent mishaps or for student debrief to prevent future mishaps. Participants value knowledge about students' execution of critical procedural steps or proficiency in techniques because these represent areas of improvement for students to better their performance of experiments. This finding is in line with Abrahams et al.'s ([2]) report that helping students to set up equipment and operate it correctly as intended is part of effective teacher facilitation for practical skills. Furthermore, participants opined that, in a usual laboratory setting, teachers experience limits in their ability to continuously monitor every student and these observation tasks are ideal for outsourcing to computer vision because they are mundane yet noteworthy.
The next level of priority for participants includes student behaviors that indicate prolonged struggle in practical work (e.g., uncertainty and frustration), and help-seeking social interactions. Participants value information in these areas because the time available in science laboratories is limited. If students spend too much time struggling or are constantly seeking help, they may miss out on planned learning objectives. In other words, participants would like this information for timely teacher intervention. In this respect, Pun and Cheung's ([12]) earlier research elucidates that science practicals can be challenging to students because of the inherent difficulty in connecting objects (e.g., apparatus setup) with scientific ideas. When viewed through Millar et al.'s ([17]) model, this translates as a challenge for teachers to accomplish the second level of effectiveness for practical work (see Fig. 1). Therefore, if computer vision can support teachers by identifying prolonged student struggle (i.e., relieve them of the need to fixate on the domain of observables), they can better attend to helping students connect objects with scientific ideas. For help-seeking behavior involving peers, participants shared differing opinions as to whether this is crucial for teachers to know. On one hand, frequent outreach to peers might indicate student struggle, which warrants immediate teacher attention. On the other hand, students ought to be allowed space for experimental exploration with peers, so that learning can be enriched for all. Regardless, participants agree that computer vision can pursue the collection of this information for teachers to make their own pedagogical judgments as to when is an appropriate moment for intervention.
Participants deprioritize information on non-critical procedural steps, students talking with peers, and students' body language. For non-critical procedural steps, these do not affect experimental outcomes and have little implications for learning objectives. For students talking with peers, such social interactions are to be expected and would not be of concern (unless it turns into a classroom management issue). For students' body language, there is little universal meaning behind displayed body language, so it is of low utility.
With regard to other instructional supports suggested by participants, the use of computer vision to process students' writings to check for issues in data recording could also be explored. Using computer vision to assess multiple choice questions has been reported by prior studies (Fisteus et al., [5]) and the advancement in current optical recognition technology would render this as highly feasible.
In sum, with these known observation priorities, we can now work to maximize computer vision's pedagogical value in practical work by only selecting observation tasks with high pedagogical value for outsourcing to computer vision.
Study Limitations
During the course of research, we experienced unevenness in participants' response in both phases. Some participants were more forthcoming with their ideas while others were more reserved. When we checked in with participants who remained silent for an extended period of time, a number of them commented that they had minor thoughts which they did not feel were worthy of mention. In other words, even though we encouraged participants to verbalize every noticed event and critique each computer vision model, certain amounts of mental screening still exist. This inadvertently causes the omission of details deemed minute by participants (but may still hold pertinence) in our data collection process.
Our target sample for this study only included educators as we aimed to bring them into the research conversation about the use of computer vision in practical work. However, educators lack technical expertise in computer vision, and can only rely on our limited sharing to envision the deployment of computer vision in science laboratories. As such, our collected data may not represent the full potential of using computer vision as an instructional support in practical work.
Finally, qualitative studies are commonly confronted with the issue of generalizability and our study is no exception. Our sample is limited in its number and only consists of educators from the Singapore education system. Therefore, the obtained participants' opinions on this subject may not be representative of all science educators. Moreover, the types of student videos shown were limited, consisting of only one experiment for each science subject. Hence, there may be noticing practices that were overlooked due to the restricted selection of videos.
Conclusion
The overall aim of this qualitative study was to examine the design of a computer vision system that can provide instructional support for teachers during students' conduct of practical work.
Results revealed seven major categories and 36 minor categories of student activity that teachers typically observe, which enabled us to derive observation routines that can emulate quality teacher noticing for computer vision input. These findings contribute to the small but growing research on teacher noticing in science practical work. Our obtained list of observation categories represents a first-of-its-kind list which takes into account concrete noticing practices of science teachers and remains applicable across all types of practical tasks. With this list, it is now possible for the research community to create computer vision models that can relieve teachers of mundane (but necessary) observation tasks. From participants' ranking of computer vision models, we further understood the type of computer vision output that teachers prefer for instructional support. These findings not only inform which computer vision models are perceived to be of higher pedagogical value by teachers, but also provide the research community with a prioritized list of models for construction. To our best of knowledge, no prior research has examined the connection between teacher noticing and computer vision in such detail. Lastly, our discussion put forth a process for deriving pedagogical value from computer vision, starting with the important step of identifying teachers' pedagogical concerns. This ensures that technological tools are not hastily applied to classrooms without due consideration of teacher needs (i.e., avoid pitfall of Maslow's hammer).
Future work would include the expansion of this study in its current form to improve generalizability and the testing of a computer vision system constructed from this study's findings to evaluate the value and effectiveness of instructional support for science teachers.
It is our hope that, using these findings, we can then pursue the development of computer vision for instructional support in science practical work in an informed manner, taking into account the realities of science laboratories and proclivities of science teachers.
Author Contribution
The sole author (Edwin Chng) contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Edwin Chng. The first and subsequent drafts of the manuscript were written by Edwin Chng. Edwin Chng read and approved the final manuscript.
Funding
No funding was received to assist with the preparation of this manuscript.
Availability of Data and Materials
The data that support the findings of this study are available from Harvard University and Nanyang Technological University, but restrictions apply to the availability of these data, which were used under license for the current study and so are not publicly available. The data are, however, available from the authors upon reasonable request and with the permission of Harvard University and Nanyang Technological University.
Declarations
Ethical Approval
The survey, interviews, and methodology for this study were approved by Harvard University-Area Committee on the Use of Human Subjects (IRB22-1470) and Nanyang Technological University Institutional Review Board (IRB-2022–991).
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Consent to Participate
Informed consent was obtained from all individual participants included in the study.
Consent for Publication
Not applicable.
Competing Interests
The author declares no competing interests.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
1 Abrahams I, Millar R. Does practical work really work? A study of the effectiveness of practical work as a teaching and learning method in school science. International Journal of Science Education. 2008; 30; 14: 1945-1969. 10.1080/09500690701749305
2 Abrahams I, Reiss MJ, Sharpe R. The impact of the "Getting Practical: Improving Practical Work in Science" continuing professional development programme on teachers' ideas and practice in science practical work. Research in Science & Technological Education. 2014; 32; 3: 263-280. 10.1080/02635143.2014.931841
3 England BJ, Brigati JR, Schussler EE. Student anxiety in introductory biology classrooms: Perceptions about active learning and persistence in the major. PLoS ONE. 2017; 12; 8. 10.1371/journal.pone.0182506
4 Falloon G. Mobile devices and apps as scaffolds to science learning in the primary classroom. Journal of Science Education and Technology. 2017; 26: 613-628. 10.1007/s10956-017-9702-4
5 Fisteus JA, Pardo A, García NF. Grading multiple choice exams with low-cost and portable computer-vision techniques. Journal of Science Education and Technology. 2013; 22; 4: 560-571. 10.1007/s10956-012-9414-8
6 Gallagher JJ, Tobin K. Teacher management and student engagement in high school science. Science Education. 1987; 71; 4: 535-555. 10.1002/sce.3730710406
7 Goodwin C. Professional vision. American Anthropologist. 1994; 96; 3: 606-633. 10.1525/aa.1994.96.3.02a00100
8 Hodson D. Practical work in science: Time for a reappraisal. Studies in Science Education. 1991; 19: 175-184. 10.1080/03057269108559998
9 Hodson D. Practical work in school science: Exploring some directions for change. International Journal of Science Education. 1996; 18; 7: 755-760. 10.1080/0950069960180702
Parker-Katz M, Bay M. Conceptualizing mentor knowledge: Learning from the insiders. Teaching and Teacher Education. 2008; 24; 5: 1259-1269. 10.1016/j.tate.2007.05.006
Pine J, Aschbacher P, Roth E, Jones M, McPhee C, Martin C, Phelps S, Kyle T, Foley B. Fifth graders' science inquiry abilities: A comparative study of students in hands-on and textbook curricula. Journal of Research in Science Teaching: THe Official Journal of the National Association for Research in Science Teaching. 2006; 43; 5: 467-484. 10.1002/tea.20140
Pun JKH, Cheung KKC. Meaning making in collaborative practical work: A case study of multimodal challenges in a year 10 chemistry classroom. Research in Science & Technological Education. 2023; 41; 1: 271-288. 10.1080/02635143.2021.1895101
Sherin MGoldman R, Pea R, Barron B, Derry SJ. The development of teachers' professional vision in video clubs. Video research in the learning sciences. 2007; Erlbaum: 383-396
Shulman L. Knowledge and teaching: Foundations of the new reform. Harvard Educational Review. 1987; 57; 1: 1-23. 10.17763/haer.57.1.j463w79r56455411
Welch WW. The role of inquiry in science education: Analysis and recommendations. Science Education. 1981; 65; 1: 33-50. 10.1002/sce.3730650106
Zhu L, Sun D, Luo M, Liu W, Xue S. Investigating pre-service science teachers' design performance in laboratory class: The inquiry-based design thinking approach. Journal of Science Education and Technology. 2024; 33: 30-44. 10.1007/s10956-023-10050-3
Millar, R, Le Maréchal, J. F, & Tiberghien, A. (1999). "Mapping" the domain-varieties of practical work. In Practical work in science education: Recent research studies (pp. 33–59). University of Roskilde Press.
Tiberghien, A. (2000). Designing teaching situations in the secondary school. Improving science education: The contribution of research, 27–47.
Wellington, J. (1998). Practical work in science. Time for a reappraisal. In J. Wellington (Ed.), Practical work in school science: Which way now? (pp. 3–15). London: Routledge.
By Edwin Chng
Reported by Author