Treffer: Computerised provider order entry and residency education in an academic medical centre
Centre for Patient Safety, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
Wilson Centre for Research in Education, Faculty of Medicine, University of Toronto, Toronto. Ontario, Canada
Department of Medicine, University Health Network, University of Toronto, Toronto, Ontario, Canada
Centre for Innovation in Complex Care, University Health Network, University of Toronto, Toronto, Ontario, Canada
CC BY 4.0
Sauf mention contraire ci-dessus, le contenu de cette notice bibliographique peut être utilisé dans le cadre d’une licence CC BY 4.0 Inist-CNRS / Unless otherwise stated above, the content of this bibliographic record may be used under a CC BY 4.0 licence by Inist-CNRS / A menos que se haya señalado antes, el contenido de este registro bibliográfico puede ser utilizado al amparo de una licencia CC BY 4.0 Inist-CNRS
FRANCIS
Weitere Informationen
CONTEXT Many academic medical centres (AMCs) have introduced institutional policies, changed processes of care and implemented new technologies to improve health care quality. The impact of such changes on medical education has received little attention. We examine the impact of computerised provider order entry (CPOE) on the educational experiences of medical trainees who work and train in AMCs. METHODS We conducted semi-structured interviews of postgraduate trainees and attending physicians in internal medicine at five AMCs (two with CPOE, three without CPOE). Trainees routinely rotate from CPOE to non-CPOE AMCs, whereas some attending physicians work at both types of AMC and are therefore well positioned to reflect on differences between CPOE and non-CPOE learning environments. Data collection and analysis used grounded theory methods. We sampled purposively until we achieved theoretical saturation. RESULTS Our study included 11 residents and six attending physicians. Computerised provider order entry had both positive and negative impacts on five aspects of postgraduate training: (i) learning (better for medication interactions and availability of learning resources; worse for learning medication doses); (ii) teaching (more medication information available to enhance case discussions; fewer face-to-face teaching opportunities) ; (iii) feedback (improved ability to observe medication ordering behaviours to inform feedback ; less provision of direct feedback); (iv) clinical supervision (facilitates efficient and safe supervision from a distance; may impede trainee independence), and (v) trainee assessment (increased opportunity to assess clinical decision-making and organisational skills). CONCLUSIONS We identify five key educational themes that are positively and negatively impacted by CPOE. These themes form a conceptual framework that could be applied to define the educational impact of other health care quality and patient safety practices. This will help educators to identify educational opportunities and protect the safety of the training experience of residents in AMCs.
AN0077754771;esf01aug.12;2019Jun04.08:19;v2.2.500
Computerised provider order entry and residency education in an academic medical centre.
Medical Education 2012: 46: 795–806 Context Many academic medical centres (AMCs) have introduced institutional policies, changed processes of care and implemented new technologies to improve health care quality. The impact of such changes on medical education has received little attention. We examine the impact of computerised provider order entry (CPOE) on the educational experiences of medical trainees who work and train in AMCs. Methods We conducted semi‐structured interviews of postgraduate trainees and attending physicians in internal medicine at five AMCs (two with CPOE, three without CPOE). Trainees routinely rotate from CPOE to non‐CPOE AMCs, whereas some attending physicians work at both types of AMC and are therefore well positioned to reflect on differences between CPOE and non‐CPOE learning environments. Data collection and analysis used grounded theory methods. We sampled purposively until we achieved theoretical saturation. Results Our study included 11 residents and six attending physicians. Computerised provider order entry had both positive and negative impacts on five aspects of postgraduate training: (i) learning (better for medication interactions and availability of learning resources; worse for learning medication doses); (ii) teaching (more medication information available to enhance case discussions; fewer face‐to‐face teaching opportunities); (iii) feedback (improved ability to observe medication ordering behaviours to inform feedback; less provision of direct feedback); (iv) clinical supervision (facilitates efficient and safe supervision from a distance; may impede trainee independence), and (v) trainee assessment (increased opportunity to assess clinical decision‐making and organisational skills). Conclusions We identify five key educational themes that are positively and negatively impacted by CPOE. These themes form a conceptual framework that could be applied to define the educational impact of other health care quality and patient safety practices. This will help educators to identify educational opportunities and protect the safety of the training experience of residents in AMCs.
Discuss ideas arising from this article at 'discuss'
Computerised provider order entry (CPOE) systems are regarded as having the potential to reduce medication errors.[1] This has led organisations such as the Institute of Medicine[2] and Joint Commission[3] to recommend their widespread implementation. The reductions in medication errors may translate into improved patient outcomes and costs, but the data are contradictory.[[4]] Notwithstanding the potential benefits, there are emerging concerns regarding unintended negative consequences associated with these systems,[[6]] including new work, changes to workflow,[9] altered communication patterns,[10] creation of new errors[11] and overdependence on technology.[12]
Trainees, including students and residents, play an integral role in the delivery of care in academic medical centres (AMCs). Although these systemwide initiatives within AMCs primarily aim to enhance health care quality and safety, they may have both positive and negative impacts on trainees' educational experiences. Yet the educational impact of quality improvement (QI) initiatives such as CPOE has received little attention, unlike other types of unintended consequence. The few studies that have attempted to address this issue in the context of CPOE have yielded mixed and inconclusive results.[[12]] Some studies suggest that trainees perceive medication ordering using CPOE systems as 'cookbook' medicine,[[12]] although others do not.[17] One study found that trainees learned how to write orders better as a result of CPOE,[16] whereas another found no difference.[15]
As part of an ongoing programme of research intended to improve understanding of the intersection between health care quality and medical education in general, we chose to examine the impact of the use of CPOE, a prominent patient safety practice, on the educational experience of medical trainees who work in AMCs.
Methods
Our research approach in this study was based in constructivist grounded theory, a qualitative research methodology that is intended to facilitate the development of a theoretical framework to describe a social phenomenon that is derived primarily from empirical data.[[18]] We conducted our study during March to September 2009. The Research Ethics Board of the University Health Network, University of Toronto (Toronto, ON, Canada) approved this study and was satisfied with the safeguards in place to mitigate against inevitable power dynamics and to protect the participants' anonymity.
Educational setting and context
Postgraduate internal medicine trainees rotate through the five teaching hospitals affiliated with the Faculty of Medicine, University of Toronto, two of which have implemented the same CPOE system. The general internal medicine (GIM) service is structured similarly at all five teaching hospitals. Each hospital has four doctor teams, each with one attending physician (similar to a consultant in the UK), one senior resident (similar to a registrar in the UK), two or three junior residents (similar to foundation doctors in the UK), and three or four medical students. Many GIM services in North American AMCs adopt a similar doctor team structure.
During the first postgraduate year of training (PGY1), residents typically spend 4 months on the GIM service at one of the five teaching hospitals. As junior residents on the team, they provide direct patient care and are largely responsible for day‐to‐day medication ordering for admitted patients. During PGY2, residents move to a different hospital and spend another 4 months as senior residents on the GIM service. Senior residents are primarily responsible for supervising junior residents and medical students; they take much less part in direct patient care and therefore less part in direct medication order entry. Medical students also provide direct care for patients and can order medications, but these orders must be countersigned by a resident or attending physician. Attending physicians provide the highest level of oversight for all residents (including the senior resident), and typically perform very little direct medication ordering.
In non‐CPOE hospitals, a medication order is either written directly in the patient chart or is given verbally over the telephone to the patient's nurse. In a CPOE hospital, medication orders are entered directly into any computer connected to the hospital's internal network. As computers are located throughout the hospital, verbal orders are rarely used. Countersigning is performed electronically in the CPOE hospitals, where orders entered by a medical student appear in a supervising resident's medication ordering queue and must be approved before the medication can be dispensed and administered.
Medication ordering practices
We focused our attention on three contexts of medication ordering and associated educational activities during our data collection and analysis to establish the educational impact of CPOE.
• 1
• 2
• 3
Study participants
No trainee in our programme spends both of his or her first 2 years in a CPOE hospital, although some will spend both their first 2 years in non‐CPOE hospitals. Therefore, we included only trainees who had trained at teaching hospitals both with and without implemented CPOE systems. As most attending staff are primarily affiliated with one teaching hospital, we included only those attending physicians who had worked at CPOE hospitals before and after the implementation of CPOE (or attending physicians who happened to work at both CPOE and non‐CPOE hospitals).
For residents, we purposively sampled to include a balance of: (i) residents at various training levels; (ii) residents who had attended medical school at institutions with or without established CPOE systems, and (iii) male and female residents. For attending staff, we purposively sampled individuals whom we believed would represent different points of view based on our prior knowledge of them. We then used a confirming/disconfirming snowball sampling strategy and asked interviewed residents and staff to recommend other potential study subjects who might: (i) have an experience or viewpoint similar to their own, and (ii) have an experience or viewpoint divergent from their own.
Data collection and analysis
We conducted semi‐structured, one‐to‐one interviews to gather our data. Each study subject was interviewed once for the study for 45–60 minutes. We audiotaped and transcribed interviews verbatim. The initial interview guides (Appendix S1, online) for these semi‐structured interviews were based on our synthesis of what is known about the impact of CPOE systems on resident education and on our contextual knowledge of the functioning of the GIM services at the various hospitals. We analysed the interview transcripts to identify key themes concurrently with data gathering (as part of our grounded theory methodology) in a process that informed iterative adjustments to the interview script, as well as decisions with respect to saturation.
The analysis was reflexively mindful of the researchers' own subject positions in the research context and of potential power dynamics between the research team and study participants. Six of the seven study investigators are attending physicians in GIM at one of the AMCs used in this study and have different types of exposure to CPOE as users and researchers (as faculty staff or during training) during both transitional and stable periods; all six are colleagues of the attending physicians in this study and could potentially have supervised the residents in this study. Great care was thus taken to anonymise the study data. In particular, all resident interviews were conducted by a research assistant (the seventh investigator on this study). The concurrent analysis was carried out primarily by one study investigator (BW), an attending physician at a non‐CPOE hospital who trained in part at a CPOE hospital, and further refined by other members of the research team (AK, NR, RW, KS) who read transcripts, met to discuss coding schemes and suggested ongoing changes to the interview guides. Together, the team developed a model for analysing the educational impact of patient safety interventions.
Results
Study participants
The study involved 17 participants, including 11 internal medicine residents and six attending staff. Five of the residents had worked at CPOE hospitals as juniors and then at non‐CPOE hospitals as senior residents, whereas six had started in non‐CPOE hospitals and then switched to CPOE hospitals. Two of the six attending staff spend time working at both CPOE and non‐CPOE hospitals.
The educational impact of CPOE
Five major themes emerged describing educational areas affected by CPOE: (i) learning; (ii) teaching; (iii) supervision; (iv) feedback, and (v) assessment. We identified both positive and negative impacts in each of these educational areas (Table 1).
1 Positive and negative impacts of computerised provider order entry on five aspects of postgraduate medical education
Learning
A number of residents reported that it was easier to remember specific medication doses after ordering medications on paper than after using a CPOE system:
'I didn't learn doses as much in my first year because [of] the ... multiple‐choice dosing on CPOE.' (Resident 4)
There were divergent views about the importance of being able to recall medication doses from memory. Some felt that it was still important to memorise medication doses for specific clinical contexts such as medical emergencies. Others believed that it is unrealistic, unnecessary and unsafe to try to commit medication doses to memory. However, when it came to learning the rationale of medication use in specific clinical situations, most trainees felt that the mode of medication ordering had little influence:
'I think knowing doses for medications is not nearly as important as knowing when to use ... medications, which the computer doesn't teach you. That, you learn by taking care of patients.' (Resident 4)
Several aspects of CPOE systems potentially enhance learning. Clinical decision support systems linked to CPOE help to enhance learning with respect to medication interactions:
'The interactions with other medications are built in [the CPOE system] and it's something that you might not have thought of when you're writing [medication orders] on paper.' (Resident 1)
Furthermore, accessing online medication dosing information is more efficient because the CPOE system links directly to these resources, making it easier to look up medication information to support medication ordering:
'The more complex things where you'd be more likely to look up dosages or contraindications... it's faster at the computer for me personally because I tend to look it up online. You just open up a separate window and have the drug reference there as well as the ordering.' (Resident 3)
Teaching
Senior residents (and sometimes attending staff) must review and countersign medication orders written by medical students before they can be carried out. In non‐CPOE hospitals, there is a greater tendency for students and residents to come together and discuss written orders in person, which creates opportunities for teaching. However, when using CPOE, countersigning of medication orders is performed electronically and can be carried out remotely, and may result in the loss of opportunities for teaching. One senior resident commented:
'In a paper system, you'd have to physically co‐sign whatever orders someone wanted so there's a lot more teaching and interaction that would happen in a paper system.' (Resident 9)
By contrast, CPOE systems display the medication orders, medication administration record and electronic patient record all in one place, and supplement case discussions with real patient information:
'I tend to do global med reviews with [clinical] clerks more and what's really nice is then you can see the pattern of Lasix dosing in a congestive heart failure patient... over the past week and you can discuss why we gave 80 mg that day and we're only giving 40 mg now... You can put it up nicely in parallel quickly with the creatinine and discuss it in that context and think about what the potassium is doing... It's much nicer to have that all there in front of you and to teach from that.' (Attending physician 1)
Supervision
Both attending staff and senior residents consistently identified that CPOE improved their ability to supervise from a distance by making medication orders easily available on any computer workstation. This was identified as a major benefit of CPOE, and was felt to help to make the role of supervisor easier and more efficient while promoting safe patient care. One senior resident recalled:
'I had an intern who ordered an extremely high dose of insulin on somebody [that] wasn't out of the range where you could order it on the computer... because some patients need that high a dose but... it was 10 times what the patient should have gotten... Even though it was an intern and I didn't need to co‐sign anything, I still went through the orders that person had done, picked it up and the patient never got [the insulin]. Whereas, in a paper system, I probably wouldn't have gone over his paper orders... because I'd have to go to find the chart... I probably wouldn't have picked it up and the patient might have gotten 10 times the dose of insulin they were supposed to.' (Resident 6)
However, because CPOE makes it easier to order medications remotely, it creates the potential that junior members of the team will have their clinical work done for them. Although this was not felt to be a universal risk, most acknowledged that CPOE provokes this concern:
'Maybe I've done it before as a supervising physician where I go ahead and co‐sign the [medical student's] orders and the resident feels their toes are being stepped on.' (Attending physician 2)
Feedback
Respondents felt that CPOE makes it easier to observe trainee medication‐ordering behaviours and provide feedback. Although this is also possible with paper‐based ordering, many logistical barriers (physically locating a chart, medication orders and medication administration records that are kept in different places) make this more challenging. Some attending staff even used CPOE as a tool that could be used to provide indirect feedback to trainees:
'The nice thing about the system is that... right there, staring you in the face, is who wrote the orders. So if all of a sudden a resident is seeing a lot of staff orders on the basic meds that were maybe ordered incorrectly, that's a more direct way to show that there's been a change and that I've had to change it as staff and so I think that's a good way [to provide feedback].' (Attending physician 1)
Interestingly, this same practice, which is viewed by attending staff as providing feedback, might be viewed by trainees as affording them less independence with respect to patient care:
'The residents have felt... disenfranchised, that someone else is ordering things on the patients and they don't know what's going on... the residents just felt like they probably should stop ordering things because they knew the attending was going to do it... it created a lot of problems with patient care and the residents were unhappy about it.' (Attending physician 3)
Not everyone agreed that this process of changing medication orders post hoc represented an effective form of feedback. Some residents stated that electronic countersigning of medical student orders using CPOE often goes unnoticed by students and can lead to modification of medication orders without any direct feedback being provided:
'I'll be going through what the medical students have ordered and something that's totally wrong, I just won't co‐sign it. But if I'm busy, I won't get around to telling them why I didn't and so that's detrimental to their educational process because they're still likely to go and do the same mistake because they never really discussed why that was not the best choice.' (Resident 5)
Many trainees therefore prefer a verbal discussion to accompany these types of medication change to reinforce feedback on their medication ordering practices.
Assessment
Attending physicians stated that CPOE allows supervisors to assess whether residents are responding to laboratory abnormalities appropriately or starting the appropriate medications (according to best evidence) in a manner that speaks to medical knowledge and clinical decision making. Others also commented that the ability to observe how quickly trainees respond to certain laboratory abnormalities (e.g. replacing electrolytes) provides insight into a resident's managerial and organisational skills. One attending staff doctor recalled:
'I see someone's electrolytes are abnormal, they should have had some intervention 'I see, oh, look at that, they've already got their IV Magnesium, that resident's really on the job' (Attending staff 5)
Learner and supervisor factors that influence the educational impact of CPOE
Several factors were found to modulate the type and degree of impact that CPOE has on the five main themes identified. These included factors related to the learner and the supervisor (Table 2).
2 Learner and supervisor factors that influence the educational impact of computerised provider order entry (CPOE)
Learner factors
The impact of CPOE on learning about medications (including doses, rationale for use and interactions) depended in part on individual learning style. For some residents, the act of generating orders on paper reinforced the learning of medication doses because it required active thinking about which medication to order and how to order it. These residents often described ordering medications using CPOE as a passive process (e.g. 'multiple‐choice dosing'). By contrast, other trainees were more visual, reflective learners and used the medication information displayed by CPOE systems as a springboard for learning about medications.
Some trainees highlighted the potential for CPOE to affect the time available for learning and teaching. This depended largely on the trainee's experience with CPOE and how easily he or she was able to adapt to its use for ordering medications. Residents in the early stages who were slow to adapt tended to report the greatest difficulty in dividing their time into that required to carry out their clinical work and time available for learning and teaching. However, once they were familiar with the system and had identified the added efficiency it afforded with respect to medication review, most residents felt that CPOE facilitated an overall saving in time, which some devoted to learning and teaching.
Supervisor factors
The attending staff or senior resident's supervision style could either dampen or enhance the positive and negative aspects of CPOE on medical education. For example, a supervisor who tended to be more hands‐on or controlling with respect to patient care would tend to undermine a junior trainee's autonomy in either system. In such cases, CPOE served to facilitate these behaviours by making it easier for the supervising doctor to circumvent his or her trainees by making medication review and ordering possible from any location. By contrast, supervisors who were diligent in providing feedback would also ensure that this occurs irrespective of the modality used to order medications. These supervisors would therefore tend to take advantage of the added information available from CPOE systems to inform and enhance the type of feedback they were able to provide to trainees.
Another critical factor concerns the supervising doctor or resident's commitment to teaching and education. Residents who were committed to ensuring that junior trainees were appropriately supervised and taught did so in either context. In the CPOE environment, this may require the use of specific strategies to ensure this occurs. This includes the deliberate review of medical student orders in person (even when it is possible to do so remotely) and the keeping of a running tally of teaching points that are fed back to trainees when opportunities to do so arise. However, residents who were seen as having little interest in their roles as teachers were no more likely to teach in a paper‐based system than a CPOE‐based system. The latter system may have served to facilitate their avoidance of teaching responsibilities, but did not establish this behaviour.
Discussion
This study provides a contemporary, in‐depth summary of the impact of CPOE, an emerging patient safety technology, on the educational experiences of postgraduate internal medicine trainees. We discovered that the use of CPOE for medication ordering broadly impacted on five areas of medication education both positively and negatively. These educational areas include learning, teaching, supervision, feedback and assessment. The degree of impact depended in part on several learner and supervisor factors, including learning style, level of comfort with using CPOE, supervision style and commitment to teaching. Indeed, some of the purported positive and negative impacts of CPOE may in fact be underlying features of the professional and educational practices and organisational culture that are revealed by CPOE rather than directly caused by it.
The majority of the studies that have previously examined the educational impact of CPOE focused primarily on its most obvious educational impact: the learning of medication doses.[[12], [15]] Although our study confirms prior concerns regarding detrimental effects on learning medication doses, we discovered that trainees viewed the learning of specific doses as less important than the learning of indications and contraindications for medication use. The latter may actually be enhanced by the decision support systems that increase awareness of important drug interactions.
Several studies highlighted another negative impact manifested in trainees' reporting of concerns that CPOE increased the time needed to order medications[[13], [17], [21]] and that this might result in medical students losing opportunities to order medications.[21] Trainees also resented the 'clerical' aspect of having to use computers in their work.[[13]] Given these findings, we specifically looked for these effects and, although we asked directly about them, our respondents did not see time as an important issue related to the use of CPOE. There are several potential explanations. A systematic review of studies that have measured the impact of CPOE on doctor time suggests that the increases in time vary significantly[22] and some sites are able to approach time neutrality because other tasks, such as locating charts, are made easier.[23] Both residents and attending staff reported that, despite minor increases in the amount of time spent entering orders, use of a CPOE system allowed for significant time savings with respect to medication review as a result of the companion electronic medication administration record. Thus, on balance, CPOE was seen as saving time in our study. It is also conceivable that our contemporary CPOE system has a better user interface that makes medication ordering easier than the pioneering systems did in the 1990s. Finally, given that half of the prior studies were published in the 1990s, their findings may have limited relevance to our current training environment because considerable advances have been made in computer technology and their use is now ubiquitous, including in health care and in medical education.
We also identify additional threats to the educational experience, most notably with reference to the decrease in face‐to‐face encounters that may result in the loss of opportunities for teaching and in the facilitating of unfavourable supervision practices that undermine trainee autonomy. In addition, we discovered a number of previously unrecognised advantages, including the safer supervision of trainees and enhanced observation of medication‐ordering behaviours to inform feedback. We speculate that one of the reasons we uncovered these novel findings may be that, unlike in previous studies, our trainees and some attending staff routinely rotate from hospitals with CPOE to those without CPOE, and vice versa, and are able to compare and contrast their experiences in real time to provide a much richer description of the various facets of the interaction between CPOE and education.
Conceptual model for evaluating the educational impact of QI or patient safety practices
A recent review of QI in medical education describes an emerging recognition that real‐world QI initiatives have the potential to affect educational experiences in AMCs.[24] Our case study uses CPOE as the context for studying the general phenomenon of the educational impact of QI and patient safety practices, and proposes a conceptual model that can be extended to assess how other QI initiatives affect medical education (Fig. 1). This model's five key elements (learning, teaching, supervision, feedback, assessment) are mapped onto those aspects of clinical work that are directly affected by the QI intervention in question to determine the nature of the educational impact. This helps to direct our attention towards characterising the educational advantages as well as unintended negative consequences that stem from specific aspects of a QI or patient safety intervention.
MAP: 1 Conceptual model for evaluating the educational impact of quality improvement or patient safety practices: (i) define the clinical workflow or task(s) that are altered as a result of a patient safety or quality improvement initiative; (ii) determine the educational themes that map onto the clinical workflow or task(s) in question, and (iii) assess the impact of the patient safety or quality improvement initiative to identify educational opportunities and unintended negative educational consequences
For example, in medication ordering, one type of clinical work that CPOE might affect relates to the countersigning of medical student orders. The
Other widely adopted QI and patient interventions are likely to have similar educational impacts that have yet to be fully characterised. The notable exception is the restriction of resident duty hours, which has received much attention, primarily because this widespread policy change directly affects trainees.[[25]] Reductions in resident duty hours have been found to improve resident work–life balance and to address concerns regarding the impact of sleep deprivation on resident performance.[29] However, concerns have emerged regarding the potential threats to medical education.[[30]] Attending physicians report that they spend less time teaching residents as a consequence of increased direct patient care responsibilities[31] and that their satisfaction with teaching activities has decreased as a result of duty hours reductions.[30]
To illustrate how this conceptual model might be applied, we can hypothesise how other health care quality practices that less explicitly involve trainees, such as rapid response teams, might affect the educational experience. In AMCs, rapid response teams affect a trainee's clinical work surrounding critically ill patients. From an educational standpoint, these patient encounters are important for trainees to gain hands‐on experiential
Once we had created our empirically derived conceptual model, we explored how it might relate to other frameworks that attempt to account for unintended consequences that result from the implementation of new health information technologies (HITs). One framework that has been used within the medical informatics community specifically in relation to CPOE is the interactive socio‐technical analysis (ISTA) framework.[35] We found that our empirical data appeared to support at least some of the features of this theoretically derived framework.
The ISTA framework includes four elements (the features of the HIT itself, the social system, the technical and physical infrastructure, the real‐world use of the HIT) and analyses the interactions among these elements to understand why unintended consequences arise when new HITs are implemented. For example, the ISTA framework suggests that the features of the HIT can directly change the existing social system. Our data showed that CPOE created a mechanism by which orders can be countersigned remotely, which changed the way senior residents and junior trainees review medication orders and potentially decreased opportunities for teaching and feedback. Another interaction suggested by the ISTA framework concerns how the actual use of HIT affects the social system. In our study, we uncovered several empirical examples of such interactions. For example, using CPOE to order medications has both positive and negative impacts on learning about medications and their dosages. Furthermore, using CPOE to review medications enhances clinical supervision practices and makes them safer.
Although the ISTA framework has previously been applied to study new errors, changes to workflow and altered communication patterns caused by CPOE, it was not designed to consider the educational consequences of HITs. Our empirically derived model highlights the important features of the educational experience of trainees in relation to a new HIT, which could be considered an element of the social system within the ISTA framework. Future studies that aim to expand on our current findings could attempt to incorporate the five key educational elements that we derived (i.e. learning, teaching, supervision, feedback, assessment) within the ISTA framework and then empirically validate that modified framework; such work could both extend the theoretical power of the ISTA framework and encourage more frequent consideration of educational impact within the medical informatics community.
Limitations
This study has several limitations. Firstly, the issues identified were as much about how our specific CPOE system was implemented as they were about the intrinsic design of CPOE itself. In addition, we did not include medical students in our study. Although we acknowledge that this is an important omission, we believe that the educational issues are sufficiently different for medical students and would consider their experience one that warrants separate attention. We also did not include specialties other than internal medicine. Given the major differences in both clinical workflow and educational objectives that exist among specialties, our findings may not be entirely transferable. Finally, we gathered our data solely through interviews and not additionally through direct observation; such methodological triangulation might have enriched our study findings.
Conclusions
In this paper, we have provided a comprehensive summary of the impact of an emerging health care quality practice, CPOE, on the educational experiences of postgraduate trainees. There were both positive and negative effects in five key areas of medical education. Educators situated in AMCs that have implemented CPOE systems should be familiar with and capitalise on the educational opportunities associated with CPOE, and mitigate those aspects of CPOE that potentially threaten the safety of the educational experience of postgraduate trainees.
<bold>Contributors: </bold> BW and AK are co‐principal authors of this manuscript. BW and AK contributed to the study conception and design, the analysis and interpretation of data, and the drafting of the article. DM, RW and EE contributed to the study conception and design. KS contributed to the study conception and design, and to the analysis and interpretation of data. NR contributed to the analysis and interpretation of data. All authors contributed to the critical revision of the paper and approved the final manuscript for publication.
<bold>Acknowledgements: </bold> none.
<bold>Funding: </bold> This study was funded by an internal grant that supports research and scholarship in medical education from the Department of Medicine at the University of Toronto. The study sponsor had no role in study design, the collection, analysis, and interpretation of data, or the writing of the article and the decision to submit it for publication.
<bold>Conflicts of interest: </bold> none.
<bold>Ethical approval: </bold> the Research Ethics Board of the University Health Network, University of Toronto (Toronto, ON, Canada) approved this study.
<bold>Appendix S1.</bold> Interview Guide with Prompts.
Graph: Supporting info item
References
1 Kaushal R, Shojania KG, Bates DW. Effects of computerised physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003 ; 163 (12): 1409 – 16.
2 Institute of Medicine. Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press 2001.
3 McGreevey M ; Joint Commission Resources, Inc. Using Technology to Improve Medication Safety. Oakbrook Terrace, IL: Joint Commission Resources, Inc. 2005.
4 Amarasingham R, Plantinga L, Diener‐West M, Gaskin DJ, Powe NR. Clinical information technologies and in‐patient outcomes: a multiple hospital study. Arch Intern Med 2009 ; 169 (2): 108 – 14.
5 van Rosse F, Maat B, Rademaker CM, van Vught AJ, Egberts AC, Bollen CW. The effect of computerised physician order entry on medication prescription errors and clinical outcome in paediatric and intensive care: a systematic review. Pediatrics 2009 ; 123 (4): 1184 – 90.
6 Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K. Exploring the unintended consequences of computerised physician order entry. Stud Health Technol Inform 2007 ; 129 (Pt 1): 198 – 202.
7 Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerised provider order entry. J Am Med Inform Assoc 2007 ; 14 (4): 415 – 23.
8 Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerised provider order entry. J Am Med Inform Assoc 2006 ; 13 (5): 547 – 56.
9 Niazkhani Z, Pirnejad H, Berg M, Aarts J. The impact of computerised provider order entry systems on in‐patient clinical workflow: a literature review. J Am Med Inform Assoc 2009 ; 16 (4): 539 – 49.
Niazkhani Z, Pirnejad H, van der Sijs H, de Bont A, Aarts J. Computerised provider order entry system – does it support the inter‐professional medication process? Lessons from a Dutch academic hospital. Methods Inf Med 2010 ; 49 (1): 20 – 7.
Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, Strom BL. Role of computerised physician order entry systems in facilitating medication errors. JAMA 2005 ; 293 (10): 1197 – 203.
Campbell EM, Sittig DF, Guappone KP, Dykstra RH, Ash JS. Overdependence on technology: an unintended adverse consequence of computerised provider order entry. AMIA Annu Symp Proc 2007 ; 94 – 8.
Ash JS, Gorman PN, Hersh WR, Lavelle M, Poulsen SB. Perceptions of house officers who use physician order entry. Proc AMIA Symp 1999 ; 471 – 5.
Massaro TA. Introducing physician order entry at a major academic medical centre: II. Impact on medical education. Acad Med 1993 ; 68 (1): 25 – 30.
Patterson R, Harasym P. Educational instruction on a hospital information system for medical students during their surgical rotations. J Am Med Inform Assoc 2001 ; 8 (2): 111 – 6.
Stair TO, Howell JM. Effect on medical education of computerised physician order entry. Acad Med 1995 ; 70 (6): 543.
Tierney WM, Overhage JM, McDonald CJ, Wolinsky FD. Medical students' and housestaff's opinions of computerised order‐writing. Acad Med 1994 ; 69 (5): 386 – 9.
Kuper A, Lingard L, Levinson W. Critically appraising qualitative research. BMJ 2008 ; 337 : a1035.
Lingard L, Albert M, Levinson W. Grounded theory, mixed methods, and action research. BMJ 2008 ; 337 : a567.
Charmaz K. Constructing Grounded Theory. Thousand Oaks, CA: Sage Publications 2006 ; 1 – 208.
Knight AM, Kravet SJ, Harper GM, Leff B. The effect of computerised provider order entry on medical student clerkship experiences. J Am Med Inform Assoc 2005 ; 12 (5): 554 – 60.
Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc 2005 ; 12 (5): 505 – 16.
Bates DW, Boyle DL, Teich JM. Impact of computerised physician order entry on physician time. Proc Annu Symp Comput Appl Med Care 1994 ; 996.
Wong BM, Levinson W, Shojania KG. Quality improvement in medical education: current state and future directions. Med Educ 2012 ; 46 (1): 107 – 19.
Brasher AE, Chowdhry S, Hauge LS, Prinz RA. Medical students' perceptions of resident teaching: have duty hours regulations had an impact? Ann Surg 2005 ; 242 (4): 548 – 53 ; discussion 53–5.
Lin GA, Beck DC, Garbutt JM. Residents' perceptions of the effects of work hour limitations at a large teaching hospital. Acad Med 2006 ; 81 (1): 63 – 7.
Lin GA, Beck DC, Stewart AL, Garbutt JM. Resident perceptions of the impact of work hour limitations. J Gen Intern Med 2007 ; 22 (7): 969 – 75.
Temple J. Time for training: a review of the impact of the European Working Time Directive on the quality of training. 2010 ; 1 – 81.
Institute of Medicine. Resident Duty Hours: Enhancing Sleep, Supervision, and Safety. Washington, DC: National Academies Press 2009.
Arora V, Meltzer D. Effect of ACGME duty hours on attending physician teaching and satisfaction. Arch Intern Med 2008 ; 168 (11): 1226 – 8.
Goitein L, Shanafelt TD, Nathens AB, Curtis JR. Effects of resident work hour limitations on faculty professional lives. J Gen Intern Med 2008 ; 23 (7): 1077 – 83.
Myers JS, Bellini LM, Morris JB, Graham D, Katz J, Potts JR, Weiner C, Volpp KG. Internal medicine and general surgery residents' attitudes about the ACGME duty hours regulations: a multicentre study. Acad Med 2006 ; 81 (12): 1052 – 8.
Reed DA, Levine RB, Miller RG, Ashar BH, Bass EB, Rice T, Cofrancesco J. Impact of duty hour regulations on medical students' education: views of key clinical faculty. J Gen Intern Med 2008 ; 23 (7): 1084 – 9.
Reed DA, Levine RB, Miller RG, Ashar BH, Bass EB, Rice TN, Cofrancesco J. Effect of residency duty‐hour limits: views of key clinical faculty. Arch Intern Med 2007 ; 167 (14): 1487 – 92.
Harrison MI, Koppel R, Bar‐Lev S. Unintended consequences of information technologies in health care – an interactive sociotechnical analysis. J Am Med Inform Assoc 2007 ; 14 (5): 542 – 9.
By Brian Wong; Ayelet Kuper; Nicole Robinson; Dante Morra; Edward Etchells; Robert Wu and Kaveh Shojania
Reported by Author; Author; Author; Author; Author; Author; Author