Previous studies have shown the importance of workflow issues in the implementation of CPOE systems and patient safety practices. To understand the impact of CPOE on clinical workflow, we developed a conceptual framework and conducted a literature search for CPOE evaluations between 1990 and June 2007. Fifty-one publications were identified that disclosed mixed effects of CPOE systems. Among the frequently reported workflow advantages were the legible orders, remote accessibility of the systems, and the shorter order turnaround times. Among the frequently reported disadvantages were the time-consuming and problematic user-system interactions, and the enforcement of a predefined relationship between clinical tasks and between providers. Regarding the diversity of findings in the literature, we conclude that more multi-method research is needed to explore CPOE's multidimensional and collective impact on especially collaborative workflow.
Computerized provider order entry (CPOE) systems have been recognized as highly valuable tools to increase the efficiency and effectiveness of medical work.1 However, their potential to change workflow and its consequence for patient safety has brought the concept of workflow to the forefront of CPOE implementation.2,3 As a result, the integration of CPOE systems into clinical workflow has been identified as one of the most important implementation considerations.4 Nevertheless, studies have shown that this integration may not be easy.5
It has been argued that interruptions in workflow after the implementation of healthcare information systems (HISs) have mainly arisen due to a narrow and simplistic workflow model that underlies these systems.6 When this simplistic model is put into practice, it often fails to address the highly cognitive, collective, collaborative, and ad hoc nature of clinical workflow.7 For example, the model of workflow in these systems tends to conceptualize order creation and communication in a predefined, linear, and stepwise fashion, whereby only physicians' computerized orders give the permission to carry them out.6 Yet, medical work is far from being such a straightforward process. Rather, it is fundamentally a multitasking, cognitive, distributive, collaborative, interpretative, interruptive, responsive, and reactive procedure.8,9 These characteristics need to be understood and considered in CPOE design.
The aim of this paper was to gain an insight into the impact of CPOE systems on clinical workflow. We addressed specifically the following questions: “What are the benefits and/or difficulties that CPOE systems bring to clinical workflow?” and “Which aspects of clinical workflow are most impacted by CPOE implementation?” An understanding of the pragmatic workflow involving CPOE can help to improve the model of workflow that underlies these systems.
As the concept of clinical workflow has different connotations, defining a conceptual model was deemed necessary. For this purpose, we first drew upon principles of the modeling of work processes in the workflow literature.10,11 This literature deals with the modeling of work processes to design information systems that not only do the work, but also manage the workflow: “the process is managed by a computer program that assigns the work, passes it on, and tracks its progress”10. These information systems contain organizational knowledge of where work flows in default cases. They are defined as systems that “help organizations to specify, execute, monitor, and coordinate the flow of work cases within a distributed office environment”11.
Guided by this description of workflow, we next did an integrative review (page 32)12 of the social and cognitive sciences, and the field of Computer Supported Cooperative Work (CSCW). The sociology of medical work has studied how division of labor and articulation work enable different professional groups to carry out tasks when managing care trajectories.13,14 The cognitive science deals with the analysis and modeling of complex human performance such as decision-making.15,16 The field of CSCW examines the computer-assisted collaborative activities such as communication carried out by a group of collaborating individuals. It has been noted that medical informatics can benefit from the insights gained in this field to design and deploy successful HISs.17 By summarizing broad themes in these fields pertaining to the concept of clinical workflow, we developed a conceptual model. The resulting model enabled us to examine the interplay between the social context of healthcare work and CPOE systems.
Healthcare is a complex activity system of specialized and non-specialized workers, their tools, and their environment.9 Healthcare work involves continuous interaction among different elements and trade-offs between multiple goals, preferences, values, incentives, and motivations in the course of care processes.18 Physical (e.g., paper records) and psychological artifacts (e.g., individual experiences) mediate the work and foster collaboration.19,20 Despite being spatially distributed, the work of different actors in healthcare is highly interconnected because they are dependent upon each other in terms of skill, knowledge, expertise, and physical assistance.21
A Model for Clinical Workflow
In the workflow literature, a workflow process is defined as “a predefined set of work steps, and partial ordering of these steps”11. Workflow processes are carried out by participants that can “fulfill roles to execute, to be responsible for, or to be associated in some ways with activities and processes.” Inspired by this literature, we define clinical workflow as the flow of care-related tasks as seen in the management of a patient trajectory: the allocation of multiple tasks of a provider or of co-working providers in the processes of care and the way they collaborate. The aspects of clinical workflow therefore can be categorized into four elements: (1) structuring of clinical tasks, (2) coordinating of task performance, (3) enabling of the flow of information to support task performance, and (4) its monitoring.10,11 These aspects are often closely connected to and dependent upon each other, as any intervention in one aspect can affect the others. Figure 1 shows a visual model of these aspects and their relationship. We will touch upon them in the following sections.
A conceptual model for clinical workflow, showing its different aspects and their relationship.
Structuring of Tasks
To avoid possible conflicts among tasks and providers, a work structure is required on the basis of which actions as well as interactions can be constructed. This is mainly the subject of “division of labor”, which deals with “dividing up work, workers, and the relationships both between and within these divisions.”13 It is referred to as “formal task-structure space” in Fig 1. The formal version of task structure is mainly drawn on the integration of organizational knowledge and domain knowledge in healthcare. Organizational knowledge is based on local cultures, norms, values, and available capacities or accessible resources while medical domain knowledge gets inputs from evidence-based findings. The resulting work structure particularly specifies “who” does “what”, “when”, “where”, and “how” by employing “which resources”, and in “what relation” to other tasks and providers (i.e., sequentially, simultaneously, or in any other order).
Medical work is comprised of tasks of individual providers as well as the tasks which connect collaborating providers. Researchers who studied cognition in medical work have described the cognitive models of an individual clinician's task performance and defined the demand characteristics of particular tasks such as information management strategies.22 But also they have started to characterize cognition as a process that is distributed across groups, cultures, and artifacts.23,24 This indicates that even seemingly discrete individual activities take place while dynamically interacting with other complex factors such as social and organizational.16,25
Coordination of Work
To perform tasks, co-workers are required not only to coordinate with each other but also to coordinate their temporal and spatial dimensions. To coordinate tasks, actors passively follow the scripted roles and relationships among the tasks coded in written rules, plans, or tacitly assumed traditions and norms.26 For temporal coordination between tasks, three levels of activities have been defined: synchronization of interrelated tasks, scheduling, and temporal allocation.27 Moreover, care is provided by different professionals in different specialties using different resources in the hospital. To gain access to them, providers and patients should move within and between these specialties.28 Therefore, the spatial dimension of tasks also needs to be coordinated.
Information Processing and Flow
Medical work is information-intensive. Hence, the collection, documentation, communication, and retrieval of patient information are among the critical activities of providers (page 251).29 The source of information may be patients, colleagues, or other informed individuals, but it may also be medical records. These disparate pieces of information should then be integrated, completed, verified, interpreted, or negotiated. This is necessary because of the contextual nature of information, which implies that data acquired from different sources are not self-explanatory.30 As a next step, information should be communicated to enable the collaboration of multiple providers involved.
To cooperate, actors must actively adjust the actions in hand with the actions of co-workings.26 For this purpose, they need to monitor for changes in task requirements. Monitoring provides an overview of ongoing activities and enables providers to supervise and control the intended execution of tasks.
As discussed earlier, the task structure using organizational- and domain-knowledge serves the core in constructing workflow. Yet, medical work is inherently ad hoc and contingent. To avoid any halt or to recover from that, providers restructure their work constantly.14 For instance, a continuing deterioration in a patient's condition or unavailability of certain resources may necessitate rearranging the patient's care plan by canceling the previous orders, by reordering task priorities, or by involving new providers and procedures.
Moreover, the familiar pattern of healthcare work is what Strauss termed “negotiated order” (page 267).29 In a patient trajectory, multiple representatives of different professional groups interact constantly. To trade off and reach a formal or informal agreement in any organizational action (such as decision-making), negotiation is necessary. In fact, in the light of information flow and the conditions of coordinative and cooperative work, clinicians often negotiate and re-construct their work. For this co-constructed workflow, actors first focus on co-construction of a shared object and then turn to re-conceptualize their workflow on the basis of this shared object.27
Search Strategy and Inclusion Criteria
A literature review was conducted in the PubMed and Cochrane library for journal articles, conference proceedings, and summaries. We used MeSH terms and keywords to identify CPOE evaluations published in the English language between Jan 1990 and Jun 2007. To detect relevant articles in the social, computer and cognitive sciences that may have evaluated CPOE systems, we also searched two other databases: the IEEE Computer Society and the Sciences Citation Index. Figure 2 shows a complete list of our search terms and search strategy and flow.
After duplicate literature, non-English publications, and those without abstractsa were removed, the search resulted in 1589 publications. Among them, we searched for studies that (1) evaluated the effects of CPOE on realistic or simulated workflow of care providers, (2) were carried out in inpatient settings, and (3) reported on either quantitative or qualitative studies. First, the title and the abstract of the primary set of publications were reviewed to find relevant articles. We had two inclusion criteria: (1) the system under evaluation must be a computerized system whereby a provider in an inpatient setting enters patient's therapeutic or diagnostic orders into a computer, and (2) at least one of the evaluation objectives must concern the workflow of providers in order entry and communication processes. Studies that reported users' perceptions of CPOE effects were also included in the review. To detect relevant literature, we used the general definition of the “flow of care-related tasks” of an individual provider or of co-working providers.
Because this review was focused on inpatient workflow, we excluded studies of ordering systems in outpatients and emergency departments. Studies that had evaluated issues other than clinical workflow, such as return of investments, number of medical errors, and so forth were also excluded. Opinion papers, reviews, letters, and system design and implementation reports that lacked an explicit evaluation focus were excluded as well, mainly because they elaborated upon system features or implementation strategies without really evaluating effects on workflow. Figure 2 lists our exclusion criteria.
One hundred and forty-two publications were identified for detailed review. To complete the search, we also examined the bibliographies of included articles, reviews of CPOE publications, and an inventory of evaluation publications.31 We identified 8 publications that did not show up in the primary set of our search. To access unavailable publications or to inquire additional information, we contacted 20 authors (80% success rate). A consensus about the final set of selected publications was reached after discussions among this paper's authors.
The first and second authors extracted the main findings of the selected publications and then categorized them based on the positive or negative/challenging effects. The preliminary categories were identified and iteratively revised until a consensus was reached after many discussions. These findings were analyzed at three levels. First, we analyzed them on the basis of our conceptual model. Then, we conducted two sub-analysis based on: (1) workflow of individual providers versus co-working providers, and (2) workflow with homegrown versus commercial systems.
Characteristics of Selected Publications
The review identified 51 publications: 31 journal articles,32–62 16 proceedings papers,63–78 and four proceedings abstracts.79–82 Table 1 (available as an online data supplement at http://www.jamia.org) lists them according to the chronological order of the publication year. It also provides additional information, including, study description, the type of systems and clinical settings, and main findings. These 51 publications reported on 45 evaluation studies, as the results of some studies appeared in more than one publication type. The research designs used were mixed-method (n = 5), quantitative (n = 25), and qualitative studies (n = 21).
Six publications reported on workflow simulation methods: in part77 or in whole.48,62, 72,74,81 Most studies were conducted in the context of commercial systems, in academic hospitals, and in adult inpatient settings. In the next section, we present the findings based on reported positive and negative/challenging effects.
Remote access to enter orders or view their status (such as the result of diagnostic tests) was highly appreciated.35,37,38,41,42,55,58,59,64,66 Such systems enabled multiple people to view the same patient's orders simultaneously.64 Furthermore, access to knowledge sources, decision support, order sets, graphical display of data, and easier charting of medications were found to be supportive for providers.35,38,52,55,57,64,82
CPOE systems removed many intermediary and time-consuming tasks for physicians (e.g., looking for data), nurses (e.g., transcribing orders) and ancillary departments (e.g., entering orders into the departmental information systems).33,37,38,41,42,55,58,59,64,66,69 One study showed that clerks, nurses, and pharmacists spent less time per day on the medication process after the implementation.47 However, in another study, no difference was found between pre- and post-implementation regarding the time pharmacists spent to process medication orders.58 One study found that physicians had more time to talk with patients after the implementation.69 Moreover, asynchronous communication through these systems resulted in fewer work interruptions to clarify illegible orders or to inquire necessary information from other providers.42,58 Four studies reported that the number of phone calls between co-working providers decreased.39,42,55,60
CPOE had positive impact on order turnaround times. Six before-and-after studies demonstrated a substantial decrease in the drug turnaround time, varying from 23 to 92%.46,47,49,60,68,76 This reduction was mainly attributed to the removal of certain intermediary tasks between order initiation by a physician, verification by a pharmacy, and administration by a nurse. Three studies compared the time interval between a physician's radiology requests and the completion of the procedures pre- and post-implementation and found a significant reduction of 24% to 69%.46,49,50 Similar shorter turnaround time was also observed for laboratory orders, varying from 21 to 50%.46,50,56 One study found a reduction of 3 hours between the time the laboratory tests were ordered and the time the results became available.39
By forcing order entry through the system and facilitating remote access, CPOE systems could decrease verbal orders. A study calculated a 75% reduction in the number of verbal and telephone orders.60 A similar trend was shown in a Children's Hospital.54 Three studies showed that the rate of order countersignatures improved.46,54,60
Negative or Challenging Effects
Using CPOE systems was found to be time-consuming for clinicians. Five studies referred to the perception held by physicians that more time was spent on ordering after the implementation.33,36,57,59,66 Five studies compared the time physicians spent on ordering using CPOE systems to paper-based systems.32,39,40,69,79 A significant increase in time was seen in all studies except one39 in which a laboratory order entry system resulted in 5.5 minutes less time. One study found that order entry sessions using a graphical format significantly took less time than a text-based format.82 Two studies mentioned the physicians' perception of having less time to spend with patients because of spending more time on CPOE systems.37,41 One CPOE study found an increase in administration documenting time for nurses.40 However, most of these studies looked at subsets of a clinician's workflow, and not the overall workflow in a day.
Usability Issues of CPOE Systems
Usability limitations and their effects on workflow were well discussed in the literature. Table 2 lists several of the difficulties experienced due to interaction with problematic hardware/software or due to an inadequate integration or ineffective interface between different information systems in a hospital. We grouped them in terms of system availability and human-computer interaction in Table 2.
Usability Limitations Identified in the Selected CPOE Literature
Problems associated with downtime,41,66 accessibility of workstations while on rounds,66 servicing computers and printers,64 poorly interfaced different information systems in one hospital;53,58,64,65 difficulties due to transfer of patients in a hybrid electronic-paper environment58,66,67
Inability to access the system when another user is accessing the same patient's record simultaneously62,66
Slow response time;37–39,41 inconveniency of logging into the system;38,71,78 troublesome manipulation of keyboards37
Complex and lengthy process of medication ordering, especially in the time of admission, discharge and transfer;38,43,64,66,67 difficulty with processing of non-standard orders53,67
No possibility to switch between two paths with numerous screens for order entry to enter or retrieve information;43 difficulty to gain an overview on patient hospital stay53,55,75
Problematic data presentations such as patient medication profile design;53,67 clutter of order and note screens;66 difficulty to see a patient's name on the screen;64 problematic highlighting of the nursing administration rounds in the system's timetable51
No possibility to enter free texts due to prefixed text entries; inadequate word-processing capabilities; inadequate space for notes43,53,64
Unfamiliar or confusing cognitive model of classifying orders in the system;53,67,71,74 suboptimal interface affordances making extra demands on user's internal resources;72 mismatch between cognitive model of tasks in the system with physicians' cognitive activities for order entry77
CPOE = computerized provider order entry.
The limitations relating to human-computer interaction mainly involved an individual provider's tasks of “entering and/or retrieving orders”. To overcome system inflexibilities, providers were sometimes obliged to take additional or alternative steps to continue the work: for example, to double chart on paper and on computer55 or to use computer printouts as flexible data medium.53 Providers also sometimes bypassed the system completely: for example, using a colleague's open logging session.71 A simulation study showed that a CPOE system may enforce a very sequential and inflexible order of activities, which may be completely bypassed under emergency situations.62
Ineffective interface between different departmental information systems can cause interruptions for providers working in different departments. Two studies referred to administrative workload increased in the ancillary departments due to transferring orders manually from one system to another, followed by subsequent frustrated calls for clarification.58,65 Moreover, some studies reported workflow interruptions due to lack of bedside systems or defected computers and printers. These issues are merely artifacts of inconvenient implementation of the technology and/or its maintenance and not representative of qualitative differences between CPOE versus paper-based systems. Nevertheless, it has been shown that such issues fairly influence workflow.53,64,66,
An important CPOE impact discussed in the literature concerns the structure of tasks that require multiple providers to be involved in teamwork. The application of CPOE systems changes teamwork in two ways: by re-delegating tasks between co-working providers, and by changing communication channels and collaboration mechanisms.
First, after the implementation, the re-delegation of tasks between providers transforms previously assigned tasks. In some cases, CPOE systems enforced predefined and standardized roles and responsibilities. Two studies highlighted the problematic role-based authorization of entering orders, in which only physicians were authorized.53,71 For a successful order entry, physicians may in turn be obliged to deal with the requirements of structured data entry. Physicians sometimes perceived it as a clerical task comparing to the lax hand-written practices.33 It has been reported that the exclusive order entry by physicians may result in leaving nurses out of the ordering loop.33,67 Similarly, in one study, the pharmacists reported that the system took away some of flexibilities of their paper-based system to allow them to take clinically justified decisions in cases they disagreed with particular physician orders.58 However, provision of decision supports and alerts regarding hospital guidelines or drug restriction policies has expanded their role in ordering practice while weakening physicians' autonomy.53,58,67
Shifting of responsibilities was also observed in the processing of laboratory orders. Georgiou, et al discovered that a computerized laboratory order entry system shifted some responsibilities of the laboratory staff to the clinicians on the wards.59,61 These clinicians were required to check for those laboratory orders that had been issued without the specimens and also to determine their accurate collection times.
Furthermore, the pattern of responsibilities for providers also changes after CPOE implementation. Two studies mentioned a new responsibility for nurses to reconcile the orders edited by pharmacists with the physician-initiated orders.67,71 In addition, nurses had to ensure that a verbal order had been entered by physicians, while this issue was not crucial before.71 In fact, it was the implementation of CPOE and thereby that of organizational rules that highlighted the issue of unsigned verbal orders.54 Because these changes are not often anticipated beforehand, providers then may be left unsure about the tasks that fall within their responsibility. One study referred to the uncertainty of who should check and take care of automatic “stop” and “expiring” orders: physicians or nurses.67 A similar uncertainty of having a responsibility caused subtle tension between laboratory technicians and nurses in another study.65
Second, CPOE systems have changed the traditional communication channels and collaboration mechanisms. After implementation, interaction with these systems may replace interpersonal contacts that may result in fewer opportunities for team-wide negotiations.53,59,78 Studies have indicated that CPOE may maintain a centralized decision making paradigm with physician dominancy despite the fact that in practice nurses may notify physicians of emergent needs for orders.71,73 Dykstra referred to systems that compelled physicians to enter their orders in computer rooms while away from other members of a care team.70 In such cases, providers may assume that the system would communicate their orders, plans, and ideas.
In the absence of direct communication (such as verbal notification) and other visual clues (such as bedside physician order writing) following CPOE implementation, a new imperative has emerged: to notify recipient providers who need to take care of orders timely.33,53,66,67,70,71 Some studies referred to the notifications taking place by means of computerized alerts or printouts. Nevertheless, for busy clinicians moving around, it is not possible to check printers and computers frequently. Hence, a delay in processing orders may occur due to a delay in an acknowledgment of these notifications.70,71
On the Basis of Our Conceptual Model
The analysis on the basis of our conceptual model showed that the modeling principles of CPOE systems generally make use of a formal, predefined division of tasks and a preconceived relationship between clinical tasks and also between care providers. With regard to division of labor, our analysis highlighted that CPOE systems authorize a formal task structure that includes role-based division of tasks and a consecutive order in task execution. Such a sharp division of tasks can in theory help care providers to recognize their responsibilities clearly and lead to better safety procedures, for example, when a physician decides on details of orders, documents them, or responds to safety alerts.84 However, studies have shown that a literal translation of this formal and hierarchical authorization in CPOE limits the effective contribution of all providers in the ordering activities.33,34,51,58,71 This in turn can jeopardize teamwork in medical practice. For instance, in the formal division of labor, the task of ordering falls under the authorization of physicians. Nevertheless, in practice, order creation is the product of negotiation, sharing of information, redistribution of responsibilities, and informal delegation of the ordering tasks among providers.6,51,71 The model of strict and physician-dominant authorization underlying CPOE therefore may partly mismatch with the negotiated and co-constructed nature of ordering practice.
Studies that analyzed the cognitive tasks of ordering practice by physicians criticized its cognitive model incorporated into CPOE systems.48,72,74,75,77 They indicated that these two may not reasonably correspond with each other. They also noted that interaction with these systems may burden physicians with cognitive overloads.48 One study found that order planning by a physician for complex patients is primarily problem-based in contrast to the mnemonic-based frameworks underlying CPOE systems.77 Such discrepancies may further compound the user-system interactions.
Most CPOE systems considerably reduce order turnaround times, which corresponds to timeliness of care. Nevertheless, they may negatively affect the temporal coordination of tasks. The straightforward order of activities with CPOE systems may hinder the synchronization of those tasks that are interdependent. In a study, after physicians entered laboratory orders into the system, their electronic requests were promptly sent to the laboratory departments.61 The laboratory technicians were then confronted with several laboratory requests without the corresponding specimens, because nurses could not prepare and send them at the same time physicians entered orders. Similarly in another study, after order entry by physicians, nurses received two order printouts, one from physicians and the second from pharmacists after order verification.71 Lack of activity synchronization among providers can be a source of frustration necessitating extra effort to clarify the issue.61,66 Moreover, as Reddy described,85 clinical tasks in the hospital are often accomplished in temporal rhythms. A nurse may know better when to administer a drug or when to draw a blood sample, because these tasks are integrated into the temporal rhythms of their workflow. Yet, using CPOE compels physicians to choose strict schedules for orders that may not always be compatible with the practice.51
Our analysis revealed that the spatial dimension of medical work also challenges the mediating role of CPOE systems. As they mostly tend to be accessible from fixed workstations, providers working at bedsides may be interrupted because they are obliged to walk to the workstations.66,71 As well as providers, patients also move between different units. This implies that the system should be accessible across formal divisional boundaries of hospital units.52,66,86 Therefore, appropriate transit orders should be considered in the computer environment.
Various CPOE systems have mixed effects on information flow. They enable the communication of legible and complete orders between providers, which has greatly reduced the transcription task workload of recipient parties. However, some studies questioned the affordances of these systems to furnish providers with an overview of patient information.53,67,75 It has also been pointed out that the ability of these systems to integrate different pieces of information and to communicate their contextual meaning is limited.30 This is compounded by the fact that the predefined data entry options on the screens may limit the sharing of psychological, social, or emotional information relating to patients.43 It has also been noted that because of fewer team-wide discussions, information accessed through these systems may not be easily interpreted by clinicians.53,67,70,71 Thus, human interpretation of information is still of critical value for information processing.87
Last, changes in work structure transform the mechanisms by which clinicians control their work. In the Results section, we referred to the challenge of monitoring newly issued orders through CPOE systems. In such cases, physicians who initiate orders may simply assume that their orders are delivered to the right providers at the right time.70 However, such over-reliance on CPOE systems may give rise to the late implementation of orders.66,70,71
Individual versus Collaborative Workflow
Regarding the concept of workflow in the literature, two areas of focus were recognized: that of one individual provider and that involving more than one provider. The first mainly highlighted the advantages and/or disadvantages experienced by an individual provider while interacting with CPOE systems to perform tasks. This has mainly informed us as to how this interaction can be improved (for examples please see32,38,43,48,62,74,75,82). The second area, however, widened the scope of interest to the collaborative flow of tasks between co-working providers. This area has shown how the work of different providers is highly interdependent; so that, any change in one's work might positively or negatively affect the others (for examples please see33,42,51,58,61,66,70,71). This area therefore has informed us how the automation of order entry process can have serious implications for the workflow between providers working in the same or different departments.
Our analysis of these concepts in the literature indicates that the first area dominated the discussion in the literature (see details in Table 1 available as an online data supplement at http://www.jamia.org) even though the collaborative nature is dominant in the collective clinical workflow, as detailed in our conceptual model (Fig 1).
Homegrown versus Commercial Systems
For this analysis, data were available in 41 evaluation studies. Among 5 studies evaluated both commercial and home grown systems, only one study45 compared the results regarding this variable. In this study, users of a commercial system were dissatisfied and reported it to be difficult, cumbersome, and time-consuming to perform routine tasks.
While workflow evaluations of home grown systems were published before 2001, most studies of commercial systems appeared in later years. Positive and negative effects appeared in both types of systems. Except one mixed-method study,66 the home-grown studies were all quantitative. The focus in these studies tended to be on evaluating the time-efficiency of physicians after CPOE. Quantitative studies of commercial systems mainly documented shorter order turnaround times. Contextual effects of CPOE such as changes in roles, responsibilities, and workload of providers, and changes in collaboration mechanisms were predominantly evaluated in the context of commercial systems.
Our review shows that the impact of CPOE on clinical workflow is double-edged. On the one hand, it shows that the implementation of CPOE systems has resolved many disadvantages associated with the workflow in paper-based practices. Many CPOE systems have improved workflow efficiency in terms of the legibility and completeness of orders; the availability of decision support features and order sets; the remote accessibility of the system; the possibility to view the same patient data simultaneously by multiple providers; and fewer work interruptions due to asynchronous communication. They have also decreased verbal orders and improved order countersignature. Furthermore, these systems contributed in time efficiency in term of shorter order turnaround times.
On the other hand, our review also reveals that the implementation is accompanied by difficulties in workflow, mainly due to changes in the structure of pre-implementation work. Negative effects included time-consuming user-system interaction; the removal of visual clues available in paper-based systems; the enforcing of predefined and stepwise order of activities as well as role-based relationship between providers; emerging problems in the synchronization of interdependent tasks; and the restricting of opportunities for team-wide discussions.
All CPOE systems are implemented within a wide socio-technical context, within which the interplay of diverse social, technical, and organizational factors influence their effects on workflow.5,88 Studies of HIS use have shown that to reduce interruptions in workflow, providers may develop “workarounds”2,89,90. Indeed, many systems may continue to operate only because users devise workarounds to avoid difficulties. The results of such ad hoc efforts are variable; they can either smooth the workflow or disturb its balance. It is notable that these workarounds are not registered in or monitored by CPOE systems; thus, they may give a false sense of work support, because despite disruptions the work is still carried out. Such contextual issues in CPOE use will be easily disregarded in design and redesign processes if they are not detected and understood in evaluation studies. Experience shows that with a multifaceted research approach there is a high chance of identifying such contextual issues (see for instance43,51,53,58,61,64,70,71). In fact, multi-method, quantitative and qualitative studies can help not only to answer “what”, “where”, and “when” questions but also to gain an in-depth understanding of “how” CPOE systems behave in their implementation environment, as well as “what the users' reactions are” and “why”91,92. These studies should take practice-oriented workflows as their starting point.
Individual versus Collaborative Workflow
The concepts related to an individual provider's workflow and that between co-workings are highly interdependent and equally important in having a smooth clinical workflow. Although we do not question the relevancy of the first concept, based on our analysis of the findings we argue that its dominancy may result in marginalizing the collaborative problem-solving, decentralized decision-making paradigm, and negotiated and co-constructive nature of clinical activities. For example, paying more attention to improving the workflow of individual physicians in order entry process (see for instance93) may result in overlooking the fact that they are dependent upon the work of other providers. In that sense, even if a system perfectly works for physicians, it may not support the collaborative practice that physicians are reliant upon. Our study therefore suggests that for CPOE to have a more positive impact, besides the individual providers' tasks, it also needs to support the collaborative nature of workflow sufficiently.
Moreover, we suggest that studies of workflow in CPOE environment should widen their units of analysis to cover the collective workflow of an individual provider in the course of a day or that of collaborating providers in a clinical process such as the medication process. Limited units of analysis may fail to discover that, for example, even though CPOE takes time for a provider it also saves the time that would otherwise be spent on walking to a ward for finding information or on responding to the calls of other providers for clarification of illegible orders or correction of interaction errors.
Homegrown versus Commercial Systems
In this review, the number of publications relating to home-grown systems was relatively low. This could be because a small number of academic institutions pioneered in developing CPOE systems. The objective and methodology of evaluation studies in this group are possibly an indication that, in the early years of developments and installations, these institutions invested time and effort on overcoming the resistance of physicians as the primary users. Furthermore, the home grown systems were developed by in-house development teams who were clinically knowledgeable. It is plausible that workflow interruptions and difficulties in system use were detected in informal evaluations and communications, and that the in-house teams could closely monitor and address workflow issues by pilot testing, redesigning, and integrating these systems to local workflows without formally documenting, reporting, or publishing the results. It is also possible that results only appeared in the form of design, redesign, and implementation reports, which were among our exclusion criteria. Thus, some of the findings in this review may not be applicable for home grown systems.
Our review shows that the focus and methodology of evaluation studies have been shifted after 2001, i.e., paying more attention to collaborative workflow and conducting more qualitative studies. This could be the result of researchers' awareness of socio-technico-organizational issues and the call to address them in evaluation studies.91,94 Or, it might be because, especially after the IOM's call to build a safer health system,95 more hospitals have been encouraged to invest in CPOE systems. For many healthcare institutions, commercial systems have been an option to save time, effort, and expertise necessary for system developments. To justify the value of the investment and/or to detect and rectify these systems' detrimental effects, these institutions needed more formal evaluations. As our review shows, most formal evaluation studies of the CPOE's contextual effects are related to commercial systems.
Strengths and Weaknesses of the Study
Several systematic reviews of CPOE systems have been done so far. To date, few, if any, studies have analyzed CPOE evaluations exclusively with respect to clinical workflow. Yet, as one of the central issues in the deployment of CPOE systems, clinical workflow is exceedingly complex and needs to be better understood.96 Our conceptual framework based on insights from relevant fields created the necessary background and allowed us to analyze CPOE's multidimensional and collective effects. Another strength of our study relates to the combination of different search terms used and the databases reviewed to find most relevant publications. We also did not confine our review to specific quantitative or qualitative studies.
Nevertheless, our study has several limitations: First, our search strategy identified 51 publications in total. It is possible that the time span we set to detect relevant publications may have missed pertinent studies published before or after that period. The number of CPOE evaluations related to workflow issues shows a growing trend by time. Therefore, expanding the time period to include the publications appeared through 2007 and 2008 might have changed our discussion and conclusion. Second, because of the complexity of workflow related concepts and the lack of agreed upon research methods to evaluate them, many of the discussions around clinical workflow have only been appeared in other forms of publications than the original research papers. A literature review, which is tightly bound by the methods of searching and the content of the articles that meet inclusion criteria, therefore may not well reflect a proper balance of what is known. Yet, it may well direct future research. Third, our study touched upon the effects of usability issues on clinical workflow. However, other search strategies may help to detect all relevant studies evaluating the effects of usability issues on clinical workflow. Next, we analyzed the effects of a broad range of CPOE applications implemented in various inpatient units. Because data related to the details of clinical units and/or features of CPOE systems under study were often incomplete in study reports, we therefore did not associate the reported effects with these factors. Further studies are required to control these factors and to detect such associations: for example, by evaluating the impact of the same system in different specialties or the effects of different systems in similar specialties. Last, as we discussed earlier, some of the findings in this review may not be relevant to home grown systems.
To our knowledge, this literature review is possibly the first to be dedicated exclusively to the impact of CPOE on clinical workflow. Our conceptual framework helped us to analyze the pros and cons of such effects. Clinical workflow is highly contingent and collaborative. Many in situ contextual factors such as the kind of specialties, the time through a day and so forth may have an influence on it. Based on the contextual factors, providers may decide to rearrange the order of activities or redelegate certain responsibilities among themselves.83 When put in practice, the formal, predefined, stepwise, and role-based models of workflow underlying CPOE systems may show a fragile compatibility with the contingent, pragmatic, and co-constructive nature of workflow. This in turn can cause an interruption in workflow and challenge the integration of these systems into daily practice.
Regarding the diversity of findings in the literature, we conclude that more multi-method research is needed to explore CPOE's multidimensional and collective impact on especially collaborative workflow. This review may inform designers, implementers, and evaluators how to pay closer attention to the collective, multidimensional, and contextual impact of CPOE systems on clinical workflow.
↵a Among these publications, the titles were evaluated to decide whether or not to include them in the detailed review.
. Computerised pathology test order entry reduces laboratory turnaround times and influences tests ordered by hospital clinicians: A controlled before and after study. J Clin Pathol 2006May;59(5):533–536.
. Examining the organizational implications of IT use in hospital-based health care: A case study of computerized order entry. The 32nd Hawai'i international Conference on System Sciences 1999; Maui-Hawai'i, 1999.