OUP user menu

★ Review Paper ★

The Effect of Electronic Prescribing on Medication Errors and Adverse Drug Events: A Systematic Review

Elske Ammenwerth , Petra Schnell-Inderst , Christof Machan , Uwe Siebert
DOI: http://dx.doi.org/10.1197/jamia.M2667 585-600 First published online: 1 September 2008


The objective of this systematic review is to analyse the relative risk reduction on medication error and adverse drug events (ADE) by computerized physician order entry systems (CPOE). We included controlled field studies and pretest-posttest studies, evaluating all types of CPOE systems, drugs and clinical settings. We present the results in evidence tables, calculate the risk ratio with 95% confidence interval and perform subgroup analyses for categorical factors, such as the level of care, patient group, type of drug, type of system, functionality of the system, comparison group type, study design, and the method for detecting errors. Of the 25 studies that analysed the effects on the medication error rate, 23 showed a significant relative risk reduction of 13% to 99%. Six of the nine studies that analysed the effects on potential ADEs showed a significant relative risk reduction of 35% to 98%. Four of the seven studies that analysed the effect on ADEs showed a significant relative risk reduction of 30% to 84%. Reporting quality and study quality was often insufficient to exclude major sources of bias. Studies on home-grown systems, studies comparing electronic prescribing to handwriting prescribing, and studies using manual chart review to detect errors seem to show a higher relative risk reduction than other studies. Concluding, it seems that electronic prescribing can reduce the risk for medication errors and ADE. However, studies differ substantially in their setting, design, quality, and results. To further improve the evidence-base of health informatics, more randomized controlled trials (RCTs) are needed, especially to cover a wider range of clinical and geographic settings. In addition, reporting quality of health informatics evaluation studies has to be substantially improved.


Medical errors are an important factor that influences the quality of patient care. According to Barach et al., nearly 100,000 individuals per year in the US die of preventable medical errors.1

Medication errors have been identified as a major type of medical errors. The Council of Europe2 and the British Department of Health3 defined medication errors as “any preventable event that may cause or lead to inappropriate medication use or patient harm … .” The Institute of Medicine reports that a hospital patient can expect on average to be subjected to more than one medication error per day.4,5 Medication errors can lead to adverse drug events (ADEs) that are defined as “any response to a drug that is noxious and unintended… .”3 A report from the Institute of Medicine that was published in 1999 stated that annually in the US 7,000 deaths can be associated with medication errors.6

In light of these figures, it is not surprising that the British Department of Health recommends the wider use of electronic prescribing to reduce the risk of medication errors.3 Electronic prescribing is supported by Computerized Physician Order Entry (CPOE) systems. The term CPOE refers to a variety of computer-based systems for ordering medications, which share the common features of automating the medication ordering process.7 The CPOE systems can range from systems that only provide a list of possible medications that the physician can then choose from, to systems providing varying levels of decision support, including checks of drug-drug interactions, drug-allergy contraindications, or checks of prescriptions concerning the patient's recent laboratory results. All those checks lead to alerts and reminders given to the ordering physician in case problems are detected.

In the scientific literature, many studies investigate the effects of electronic prescribing on the quality of patient care. Many studies showed a reduction in medication errors (such as Bates et al., 1999),8 while others described negative effects such as the facilitation of medication error risks9 or an increase in mortality.10 The available reviews on this topic seem to not come to an unambiguous conclusion on the overall effects of electronic prescribing, concluding that published evidence is often insufficient.1116 None of the published reviews has—to our knowledge—tried to quantitatively summarize the available evidence on the effects, and to systematically investigate the factors contributing to those effects.

Therefore, we performed a systematic and quantitative review to determine the effect of electronic prescribing on the risk of medication errors and adverse drug events (ADEs), and analyzed factors for those effects. This review should inform the health informatician of the effects that can be expected of electronic prescribing, and on the factors that influence the success of such systems. We see this review as a contribution to support evidence-based health informatics.17


Identification of Trials

We searched MEDLINE (1966 to April 2006) and EMBASE (1976 to April 2006) for studies on electronic prescribing that evaluated its effect on medication errors and ADEs. We combined MeSH terms, such as “Medical Order Entry Systems,” “prescriptions, drug,” “drug therapy, computer-assisted” with general search terms such as “order entry,” “CPOE,” “POE,” “order communication,” “prescription system,” “drug prescription,” “prescribing,” “ordering,” and “computerized reminders.” Those search terms were combined with terms that searched for evaluation studies that were taken from an earlier literature survey.18 The complete query is available upon request.

We also searched the Cochrane Database of Systematic Reviews and examined the reference lists of the relevant reviews in order to find further studies. Finally, we performed a hand search of the Journal of the American Medical Informatics Association (1994–2006), of the International Journal of Medical Informatics (1997–2006), and of the Methods of Information in Medicine (1990–2006). We completed this by hand-searching the references of retrieved study papers. We did not restrict the search to any single language.

Inclusion Criteria and the Selection of Studies

Intervention: We included studies wherein the intervention was electronic prescribing. As an intervention, we included all the computer-based application systems to order drugs that are used at the point of care. We included electronic prescribing systems independent of the level of decision support that they provided (e.g., with or without alerts on drug-drug interaction), and for all types of drugs. We excluded the studies that used electronic prescribing only for ordering diagnostic tests or therapeutic procedures.

Control: We only included studies wherein either an electronic prescribing system was compared with handwritten ordering, or where an electronic prescribing system with a more sophisticated functionality (e.g., a drug-drug interaction alert) was compared with another less sophisticated system.

Population and setting: We included studies where physicians were the primary users of the electronic prescribing system. We excluded studies where other groups (e.g., nurses, pharmacists) were the primary users. We included all clinical settings such as outpatient care, inpatient care, and intensive care. We included all patient groups.

Study design: We included randomized controlled trials, non-randomized controlled trials, before-after trials, as well as time-series analysis with multiple measurements. We only included field studies and excluded all lab studies and simulation studies.

Outcomes: We included studies that evaluated the effect of electronic prescribing on medication errors, potential ADEs, and ADEs. We defined medication error as all errors in the process of ordering, transcribing, dispensing, administering, and monitoring medication. This included an inappropriate drug, dosing, frequency, route, or timing (when related to patient safety), involving problems such as illegible or unsigned orders, and problems related to drug-allergy, drug-drug, drug-lab, and other interactions. Potential ADEs were defined as a medication error with significant potential to harm a patient that may or may not actually reach a patient. Adverse drug events (ADE) were defined as patient injuries resulting from drug use. We excluded studies where medication errors or ADEs were not the primary focus of the study, and studies that were still ongoing. We excluded papers if groups were definitely not comparable. If the data reporting was unclear, we contacted the authors and requested further information.

Data Extraction and Study Quality Assessment

We extracted data from the text, tables, and graphs of the original publications. Two reviewers (EA and CM) examined the data and reached consensus after discussion. In addition, one reviewer (PSI) independently reviewed all the extracted data. All of the cases of discordant data were resolved by discussion.

To detect medication error rates and ADE rates, we used the definition that is provided in each paper (please see Table 3, available online at www.jamia.org, for the definition of medication error and ADE for each respective paper).

When no absolute numbers were provided for medication error or ADE, we calculated these numbers based on the given data (e.g., if the frequency of ADEs was only given per 1.000 patient-days, the absolute ADE number could be calculated from the number of patient-days). To determine the study size, we used the number of orders as an observation unit of the analysis. If the number of orders was not available, the number of patients or patient-days was used. If multiple data were reported in a study (e.g., in time-series analysis), we used the data of the last reported measurement.

We classified the functionality of the CPOE system either as

  • no decision support: selection of drugs from a list, information on available doses and on costs, access to drug monographs, no further decision-support;

  • limited decision support: evidence-based patient-specific recommendation of a drug, dosing, frequency etc.; or

  • advanced decision support: at least some drug-allergy, drug-drug interaction, drug-lab, or other patient-specific alerts.

All results were reported in systematic evidence tables. The study quality was assessed by using a checklist (please see Table 1, available online at www.jamia.org), which was developed based on a 16-item assessment tool by the German Scientific Working Group Technology Assessment for Health Care that was applied independently by two reviewers (PSI, EA). Differences in judgment were then solved by discussion.

Statistical Analysis

For each study, the risk ratio (RR) with its 95% confidence interval (CI) was calculated by comparing medication error rates, potential ADE rates, and ADE rates between the intervention and comparison group. If available, the number of orders was used as the denominator. Otherwise, the number of patient-days or the number of patients was used as the denominator.

We used a graphical approach based on forest plots to perform sub-group analysis and arranged studies by increasing risk ratios within subgroups. Subgroups were a priori defined as potentially relevant such as the clinical setting (inpatient, outpatient, or intensive care), patient group, type of drugs, type of system (home-grown or commercial), functionality (no, limited, or advanced decision support), study design, and method to detect errors.

All the analyses were performed with the software package STATA 9.2 (StataCorp, College Station, Texas, USA).


Overall, we identified 172 evaluation studies and 15 systematic reviews. From the 172 evaluation studies, 27 studies met all the inclusion criteria and were, therefore, included in a detailed review.8,1944 For details, see Figure 1, available online at www.jamia.org. We obtained additional data from eleven authors.

Characteristics of the Trials

Table 2 shows the general characteristics of all the analysed 27 studies.

View this table:
Table 2

Characteristics of 27 Studies on Electronic Prescribing

PaperTime of studyName of hospitalClinical settingLevel of carePatients included in studyDrugs included in studyName of systemType of systemElectronic prescribing introduced since whenDescription of functionalityType of functionalityControl situation
Bates (1998)1993–1995Brigham and Women's Hospital, Boston, USASix intensive, medical and surgical units in a 700-bed academic tertiary-care hospitalNormal careNo restrictionsNo restrictionsBICSHome grownElectronic prescribing introduced as part of BICS since 1993 with increasing functionalityCPOE system with default doses, display of lab results, consequent orders, limited drug-allerg, dru-drug interaction checking and drug-laboratory checkingAdvanced DSSOrders were handwritten in order books
Bates (1999)1992–1997Brigham and Women's' Hospital, Boston, USAOne intensive and two general medical care units in a 700-bed academic tertiary-care hospitalNormal careNo restrictionsNo restrictionsBICSHome grownElectronic prescribing introduced as part of BICS since 1993 with increasing functionality(Period 3): CPOE system with display of lab values, drug-allergy checking, redundant medication checking, drug-drug interaction checking, drug-lab checking, order setsAdvanced DSSOrders were handwritten in order books
Bizovi (2002)1999–2000Oregon Health & Science University, USAEmergency department of a tertiary care teaching hospitalNormal careEmergency care patientsNo restrictionsEmSTAT, CyberPlus Corporation (in use since 1997)CommercialElectronic prescribing was introduced in the emergency department in 2000Prescription writing module with display of patient name, age, sex, patient's weightNo DSSHandwritten prescriptions
Chertow (2001)1997 - 1998Brigham and Women's Hospital, Boston, USAMedical, surgical, neurology, and obstetrics and gynaecology services of 720-bed urban tertiary care teaching hospitalNormal carePatients with renal insufficiencyRenally cleared or nephrotoxic medicationsBICSHome grownElectronic prescribing introduced as part of BICS since 1993 with increasing functionalityDecision-support system that calculates estimated creatinine clearance and proposes a recommended dose and frequency for the ordered drugLimited DSSCPOE without special advice for patients with renal insufficiency with regard to dose and frequency
Colpaert (2006)2004Ghent University Hospital, Belgium22-bed intensive care unit at tertiary care hospitalIntensive careIntensive care patientsNo restrictionsCentricity Critical Care Clinisoft, GE Healthcare EuropeCommercialElectronic prescribing was implemented in the intensive care unit 10 months before beginning of studyICU with incorporated CPOE: Protocol-based recommendation on drug, dose and frequency, alerts to important drug-drug interactions and drug-related complications, highlighting of patient allergy statusAdvanced DSSPaper-based ordering
Cordero (2004)2001–2002Ohio State University Medical Center, USANeonatal intensive care unitsIntensive careVery-low-birth-weight infantsCaffeine and GentamicinInvision 24, Siemens Medical SolutionsCommercialElectronic prescribing was introduced to selective services in February 2000 and introduced to the study units in March 2002CPOE with best-practice based order sets, drug-allergy checks, drug-drug interactions, order duplication, corollary orders, weight-based dosage, maximum dosage, drug-route restrictionAdvanced DSSAll order were entered manually in the chart kept at the patient bedside
Evans (1998)1992–1995LDS Hospital, Salt Lake City, Utah, USA12 bed shock trauma respiratory intensive care unitIntensive careIntensive care patientsAntiinfectivesHELPHome grownThe antiinfectives management program was implemented in the study unit in 1994Computerized antiinfectives-management program with advice on antiinfective regiment (drug, dosing, dosing interval) based on clinical data and available evidence, drug-allergy checks, drug-drug interactions, information on toxicity and costsAdvanced DSSNo antiinfectives-management program was used
Feldstein (2006)2000–2004Kaiser Permanente Northwest, a health maintenance organization, USA8 primary care clinics with 126 primary care cliniciansNormal careNo restrictionsWarfarinEpicCare EMR, Epic Systems* (in use since 1996)CommercialWarfarin alerts were implemented in January 2003CPOE system with alerts on five warfarin-interacting drugsAdvanced DSSCPOE without alerts
Fraenkel (2003)1995–1998Royal Brisbane Hospital, Queensland, Australia12-bed adult general intensive care unit in a 790-bed tertiary referral teaching hospitalIntensive careIntensive care patientsNo restrictionsCareVue, Philips Medical SystemsCommercialIntensive care system with electronic prescribing was introduced on study unit in November 1996Medication administration record module with pre-defined dose ranges, drug scheduling and drug preparation notes, prompt for drug administrationNo DSSPaper-based documentation
Galanter (2005)2002–2003University of Illinois Hospital and Medical Center, USANot further describedNormal carePatients with renal insufficiencyRenally cleared or nephrotoxic medicationsMillenium and Discern Expert, Cerner CorporationCommercialAlerts were implemented during study periodCPOE system that calculates the creatinine clearance and checks for potentially contraindicated drugsAdvanced DSSCPOE without alert on potentially contraindicated drugs for patients with renal insufficiency
Gandhi (2005)1999–2000Four Boston adult primary care practices affiliated with an academic medical center, USA4 primary care practicesNormal careOutpatients over Age 18No restrictionsOne commercial, one home-grown systemMixedNo further information providedComputerized prescribing with printed prescriptions, required fields and non-mandatory default doses, without any automatic checksNo DSSHand-written prescriptions
Igboechi (2003)1999–2002Metropolitan Hospital Center, New York, USAInpatient and outpatient services of 350-bed acute tertiary care hospitalNormal careNo restrictionsNo restrictionsUlticare System Database, Per Se TechnologiesCommercialElectronic prescribing was introduced in June 2001 after recent upgrade to a new clinical systemCPOE system with drug-allergy alerts, drug interaction alerts, dosage range checks, lab values, and drug information resourceAdvanced DSSHandwritten orders
Kim (2006)2001–2004John Hopkins Children's Center, Baltimore, USAPaediatric oncologyNormal careChildren with neoplastic disordersChemotherapeuticaRxTFC Pharmacy Information Systems, GE Medical Systems Information TechnologiesCommercialElectronic prescribing was introduced as part of the available pharmacy system in 2002CPOE system with dosing support and drug-allergy checking*Advanced DSSPaper-based order entry
King (2003)1993–1999Children's Hospital of Eastern Ontario, CanadaThree medical and two surgical wards of a tertiary care paediatric hospitalNormal careChildrenNo restrictionsSunrise Clinical Manager, EclypsCommercialElectronic prescribing was introduced, with periodic upgrades, on the two intervention wards in 1996CPOE without decision support (no further information provided)No DSSHandwritten order entry
Kirk (2005)2003National University Hospital, SingaporePaediatric unit in a university teaching hospitalNormal carePaediatric patients (< 16 years of age)Paracetamol and promethazineNot specifiedHome grownExisting clinical decision support system (in use since 2000) system was modified in March 2003 to also support electronic prescribing for paediatric departmentsComputer calculated dosing based on weightLimited DSSTraditional computer-based prescription, no dosing advice
Koide (2000)1994–1996University of Tokyo Hospital, Japan1040-bed tertiary care and teaching hospitalNormal carePatients with psoriasisEtretinate, a medication indicated for psoriasisMedical information system (in operation since 1973)UnknownElectronic prescribing in operation since 1988, alerts for etretinate added in 1995Alert system monitors ALT values and AST valuesLimited DSSCPOE without alerts
Maurer (2003)1999–2000Centre Hospitalier Le Raincy-Montfermell, Montfermell, FranceGeneral Pneumology Unit with 60 bedsNormal carePneumology PatientsNo restrictionsExcel-based applicationHome grownElectronic prescribing introduced in January 2000Prescription sheet with drug list and recommended dosing based on weight and renal functionLimited DSSHandwritten prescriptions
Mitchell (2004)2002Southmead Hospital, Bristol, UKGeneral surgeryNormal careNo restrictionsNo restrictionsClinical Manager 3.0A, iSoft UK PLCCommercialElectronic prescribing introduced in general surgery as 3-months-pilot in 2002Comparing unit with electronic medication administration record and electronic prescribing with areas of hand-written prescriptionsNo DSSPaper-based ordering
Mullett (2001)1998–1999Primary Children's Medical Center, University of Utah, USA26-bed paedriatric intensive care unit of a 232-bed primary children's medical centerIntensive careChildrenAntiinfectiveHELPHome grownPediatric antiinfectives decision support tool was implemented on intensive care units in January 1999Computerized antiinfectives-management program with dosing recommendations based on evidence and clinical dataLimited DSSHandwritten orders
Oliven (2005)2001*Bnai-Zion Medical Center, Haifa, IsraelTwo 44-beds general medical wards in an acute care university hospitalNormal careNo restrictionsNo restrictionsUnknownHome grownElectronic prescribing was introduced three years before the studyCPOE system with administration reminders, reminders for blood-drug-level test or further monitoring, extended drug information, treatment guidelines, drug-drug interaction, drug-allergy, drug-disease, drug-lab alerts, access to blood test resultsAdvanced DSSHandwritten drug orders
Pestotnik (1996)1988–1994LDS Hospital, Salt Lake City, Utah, USA520-bed acute-care referral hospitalNormal careNo restrictionsSurgical antibioticsHELPHome grownSurgical antibiotics management modules were developed since 1985Antibiotic management program with patient-specific guideline-based clinical decision supportAdvanced DSSNo antibiotic management program available, computerized ordering mainly related to nursing orders and supply orders*
Peterson (2005)2001–2002Brigham and Women's' Hospital, Boston, USAMedical, surgical, neurology, and gynaecology services of tertiary care academic hospitalNormal careElderly patients (65 years and older)Psychotropic medicationBICSHome grownDecision-support for psychotropic medication was developed and implemented for the studyCPOE with evidence-based geriatric-specific recommendations for dosing and substitutionLimited DSSCPOE with dosing suggestions and recommendations for substitutions
Potts (2004)2001–2002Vanderbilt Children's Hospital, Nashville, USA20-bed paediatric critical care unit in a tertiary-care children's hospitalNormal careChildrenNo restrictionsWizOrder (developed in 1994)Home grownCPOE was implemented in critical care unit in December 2001CPOE with decision support such as drug allergy alerts, dose checking, drug interaction alerts, clinical pathways-based order sets, links to drug monographs, drug-lab alertsAdvanced DSSNo CPOE
Shulman (2005)2001–2002University College Hospitals London, Middlesex hospital, London, UKIntensive Care UnitIntensive careIntensive care patientsNo restrictionsQS 5.6 Clinical Information System, GE HealthcareCommercialQS with electronic prescribing was introduced in April 2002CPOE component without decision supportNo DSSHand-written drug charts
Spencer (2005)2002–2003University of North Carolina, Chapel Hill, USATwo general medicine floors of a 688-bed academic hospitalNormal careNo restrictionsNo restrictionsSiemens Medical Solutions, Malvern, PACommercialSiemens CPOE system was implemented on two study units in October 2002 resp. January 2003Patient allergy screening was partly in use*Advanced DSSNo CPOE
Tamblyn (2003)1997–1998General Practicioners in Quebec, Canada107 primary care physiciansNormal careElderly patients (66 years and older)No restrictionsUnknownCommercialElectronic prescribing was introduced to intervention group physicians two months after randomizationHealth-record software which documented medication prescribed, offering drug-disease contraindications and information on previous drugsAdvanced DSSNo computerized decision support
Teich (2000)1991–1995*Brigham and Women's' Hospital, Boston, USA720-bed urban academic medical centerNormal careAdultsNo restrictionsBICSHome grownElectronic prescribing introduced as part of BICS since 1993 with increasing functionalityCPOE system with dosing recommendation, drug-allergy, drug-drug interactions, duplicate medications, possible alternative medication, consequent order recommendationAdvanced DSSPaper-based ordering
  • * this information was provided by the author upon request.

The majority of studies were conducted in the United States and in normal inpatient care units. Most studies were unspecific concerning the included patients and concerning the ordered drug. Half of the studies evaluated a commercial system, and half evaluated a home-grown, self-developed system. All of the systems offered a display of basic patient data (name, age), a list of drugs from which a provider can choose, and a list of potential doses for a chosen drug. Half of the studies used a system with advanced decision support, the others used systems with no or limited decision support. In most trials, electronic prescribing was compared to handwritten ordering.

Table 3, available online at www.jamia.org, shows the further study characteristics of the 27 studies, showing that most of them were before-after studies (including time-series analysis), with only two studies being randomized controlled trials.

Of those 27 studies, 15 only evaluated medication errors, two were only on ADE, and 10 reported both on medication errors and ADEs. Table 4, available online at www.jamia.org, shows the evidence table for outcomes.

Effect on Medication Errors

Twenty-five studies reported on the risk of medication errors (Figure 2). Twenty-three of these studies showed a significant relative risk reduction, with a risk ratio between 0.01 and 0.87. This indicates a relative risk reduction for medication errors of 13% to 99%. One study reported a statistically significant increase of 26% for the risk of medication errors (Spencer 2005).42 In this study, there was no connection between the CPOE system and the pharmacy system, so that orders had to be entered twice, in turn leading to transcription errors in the intervention group. In addition, the study authors only analysed those errors that were reported by the staff members themselves (voluntary reporting system)—this constitutes a potential detection bias. The one study (Mitchell 2004)35 showing inconclusive results (RR = 1.02, 95%–CI: 0.88; 1.19) was one of the smallest studies in this review, with only 320 orders analysed.

Figure 2

Risk ratios of 25 studies analyzing the effect of electronic prescribing on medication errors.

Effect on Potential Adverse Drug Events

Nine studies reported on the risk of potential ADE (Figure 3). Six of these studies showed significant relative risk reduction, with a risk ratio between 0.02 and 0.65. This indicates a relative risk reduction for potential ADEs of 35% to 98%.

Figure 3

Risk ratios of nine studies analyzing the effect of electronic prescribing on potential ADEs.

Two studies showed inconclusive effects (Bizovi 2002 and Gandhi 2005).20,28 Both of these studies were rather small studies. The remaining also very small study by Mitchell 2004 showed a significant increase in the risk for potential ADEs.35

Effect on Adverse Drug Events

Seven studies reported the risk of ADE (Figure 4). In one of them (Shulman 2005),41 no events occurred in the intervention or in the comparison group and, therefore, this study was excluded from further analysis.

Figure 4

Risk ratios of seven studies analyzing the effect of electronic prescribing on ADEs. Shulman was excluded, because there was no event in either the intervention group or the comparision group.

Four of the six remaining studies showed a significant relative risk reduction for ADEs, with a risk ratio between 0.16 and 0.70. This indicates a relative risk reduction for ADEs of 30% to 84%. One study (Mullett 2001)36 showed a not statistically significant relative risk reduction of 13%, and the remaining study (Bates 1998)19 showed a small, not statistically significant increase of 9%.

Sub-group Analysis

A graphical sub-group analysis was only performed for those 25 studies that focused on medication errors.

The sub-group analysis comparing medication error reduction by commercial systems versus home-grown systems indicated a higher relative risk reduction by home-grown systems (Figure 5).

The sub-group analysis for those seven studies that compared advanced electronic prescribing with limited electronic prescribing and those 18 studies that compared electronic prescribing with handwritten ordering showed a higher relative risk reduction for the comparison made with handwritten ordering (Figure 6).

Figure 5

Sub-group analysis of the effect of home-grown versus commercial systems on the relative risk reduction of medication errors (n=25).

Figure 6

Sub-group analysis of the type of comparison group on the relative risk reduction of medication errors (n=25).

The 14 studies with an advanced decision-support reported a higher relative risk reduction than the 11 studies with limited or no decision-support (Figure 7). Most studies with limited support, however, were compared to computer-based ordering, whereas most studies with advanced decision-support or without decision-support were compared to handwritten ordering.

Figure 7

Sub-group analysis of the effect of the level of functionality on the relative risk reduction of medication errors (n=25).

The sub-group analysis comparing different methods to detect errors showed higher relative risk reductions with the manual chart review of prescriptions (11 studies) than with the automatic database analysis of prescriptions (6 studies) (please see Figure 8, available online at www.jamia.org).

The other graphical sub-group analysis did not indicate differences between groups: The risk ratios between groups seem to be similar for the level of care (normal care versus intensive care), patient groups included (elderly, children, unspecific), type of drugs included (specific or unspecific), or study design (before-after trial, controlled trial, RCT).

Study and Reporting Quality

The reporting quality was poor for most of the studies. The details are shown in Table 5, available online at www.jamia.org. For example, many studies did not clearly specify the inclusion or exclusion criteria of the participating institutions or patients, did not report on the baseline characteristics of participants or as to whether the comparison and intervention groups were treated similarly (with the exception of the intervention), or did not comment on missing values or drop-outs.

In addition, only two studies were randomized trials, and few studies had comparison groups recruited over the same period. Only in half of the studies, the outcome seemed to be measured validly and reliably, however, the measurement was mostly not blinded. Nearly all the studies used adequate hypothesis tests and reported on the statistical precision of the main outcomes, but only less than half of the studies attempted to adjust for confounding or clustering. Most studies were single centre studies, so that the generalisability of the results to other centres, especially of a different type, is unclear.


Main Findings

Thwenty-three of the 25 studies that analysed the effects on the medication error rate showed a significant relative risk reduction of 13% to 99% for medication errors. Six of the nine studies that analysed the effects on potential ADEs showed a significant relative risk reduction for potential ADEs of 35% to 98%. Four of the seven studies that analysed the effect of electronic prescribing on ADEs showed a significant relative risk reduction for ADEs of 30% to 84%. These findings, that are in line with results from other reviews, indicate that electronic prescribing can substantially reduce the risk for medication errors, potential ADEs, and ADEs.12,13,16,4547 The low number of studies focussing on ADE may be due to the fact that medication errors can be detected more easily than ADEs.16

From a clinical perspective, the observed effect sizes (e.g., up to 99% reduction of medication errors, up to 84% reduction of ADEs) seems to be rather large. However, the medication errors and ADEs are just surrogate outcomes, and are not necessarily directly related to changes in the patient-relevant medical outcomes.48 The actual improvements in medical outcomes (e.g., reduction in mortality rates or hospitalisation days) have not yet been sufficiently analysed by quantitative, controlled trials.15,45,48 One exception is the study by Han et al.10 that observed an increase in mortality after the introduction of a CPOE system (these heavily discussed findings could not be reproduced later on by others such as Del Baccaro et al.49). Altogether, more systematic trials looking at patient-relevant medical outcomes should be conducted in the future. Furthermore, more studies should also be conducted to analyse the costs for any benefit that may be obtained. For this review, we concentrated on medication errors and ADEs, as a sufficient amount of published evidence is available.

Results of the Sub-group Analysis

The included studies showed substantial heterogeneity (see their detailed description in Table 2 as well as in Table 3, available online at www.jamia.org). This reflects the diversity of electronic prescribing systems and their respective use.

The sub-group analysis indicated that the level of care, as well as the included patient groups and drugs did not affect the overall risk ratios. The graphical presentation, however, indicated that home-grown systems show a higher relative risk reduction than commercial systems. This is not surprising, as home-grown systems can be modified and adapted easier and more quickly to the local needs. Rigby50 calls this “alpha sites” and argues that those sites are atypical concerning larger technical, emotional, and financial support.

Also, studies comparing handwritten prescribing with electronic prescribing seem to show a higher relative risk reduction than those studies comparing groups with a different level of electronic ordering. This is also not surprising, as certain error types such as illegible orders, that are often present in handwritten prescribing, are completely removed by electronic prescribing.

Electronic prescribing systems with advanced decision support seemed to show a higher relative risk reduction compared to those with limited or no decision support. It is here that the patient-specific alerts seem to best support the quality of prescribing.

Finally, different methods for detecting medication errors seem to show a different relative risk reduction. Using a manual chart review, relative risk reduction seems to be highest. However, chart review can hardly be blinded, and reviewers may be biased towards CPOE.

Study Quality

Overall, the reporting quality and study quality of the included studies were not always satisfactory.

One major problem was that many studies did not provide sufficient information in order to adequately assess the comparability of the intervention and comparison groups (e.g., by not reporting baseline characteristics, by not reporting about adjunct initiatives such as training sessions or about drop-outs etc.). This made it difficult to analyse as to whether the differences found between the study groups really originated from electronic prescribing, or from other factors. Subsequently, about two-thirds of the studies did not attempt to adjust for possible confounding factors. Then, many studies also used designs of lower level of evidence such as before-after trials, here it is unclear whether any context such as staffing or workflow of the study departments may have changed over time, in turn influencing the observed effects. The same is true for any non-randomized allocation of clinicians and/or patient to study groups. This all affects the validity of the analysed studies. We dealt with this by describing in detail the individual study quality (see Table 5, available online at www.jamia.org), and by excluding those studies that seem to have non-comparable groups.51,52

There were other limitations of the included studies: Four studies analysed the data from voluntary error reporting systems (e.g., nurses and physicians voluntary reported on errors), which may underestimate the real number of errors and ADE in those studies (detection bias). Other studies retrospectively analysed routinely collected information on medication errors and ADEs. Here, it is partly unclear as to how they assessed the quality of this routine data. Overall, only a few studies provided information on the objectivity and reliability of the instruments that were used to (manually or automatically) detect errors.

Summarizing, the reporting and study quality were not always adequate. This must be taken into account when interpreting the results of the studies. Efforts should be made to improve the reporting quality and study design and analyses of evaluation studies. Publication guidelines that are comparable to CONSORT53 or STROBE54 should be developed, taking into account the often quasi-experimental design of the health informatics evaluation studies. One example for such publication guidelines for health informatics evaluation studies are the STARE-HI guidelines (Standards for Reporting on Evaluation Studies in Health Informatics) that will shortly be published.55

Limitations of the Review

Our review may present some limitations.

First, the published trials on electronic prescribing differ substantially in their respective terminology. We used many different Keywords to search for trials; searched in several databases as well as for non-English papers; and applied hand-searching. Nevertheless, we may have overlooked some individual studies. In addition, there is the possibility of publication bias, as we have only included published studies.

Second, the definition of the measured outcome variable differed between the studies. For example, some studies included the legibility of orders in the definition of medication, and others did not. This can affect the measured medication error rate.

Third, when several end-points were reported (e.g., in a time-series analysis), we only included the final endpoint. This may overestimate the positive effects of electronic prescribing, as the intermediate results partly show less (or even negative) effects during an early introduction period. In addition, our analysis relied on the assumption that one order could have maximal one error. However, this was not explicitly reported by several studies.

Fourth, our categories that were used to stratify the sub-groups might have been too crude. For example, the study design was very heterogeneous and the diversity might not have been represented adequately by the categories we chose. A multivariate sub-group analysis controlling for the multiple factors simultaneously was not performed because of the limited number of studies. Then, further factors such as the organisation of medication processes in the various settings, age and computer knowledge of users, management support, or the quality and usability of the CPOE system may further affect the effects. Information on those items was not included in most studies and could, therefore, not be used for sub-group analysis.

Fifth, the rather large percentage of home-grown systems in our review may have biased this review towards more positive results. Moreover, 8 of the studies included came from just two sites: the Brigham and Women's Hospital, Boston, MA with the BICS system, and the LDS Hospital, Salt Lake City, Utah, with the HELP systems. Therefore, we cannot exclude a selection bias, with the observed effects not being representative for general routine clinical settings. Some studies on BICS and HELP may not be completely independent of each other (we therefore excluded Evans et al., 1994),56 but when they used different outcome criteria or addressed different patient populations, we decided to include them.

Sixth, we were unable to use meta-analysis to pool the effect sizes, given the large heterogeneity among the studies. Meta-analysis would help to quantify the overall effect, and has already been performed e.g., for computer-assisted management of anticoagulant therapy57 or for clinical reminder systems for preventive care,58 but needs a sufficient number of homogeneous studies. We therefore conducted a graphical sub-group analysis—even this, however, should be viewed with care, given the tremendous variability in study design and in the methodological rigor of the included studies.

Seventh, most of the evaluated systems have only been implemented shortly before or during the study, and sometimes only as pilots in selected areas. This may have led to erroneous estimations concerning the long-term effects of electronic prescribing. More long-term studies seem to be needed.

Finally, researchers such as Berger15 note that electronic prescribing systems could increase the number of medication errors and ADEs when not appropriately designed and implemented. There is, in fact, increasing evidence from (mostly qualitative) research pointing to risks with electronic prescribing by CPOE systems. Koppel et al.,9 e.g., described an increase in medication errors risks by CPOE. Campbell et al.59 identified categories of unintended consequences of CPOE. This (qualitative) evidence is, however, not (yet) reflected in the results of quantitative controlled evaluation studies and therefore also not in our review. The discussion of the reasons for this (such as a possible publication bias) is outside the scope of the present paper. For a more detailed discussion see Ammenwerth et al.60

Implication and Conclusion

After having been envisioned by Morris Collen in 1970 (“Physicians should enter their medical orders directly into the computer”),61 electronic prescribing systems have now been in routine use for approximately 15 years. Our review is, to our knowledge, the first that quantitatively calculates and compares the risk ratios for each evaluation study. We also conducted a sub-group analysis, hereby analysing factors that influence the effects of electronic prescribing; and we analysed and discussed carefully the heterogeneous and often weak quality of published studies.

Our results indicate that electronic prescribing seems to be a useful intervention for reducing the risk of medication errors and ADEs. We found that there is especially good evidence for a positive effect of electronic prescribing offering advanced decision support functionality in hospital settings. Less evidence is available for systems outside the U.S., for primary care settings, and for specific commercial systems. Therefore, more studies are necessary.

The study quality was often weak, with many before-after trials and an unclear comparability of the study groups. More randomized controlled trials from more sites are needed to further improve the evidence and to identify the setting that those systems are most useful in. Researchers planning comparable studies are advised to focus early on the issue of study quality and study validity, to avoid as many sources of bias as possible (see also Brender, 200262). A larger number of high-quality studies for certain settings and patient groups would also allow for the use of meta-analysis to pool the effect sizes. In addition, the reporting quality of health informatics evaluation studies has to be substantially improved; here, the initiatives for publication guidelines such as STARE-HI55 are urgently needed. All this should help to increase our knowledge on electronic prescribing and globally increase the evidence-base of health informatics.17

Systematic reviews of quantitative trials can indicate which effect can be obtained from an intervention. They do not show, however, how to implement electronic prescribing. It seems obvious that the quality of the implementation process has an impact on the success of a CPOE implementation.63 Insufficient implementation planning, or systems that are not integrated into the general information systems, may lead to negative effects on the process and even the outcome of care.6467 Negative effects of electronic ordering were identified only in two studies in our review, even when increasing evidence from qualitative research points to possible adverse effects of electronic prescribing. Each implementation should be carefully monitored concerning an increase in medication errors and ADEs, and more qualitative research should be conducted to assess the reasons for the respective success or failure, and to guide further successful implementation processes.


The authors thank all reviewers for their helpful comments on earlier versions of this paper.


View Abstract