OUP user menu

Improving the utilization of admission order sets in a computerized physician order entry system by integrating modular disease specific order subsets into a general medicine admission order set

Rajika L Munasinghe , Camelia Arsene , Tarun K Abraham , Marwan Zidan , Mohamed Siddique
DOI: http://dx.doi.org/10.1136/amiajnl-2010-000066 322-326 First published online: 1 May 2011


Case description We evaluated the effects of integrating order subsets for the most common medical diagnoses into a general medical admission order set of our electronic medical records (EMR) in order to improve order set integration by clinicians.

Methods of implementation We identified the most common primary and secondary diagnoses for patients admitted to our medical service and developed order subsets comprising only of the orders necessary for the management of these individual diagnoses. Using the capabilities of our computerized physician order entry (CPOE), we nested these order subsets into the general order set and evaluated the resulting change in order set utilization by our clinicians.

Example and observations The total number of order sets used by clinicians in all departments increased fivefold during the 16-month period following the implementation of the integrated order sets in July 2008. A before and after time series was used to analyze the trend in increased order set usage and showed an effect of the intervention (p=0.023).

Discussion Integration of disease specific order subsets into a single general admission order set significantly improved the overall adoption of order sets by clinicians. This provides health care systems with the opportunity to improve patient safety and implement evidence based care in clinical practice.

  • Computerized physician order entry
  • electronic medical record
  • standardized order
  • nested order
  • usual
  • CPOE
  • improving the utilization of admission order sets
  • machine learning

Case description

General and disease specific admission order sets are used by many hospitals. They are expected to reduce the time and effort needed to place common orders and reduce the possibility of missing important orders that improve patient safety and compliance with core measures related to quality of patient care. Previous studies have demonstrated that such admission order sets improve immunization rates and appropriate antibiotic use and probably reduce length of stay in hospitalized patients.1

We describe our experience with increasing electronic admission order set use by integrating disease specific order subsets into a standard admission order set in the medicine service of a busy academic medical center.

Sinai-Grace Hospital (SGH) is a 404-bed hospital with 700 physicians, 670 nurses, and more than 1800 support staff. SGH discharges more than 22 500 patients per year and its emergency department is among the busiest in Detroit, treating more than 84 000 patients a year.

The hospital uses the Powerchart electronic clinical information system developed by Cerner Corporation as an electronic medical records (EMR) system for documentation of clinical notes, diagnostic studies, and physician orders. Prior to the implementation of the EMR, paper based general admission order sets had been introduced but the utilization of these order sets was low.

Both paper based and electronic order sets have limitations. Similar to the inherent shortcomings of paper based order sets,2 lack of national guidelines for different issues such as computerized physician order entry (CPOE) based drug–drug interaction information, checking or recording of drug allergies electronically,3 implementation without organization standards, and clinical review, or not being properly maintained present a challenge for electronic order sets adoption.2 However, CPOE has been shown to have potential in reducing medical errors4 and costs,5 and improving the quality of care.6

At SGH, as in other organizations,7 ,8 evidence based order sets were developed with considerable input from specialists to ensure compliance with disease management guidelines and best practice, but the ability to accomplish these objectives was constrained by the low order set use by clinicians. We did not conduct a user satisfaction survey,9 but informal discussions with our staff showed high levels of dissatisfaction at the initial implementation of the order sets. Another challenge was that, although our patients were admitted with a principal diagnosis that required the use of the admission order set, management of several other conditions required the use of extra order sets or writing out additional orders using a regular order sheet.

Methods of implementation

The hospital implemented CPOE in May 2007 as part of the transition to the EMR. Prior to the implementation, all paper order sets were reviewed and equivalent electronic order sets developed and placed in a medicine order set catalog. The conversion to electronic order entry was implemented over a 24–48 h period and all paper based order entry discontinued shortly afterward. The most commonly used orders were pre-selected and personal order sets were also permitted. All medical staff received training in CPOE.

Despite several educational programs and repeated reminders, the use of the electronic admission orders sets remained low. An informal survey of house staff indicated that the utility of these order sets was limited because many of the patients admitted to the medicine service had several other conditions that required concurrent management during hospitalization. Clinicians preferred to build their own set of admitting orders rather that use one or more disease specific order sets. In order to address this need, a program was developed that allowed the disease specific order subsets to be integrated into a general order set that could be called up by the clinician admitting the patient based on the applicable diagnoses. These disease specific order subsets were different from the disease specific order sets because general admission orders such as diet, activity, and routine laboratory tests were not repeated within the subsets. The clinician would build the complete set of admitting orders relevant for the patient in a modular fashion by starting out with the general order set and calling up the disease specific order subsets in a stepwise manner.

This program was started in June 2008 with a complete analysis of the most common primary and secondary diagnoses of patients admitted to our hospital. Based on this information, the modular order subsets were developed using existing electronic and paper order sets with input from the respective specialists. For the initial implementation, disease specific orders were integrated into the general admission order set and named the comprehensive medical admission order set. The program was first implemented for the academic service in July 2008 and refined based on input from its users. After completing a few modifications based on clinician feedback, hospital-wide implementation was carried out in October 2008 and order set use was monitored using the reporting capabilities of the EMR.

Example and observations

A screenshot of the comprehensive general admit order set with the option to select the nested subsets based on patient co-morbidities is shown in figure 1.

Figure 1

Screenshot of the comprehensive medicine admission order set showing its nested components.

The number of electronic order sets used during the 16-month period following the implementation of the comprehensive medicine admission order set is shown in table 1 (July 2008–October 2009).

View this table:
Table 1

The number of electronic order sets used for different diagnoses after converting to computerized physician order entry

Order set name2008 Jul2008 Aug2008 Sep2008 Oct2008 Nov2008 Dec2009 Jan2009 Feb2009 Mar2009 Apr2009 May2009 Jun2009 Jul2009 Aug2009 Sep2009 Oct
Disease specific admission order set
  Adult sickle cell pain crisis1212152123441
  Asthma/chronic obstructive pulmonary disease exacerbation26313347302418412720433428427346
  Chest pain1539291135562221101119101116291237
  Gastrointestinal bleed17925841093749121412
  Glycemic control50284146191818142416181621161426
  Heart failure24283933231413222119263233154031
  Hypertension and cardiac medication232131221
  Renal failure25105107712795149151715
  Urinary tract infections/pyelonephritis1127113193109
Comprehensive medicine admission order set74205304275535888929115512221110111513451303123214281467

The total number of electronic order sets used each month increased from 349 in July 2008 to 1711 by October 2009. The number of newly implemented comprehensive nested medicine admission order sets used increased from 74 per month to 1476 per month during this period, almost entirely accounting for the fivefold increase in electronic order set use at our institution.

A before and after time series was used to analyze the trend in increased order set usage. A total of 30 monthly observations of order set usage were examined from May 2007 (when CPOE was first implemented) to October 2009. Analyses showed an effect of the intervention (p=0.023). We also developed an autoregressive integrated moving average (ARIMA) time series model adding a dummy variable that explained about 18% of the variation in the series, while the Ljung-Box statistic had a p value of 0.666, reinforcing the fact that the model was correctly specified (figure 2).

Figure 2

Time series analysis.

The impact of the comprehensive medicine admission order set on the use of disease specific order sets is best illustrated by the analysis of order set usage for patients admitted for non-acute coronary syndrome chest pain (figure 3). The use of order sets was comparatively high for this diagnosis even before the implementation of the comprehensive medicine admission order set, with the stand alone order set for chest pain used 151 times in July 2008. The use of this stand alone chest pain order set decreased to 30 by October 2009, while the use of the nested chest pain orders in the comprehensive medicine order set increased to 213 order sets per month.

Figure 3

Comparison of the nested and non-nested numbers of order sets for non-acute coronary syndrome chest pain.

A similar increase in nested order sets included in the comprehensive medicine order set was noted for other diagnostic entities (figure 4). The order set for community acquired pneumonia (CAP) showed the lowest utilization, while the comprehensive medicine admission order set without any of its nested components was used most often.

Figure 4

Number of nested order sets within the complete medical order sets for the Internal Medicine Department. CAP, community acquired pneumonia; COPD, chronic obstructive pulmonary disease.


The typical adult patient admitted to the medical service of an acute care hospital in the United States has more than one diagnosis that requires concurrent management, especially for older patients having multiple co-morbidities.10 ,11

Standard admission order sets are being used with increasing frequency and serve as a checklist to reduce the inadvertent omission of orders important for patient care, meeting key performance measures, and promoting best practice.12 The experience with adoption of standard admission orders by healthcare providers seems to vary by diagnosis and complexity of care.13 ,14 In a study conducted by McAlearney et al among pediatric patients, order set utilization rates were higher for diagnoses such as asthma (88%) and appendicitis (80%) but considerably lower for CAP.13 Our experience with adopting adult medicine admission order sets was even less encouraging prior to the implementation of the integrated order set, probably due to the multiple patient co-morbidities.

During the period that paper based admission order sets were implemented at our hospital, low adoption rates were attributed to the logistics of distributing, cataloging, and stocking these paper order sets throughout the hospital with inconvenient availability to the clinical staff. However, this limitation ceased to be a factor after converting to CPOE, after which immediate access to the entire catalog of standard admission order sets was made possible via a simple drop down menu. As others have found,13 ,1517 the continued under-utilization of these order sets after transition to EMR indicates that factors other than physical access to order sets contributed to clinician preference in using this utility.

During the planning phase of this project, we learned a few lessons by identifying several factors contributing to clinician dissatisfaction with this procedure:

  1. Many medical patients had more than one diagnosis at the time of admission and orders for these secondary diagnoses had to be placed separately or by selecting additional order sets for these diagnoses. When a second or third order set is used for this purpose, the clinician has to review and leave out duplicate orders that have already been entered as part of the first order set.

  2. It is also important to strike a balance between ensuring the medical order set is complete by including the common orders but not overburdening the user with an overwhelming choice of selections that are meant to address all eventualities.

  3. In designing this modular, subset based, comprehensive medical order set for our hospital, we tried to minimize these factors by using the general admission order set to include the most commonly needed orders and avoided repeating these orders in the disease specific subsets. Orders necessary to comply with best practice such as adult immunization, smoking cessation counseling, and pain management were all included in this set. Additionally, measures to reduce complications related to hospitalization such as venous thromboembolism and hospital acquired infections, were also incorporated. Order subsets for common co-morbidities such as hypertension, diabetes, and congestive heart failure (CHF) have been specifically designed to meet some of the best practice standards such as basal-bolus administration of insulin for improved glycemic control, hemoglobin A1c measurements, and treatment with ACE inhibitors for patients with CHF.

  4. We also found that use of the nested order subsets for CAP, stroke, and end-stage renal disease did not increase to the same extent as for other diagnoses. In the case of CAP and stroke, this is likely a result of the need to initiate the major portion of the orders for laboratory studies, imaging, and medication administration within very specific time limits, frequently when the patient is still in the emergency department.

The principal limitation of this descriptive study is that we are not able to establish that the use of standardized order sets resulted in improved patient outcomes and cost savings to the institution. Also, this is not a randomized trial, so other factors might have contributed to the increased order set usage. While a comparison of key indicators of patient outcome and cost of care among patients admitted with and without the use of order sets is feasible, random assignment was difficult due to the large number of clinicians involved in patient care. We did not have the necessary resources to conduct a randomized study and collect objective outcome data on our patients to determine if the increased order set use resulted in meeting treatment goals and better care, and we were not able to present data on the percentage of patients for whom the condition specific order sets might be appropriate.

Our strategy to use the functionality of CPOE of an EMR by designing a dynamic modular order set that can be customized to meet the specific needs of the patient being admitted has been well received by the majority of the clinicians at our hospital. Although there are limitations in regards to CPOE use such as time needed to learn the system and difficult user–system interactions,18 current evidence indicates that CPOE systems have the potential to reduce costs and improve the quality of care for our patients.1 ,16 ,19 ,20 Our paper describes a strategy to increase appropriate use of CPOE, as its benefits are unlikely to be achieved if CPOE is not used.

Competing interests


Provenance and peer review

Not commissioned; externally peer reviewed.


We would like to thank Dr Joann Riggins-Woodhouse and Mrs Rita Miller from the Electronic Medical Records Clinical Transformation team at Sinai-Grace Hospital, Detroit Medical Center, for their assistance with this project.


View Abstract