OUP user menu

A Randomized Trial of Electronic Clinical Reminders to Improve Medication Laboratory Monitoring

Michael E, Matheny , Thomas D. Sequist , Andrew C. Seger , Julie M. Fiskio , Michael Sperling , Don Bugbee , David W. Bates , Tejal K. Gandhi
DOI: http://dx.doi.org/10.1197/jamia.M2602 424-429 First published online: 1 July 2008


Objective: Recommendations for routine laboratory monitoring to reduce the risk of adverse medication events are not consistently followed. We evaluated the impact of electronic reminders delivered to primary care physicians on rates of appropriate routine medication laboratory monitoring.

Design: We enrolled 303 primary care physicians caring for 1,922 patients across 20 ambulatory clinics that had at least one overdue routine laboratory test for a given medication between January and June 2004. Clinics were randomized so that physicians received either usual care or electronic reminders at the time of office visits focused on potassium, creatinine, liver function, thyroid function, and therapeutic drug levels.

Measurements: Primary outcomes were the receipt of recommended laboratory monitoring within 14 days following an outpatient clinic visit. The effect of the intervention was assessed for each reminder after adjusting for clustering within clinics, as well as patient and provider characteristics.

Results: Medication-laboratory monitoring non-compliance ranged from 1.6% (potassium monitoring with potassium-supplement use) to 6.3% (liver function monitoring with HMG CoA Reductase Inhibitor use). Rates of appropriate laboratory monitoring following an outpatient visit ranged from 14% (therapeutic drug levels) to 64% (potassium monitoring with potassium-sparing diuretic use). Reminders for appropriate laboratory monitoring had no impact on rates of receiving appropriate testing for creatinine, potassium, liver function, renal function, or therapeutic drug level monitoring.

Conclusion: We identified high rates of appropriate laboratory monitoring, and electronic reminders did not significantly improve these monitoring rates. Future studies should focus on settings with lower baseline adherence rates and alternate drug-laboratory combinations.


Many medications commonly used by clinicians have recommendations for routine laboratory monitoring intended to ameliorate the risk of developing an organ toxicity or electrolyte imbalance. These recommendations are put in place by the FDA in the form of black box warnings included in the product inserts,1 or suggested by various published guidelines as a result of adverse events found in phase 3 trials or post-marketing studies. However, these lab monitoring recommendations are not routinely followed.212 Approximately one-third of patients started on a new medication fail to receive appropriate baseline laboratory assessements,13,14 with failure rates above 50% for medications such as amiodarone that require more extensive testing.15 Laboratory monitoring for maintenance therapy represents an even larger problem than baseline monitoring, with 20% of patients not receiving renal function monitoring upon initiation of metformin therapy and over 50% failing to receive annual follow-up testing.16 The failure to perform appropriate laboratory monitoring in the outpatient setting has been shown to translate into inpatient admissions and significant morbidity. Approximately one-quarter of preventable medication-related admissions can be attributed to failures of appropriate monitoring.17

Electronic health records improve health care delivery by providing superior access to patient information and enhancing clinical decision-making. Point-of-care reminder systems have been shown to improve care for both chronic disease management and preventive services.1822 These systems might also be useful for laboratory monitoring if delivered when busy clinicians are reviewing or renewing medications in the office setting.23,24 We conducted a prospective, randomized trial to evaluate the impact of electronic reminders delivered to primary care physicians within an electronic health record on rates of appropriate routine medication laboratory monitoring.


Study Setting

The study was conducted within the Partners HealthCare System, which includes two academic teaching hospitals (Brigham and Women's Hospital and Massachusetts General Hospital) and a number of community hospitals and outpatient clinics. Primary care physicians practicing at 20 outpatient clinics were recruited to participate, including 4 community health centers, 9 hospital-based clinics, and 7 off-site practices. All participating clinics used an ambulatory electronic health record (Longitudinal Medical Record) that allowed for maintenance of patient problem, medication, and allergy lists as well as for viewing test results and electronic medication prescribing.2527 The Partners Institutional Review Board approved a waiver of individual informed consent for physicians and patients in this study, as the goal of the intervention was to promote receipt of services that are widely accepted as the standard of care.


Evidence-based guidelines were reviewed for routine medication laboratory monitoring,2832 and medications were selected for inclusion in the reminder system based on 1) prevalence of use, and 2) potential morbidity associated with the failure to perform appropriate laboratory monitoring. We focused on laboratory monitoring for maintenance therapy rather than medication initiation based on prior studies indicating a larger performance gap in this area.16 This selection process was also performed to restrict information overload and resulting physician desensitization to the presented reminders.33 Annual intervals for laboratory monitoring were chosen for all of the reminders to standardize the recommendations. We focused on laboratory monitoring of potassium, creatinine, liver function, thyroid function, and therapeutic drug levels for appropriate medications (Table 1). The clinical content of the electronic reminders was developed by a team of practicing general internists and pharmacists. An EHR clinical content committee did not exist at the time of this study, but has since been formed.

View this table:
Table 1

Summary of Drug-laboratory Monitoring Recommendations that were Evaluated

MedicationLaboratory Test
Non-steroidal Anti-inflammatory Drug (NSAID)Creatinine
Angiotensin Receptor Blocker (ARB)Creatinine
Potassium Supplement (K Supplement)Potassium
Potassium Sparing Diuretic (K Sparing Diuretic)Potassium
Thiazide DiureticPotassium
Angiotensin Converting Enzyme Inhibitor (ACE Inhibitor)Potassium
HMG Co-A Reductase Inhibitor (Statin)ALT
Carbamazapine* Therapeutic Drug Level
Cyclosporine * Therapeutic Drug Level
Phenobarbital* Therapeutic Drug Level
Phenytoin* Therapeutic Drug Level
Proc-NAPA* Therapeutic Drug Level
Valproate* Therapeutic Drug Level
  • * All of the medications that required therapeutic drug monitoring were aggregated.

Medication use was based on coded information entered in the patient's electronic medication list. The dates of all laboratory tests were extracted from the Partners central data repository. Evaluation of accuracy among outpatient medication use in the LMR revealed that chronic medication use was fairly accurate, while non-prescription medication use and antibiotic use was relatively insensitive (many medications actually prescribed or taken in these categories were missing).34,35 All of the medications included in this study were prescription medications used on a chronic basis. The decision support algorithm for the reminders was run each time a clinician opened a patient's chart and reminders were generated if a patient was on the medication for at least 365 days and there was no relevant laboratory test for that medication within the prior 365 days. For physicians in the intervention group, the reminders were displayed within the patient summary screen of the LMR with other information such as the medication list, diagnosis list, and allergies (Figure 1). The reminders were suppressed for physicians in the control group.

Figure 1

Electronic reminders are displayed within the main patient summary screen alongside the patient problem and medication lists.


We performed a stratified randomization of the 20 primary care sites based on clinic characteristics to balance the distribution of gender (women's health centers) and socioeconomic factors (academic versus community clinic) between intervention and control groups. We randomly assigned ten clinical sites to receive electronic reminders for overdue laboratory monitoring, and ten clinical sites to serve as control sites.


Patients and physicians were enrolled during a six month period from January 01, 2004 to June 30, 2004 on the first occasion that a physician opened a patient chart during an outpatient clinic visit and the reminder algorithm identified the patient as being overdue for recommended testing. We also identified all patients with outpatient visits during the study period that were being treated with the targeted medications and were being appropriately monitored to calculate the incidence of appropriate monitoring in the overall clinic population.

Data Collection

Baseline patient and physician characteristics were obtained from administrative databases. Data on reminders were stored at the time of generation, including the date of the reminder and the treating physician. Medication therapy was identified from the coded medication list in the electronic health record. Dates and values of laboratory tests were obtained from the Partners central data repository, which included both inpatient and outpatient values, and could have been ordered by any provider.

Statistical Analysis

We defined the unit of analysis as a single outpatient clinic visit. The primary outcome was the patient's receipt of the appropriate laboratory testing within 14 days of the clinical encounter. In our institution, the laboratory tests included in this study are routinely completed within 12 to 24 hours of the time of blood specimen receipt.

All reminders for therapeutic drug level monitoring were aggregated into one category for analysis as these recommendations are conceptually distinct from recommendations that monitor drug toxicity indirectly. Multi-variable logistic regression models were used to assess the impact of the reminder system on rates of appropriate laboratory monitoring after adjusting for patient age, sex, race, and insurance status as well as provider age and sex. The GENMOD procedure within the SAS (Version 9,1, Cary, NC) statistical software package was used to fit models that also accounted for clustering of patients within clinical sites.36 We estimated 90% power to detect a 10% absolute increase (80% to 90%) in compliance rates in the intervention arm based on a sample size of 266 clinic visits and a two sided α of 0.05.


For the population-based adherence analysis, there were 21,083 patients seen by 464 physicians in 45,662 clinic visits who were on at least one of the monitored medications for one year at the time of the visit. For the randomized intervention, there were 1,922 patients seen by 303 physicians in 2,507 clinic visits during which a patient was overdue for recommended laboratory monitoring. Among those enrolled in the randomized intervention, patient race and insurance status were significantly different by intervention status, and there were no differences in physician characteristics (Table 2).

View this table:
Table 2

Baseline Patient and Physician Characteristics of Patients Overdue for Routine Laboratory Monitoring

Patient Characteristics(n = 998)(n = 924)
   Mean age, yr (±SD)60.2 (±14.6)60.2 (±14.3)0.996
   Female (%)605 (60.6)530 (57.4)0.150
   Race (%)
   White596 (59.7)509 (55.1)0.042
   African American82 (8.2)107 (11.6)0.041
   Hispanic66 (6.6)153 (16.6)<0.001
   Other43 (4.3)30 (3.2)0.234
   Unknown211 (21.1)125 (13.5)<0.001
   Insurance (%)
   Medicare328 (32.9)326 (35.3)0.268
   Medicaid74 (7.4)126 (13.6)<0.001
   Private579 (58.0)447 (48.4)<0.001
   Self-Pay17 (1.7)25 (2.7)0.160
Physician Characteristics(n = 158)(n = 145)
   Mean age, yr (±SD)40.6 (±11.2)40.5 (±11.1)0.921
   Female (%)93 (58.8)90 (62.1)0.638

The prevalence of targeted medication use varied widely among the total clinic population. The most commonly used medications were hydroxymethylglutaryl Co A reductase inhibitors (statins) (20,376 visits) and non-steroidal anti-inflammatory drugs (NSAIDs), (17,794 visits,) and the least commonly used were the drugs requiring therapeutic monitoring (1,269 visits) and potassium supplementation (1,330 visits). Medication-laboratory monitoring non-compliance (defined as no appropriate laboratory testing in the previous year) ranged from 1.6% (21/1330) in potassium supplementation to 6.3% (1287/20376) for statin use.

Rates of appropriate laboratory monitoring within 14 days of an office visit ranged from 14% for therapeutic drug levels to 64% for potassium levels in the presence of potassium-sparing diuretics. In adjusted analyses, reminders for appropriate laboratory monitoring had no impact on rates of receiving these tests for creatinine, potassium, liver function, renal function, or therapeutic drug level monitoring (Table 3).

View this table:
Table 3

Population Guideline Adherence and Impact of Electronic Reminders on Medication Laboratory Monitoring

Medication–Lab ReminderArm# Visits# Visits with Lab Overdue (%)# Lab Ordered when Overdue (%)Odds Ratio (Adjusted)p-Value
NSAID–CrC9307428 (4.6%)136 (31.8%)1.24 (0.71–2.15)0.457
I8487442 (5.2%)150 (33.9%)
ARB–CrC83227 (3.2%)17 (63.0%)0.24 (0.04–1.34)0.104
I75131 (4.1%)17 (54.8%)
Metformin–CrC78116 (2.1%)6 (37.5%)0.53 (0.05–5.34)0.594
I85620 (2.3%)7 (35.0%)
K Supplement–KC7519 (1.2%)5 (55.5%)0.91 (0.03–24.44)0.956
I57912 (2.1%)7 (58.3%)
K Sparing Diuretic–KC87528 (3.2%)17 (60.7%)0.82 (0.12–5.60)0.836
I76119 (2.5%)13 (68.4%)
Thiazide Diuretic–KC250889 (3.5%)46 (51.7%)1.30 (0.63–2.67)0.473
I199762 (3.1%)40 (64.5%)
ACE Inhibitor–KC279080 (2.9%)40 (50.0%)1.00 (0.43–2.30)0.993
I2279119 (5.2%)57 (47.9%)
Statin–ALTC10935674 (6.2%)358 (53.1%)0.89 (0.43–1.81)0.740
I9441613 (6.5%)291 (47.5%)
Thyroxine–TSHC123344 (3.6%)25 (56.8%)1.19 (0.40–3.53)0.747
I89738 (4.2%)22 (57.9%)
Therapeutic Level* C75526 (3.4%)4 (15.4%)0.55 (0.03–8.94)0.677
I51416 (3.1%)2 (12.5%)
  • * This represents the aggregated reminders for Therapeutic monitoring the following medications: carbamazapine, cyclosporine, Phenobarbital, phenytoin, Proc-NAPA, valproate.

  • C = Control; I = Intervention. Please refer to Table 1 for medication abbreviations.


There is a substantial body of evidence documenting performance gaps related to appropriate monitoring of laboratory tests in the presence of medication use, as well as the poor outcomes associated with such shortcomings. Our study found only a relatively small proportion (approximately 5%) of patients on targeted medications were overdue for appropriate laboratory monitoring at the time of office visits with primary care physicians, and that nearly half of these patients had appropriate testing within two weeks of the office visit. Electronic reminders to physicians were not effective in increasing the rates of appropriate laboratory monitoring in this setting.

The prevalence of overdue routine laboratory monitoring among patients in this study was low compared to previous work in this area, where other investigators have documented much higher rates of overdue creatinine testing with non-steroidal anti-inflammatory medication use (30%) and metformin use (52%).15,16 These low rates of overdue testing may relate to the fact that our health system employs a unified clinical data repository across all outpatient and inpatient settings, and thus captures lab values from episodes of clinical care that may not be available in settings that lack such a comprehensive system. Alternatively, the low overdue rate might reflect the benefits of an electronic medical record system that increases access to patient information such as up to date medication lists. While we did not collect additional data on redundant testing, it is important to note that these high rates of testing likely come at the expense of over-utilization, as prior studies have documented rates of redundant inpatient laboratory testing within our system of nearly 10%.37

While a number of studies have shown that computerized reminders are efficacious in the inpatient setting, their use in the outpatient setting is less well described. A few randomized trials have shown improvements in ordering appropriate laboratory tests upon medication initiation with the use of outpatient computerized reminders.3840 However, only one previous randomized trial has evaluated the effect of outpatient computerized reminders on recommended routine monitoring. In an Israeli healthcare provider network, approximately 20% of patients taking diuretics were overdue for an annual potassium check, and reminders were effective in increasing monitoring compliance.41

Barriers to provider utilization of clinical reminders are limitations of our study that might have contributed to the negative findings. For example, our previous work indicates that only one-third of providers in our system notice reminders during clinical encounters.25 The reminders are visible from the patient summary screen, which is a common portal for physicians. However, since our reminders are passive, it is possible for a clinician to not use this screen and thereby circumvent the reminder system. Physicians may also be less likely to utilize reminder systems as workload increases.42 While inaccuracy of recommendations is often cited as a barrier to effective clinical decision support tools,43 this was unlikely in our system as we were able to capture both laboratory results and medication prescriptions in a relatively complete manner. Other studies have noted decreases in the use of reminders over the course of a study, attributed to reminder fatigue.44,45 Clinical content could also have limited the utility of the reminders if physicians felt that the recommendations were unnecessary, or if the interval was too frequent. In order to address these issues in our evaluation, we chose a conservative monitoring interval of one year and drug-lab recommendations in which there were clear FDA concerns or significant adverse effects from a lack of monitoring. Laboratory test values obtained outside our health system were not electronically available, but could have been known by providers. This could falsely increase overdue rates and contribute to apparent reminder non-compliance by displaying reminders for labs appropriately performed elsewhere. However, it is important to note that the vast majority of our patients' labs are performed within the Partners system, and would therefore show up in our Central Data Repository. In addition, patient non-compliance is included in reminder non-compliance, since only ordering a laboratory test is insufficient to fulfill guideline compliance. Finally, clinicians who viewed the reminders still needed to order the lab tests in a separate step using a paper lab ordering form. This additional step could have been an obstacle to taking action based on the reminder.

A final explanatory mechanism for our failure to demonstrate an effect with the reminder system is the ceiling effect associated with the high baseline rates of adherence. However, a post-hoc power calculation using higher rates of compliance (90% baseline rate, detecting 5% absolute difference) required 582 clinic visits per arm to achieve 90% power, suggesting adequate power among the reminders in this study to detect such a treatment effect.

In summary, we found relatively high rates of appropriate laboratory monitoring in our study population, and an electronic reminder system did not significantly improve rates of such testing. Future studies should focus on laboratory monitoring for different classes of medications, laboratory monitoring at medication initiation, and settings with lower rates of baseline adherence to monitoring recommendations.


The authors thank the study participants, including the patients and primary care physicians within the Partners HealthCare System. We also thank Anne Fladger and her staff for their assistance. This work was funded in part by grant 5-U18-HS-011046 from the Agency for Healthcare Research and Quality and by grant 1-T15-LM-07092 from the National Library of Medicine.


View Abstract