Objective Falls are physically and financially costly, but may be preventable with targeted intervention. The Minimum Data Set (MDS) is one potential source of information on fall risk factors among nursing home residents, but its limited breadth and relatively infrequent updates may limit its practical utility. Richer, more frequently updated data from electronic medical records (EMRs) may improve ability to identify individuals at highest risk for falls.
Methods The authors applied a repeated events survival model to analyze MDS 3.0 and EMR data for 5129 residents in 13 nursing homes within a single large California chain that uses a centralized EMR system from a leading vendor. Estimated regression parameters were used to project resident fall probability. The authors examined the proportion of observed falls within each projected fall risk decile to assess improvements in predictive power from including EMR data.
Results In a model incorporating fall risk factors from the MDS only, 28.6% of observed falls occurred among residents in the highest projected risk decile. In an alternative specification incorporating more frequently updated measures for the same risk factors from the EMR data, 32.3% of observed falls occurred among residents in the highest projected risk decile, a 13% increase over the base MDS-only specification.
Conclusions Incorporating EMR data improves ability to identify those at highest risk for falls relative to prediction using MDS data alone. These improvements stem chiefly from the greater frequency with which EMR data are updated, with minimal additional gains from availability of additional risk factor variables.
Electronic medical records
minimum data set 3.0
nursing home falls
The most frequently reported adverse event among frail nursing home residents,1 falls are associated with increased mortality and morbidity, and with reduced functioning.2 Nursing home fall rates range from approximately 0.2–3.6 falls per bed per year,3 with a mean of 1.5 falls per bed per year for an average nursing home with 100 beds. In 2000, the average estimated direct medical cost associated with a nonfatal fall was $73074 for adults older than 65 years of age.
Identifying nursing home residents at highest risk for falls can facilitate targeted intervention,5 potentially reducing incidence and associated costs.6 Prior studies have identified a number of important risk factors for falls, including better ambulatory status (i.e., not bed or chairbound), gait and balance deficits, history of falls, visual deficits, diminished lower body strength, and psychotropic drugs.3 As a practical matter, however, identifying nursing home residents most at risk for falls in real time has remained challenging.7,8
Many previous studies of fall risk factors in nursing homes9–11 have relied solely on data from the Minimum Data Set (MDS), a standardized Centers for Medicare and Medicaid Services (CMS) screening and assessment tool completed for all residents of Medicare and/or Medicaid-certified nursing homes in the United States on at least a quarterly basis. These studies identify resident fall risk factors at a baseline, and use those baseline risk factors to estimate likelihood of a fall during a subsequent follow-up period. None of these studies consider a time-variant risk profile as risk factors change over time.
While the MDS contains information on a wide array of risk factors for falls, these data are collected infrequently, presenting practical challenges for analysts attempting to model time-variant risk. Additionally, the MDS does not include some potentially important fall risk factors. In this study, we investigate the extent to which the rich, real-time data on risk factors available in nursing home residents’ electronic medical records (EMRs) can augment prediction of fall risk based on the MDS alone. We show that predictive fall risk models incorporating both EMR and MDS data have greater predictive power than models relying on MDS data alone.
Minimum data set
For long-stay residents, CMS requires MDS assessments to be completed on admission, at discharge, and when a nursing home resident experiences a significant change in clinical or functional status; by mandate, MDS assessments must be conducted at least quarterly for each resident even in the absence of any such triggering event.
MDS data are intended to be used for resident care planning, Medicare and Medicaid payment, and for ongoing monitoring and quality improvement purposes. To support this range of objectives, the MDS includes a wide range of standardized, clinically meaningful measures defined identically across all nursing home residents. This standardization is a key advantage of the MDS as a source of information on fall risk factors. However, the MDS instrument does not include measures for a number of important risk factors for falls, including detail on use of specific prescription drugs and on nonclinical factors such as a recent resident room change. Additionally, because MDS assessments are conducted relatively infrequently, the MDS data may not capture changes in clinical and functional status that are insufficiently “significant” to trigger a new assessment, but which contribute to fall risk. We explore whether this reporting lag may limit the usefulness of the MDS for identifying individuals at risk for falls in real time.
Electronic medical records
EMRs have been widely praised for their potential to improve patient care, enabling improved quality and coordination while simultaneously reducing costs.12 EMRs have been increasingly adopted in recent years in many healthcare settings; by 2013, 94% of hospitals13 and 78% of office-based physician practices14 had adopted EMRs.
EMR adoption rates in nursing homes are thought to lag substantially below levels in other healthcare settings; between 18% and 48% of nursing homes are estimated to implement electronic records of any kind, where almost 43% are estimated to electronically record nursing and physician notes.15,16 Observers anticipate an acceleration in EMR adoption by nursing homes as CMS implements Stage 2 of the Medicare and Medicaid Electronic Health Care Record (EHR) Incentive Programs, which became effective in 2014. Under Stage 2 requirements, eligible providers, potentially including those operating in nursing homes and other long-term care settings, will be required to send electronic care summaries during transitions of care.13 Independently of meaningful use requirements, evidence suggests that nursing home adoption of electronic health records is slowly increasing.17
Our investigation provides additional evidence for nursing homes considering adoption of an EMR system, by exploring the practical utility of EMR data in identifying high-risk residents to facilitate early intervention. In contrast to the MDS, resident EMRs are updated on an ongoing basis to support their use in day-to-day clinical care. To the extent that health status changes are clinically meaningful, but fall short of the standard for triggering an MDS assessment, the EMR may more accurately capture real-time fluctuations in fall risk factors. For this reason, we might expect improved predictive power in a fall risk model including EMR data relative to one based on MDS data alone, even if included risk factors were identical.
In addition, note that the EMR is intended to provide a comprehensive record of all information for provider use in care planning, in contrast to the explicit intent of the MDS to provide a minimum set of assessment data. The EMR may contain information on a wider array of fall risk factors than the MDS. Addition of such factors to a fall risk model may thus further enhance predictive power beyond any improvement attributable to increased frequency of observations alone.
To investigate whether introduction of EMR data can meaningfully improve predictive power relative to more commonly used models using MDS data alone, and to evaluate the extent to which any such improvement is attributable to increased frequency or increased breadth of data available in the EMR, we analyzed data provided by a large for-profit California nursing home chain that employs a centralized EMR system from a leading vendor.
Data analyzed in this study came from residents of nursing homes in the control group of a larger random assignment evaluation of an intervention intended to reduce nursing home falls. Twenty-six nursing homes were selected to participate in the evaluation from among all 42 nursing homes owned and operated by the nursing home chain, according to the following criteria. First, we excluded three nursing homes with substantial quality problems, indicated by the presence of severe deficiencies or enrollment in the Special Focus Facility program in the past 12 months, as reported on the CMS Nursing Home Compare website. Second, we excluded 8 nursing homes that had not been using the chain’s centralized EMR system for clinical documentation prior to the start of the evaluation period in October 2012. Finally, we excluded five nursing homes with insufficient motivation or interest to participate in the falls prevention intervention, as indicated by chain leadership. Of the remaining 26 nursing homes, 13 were randomly selected to implement the intervention, while the remaining 13 served as the control group, continuing to offer standard care. Data from 5129 individuals (133 781 observations) residing in these 13 control group nursing homes from October 5, 2011 through April 22, 2014 were used to develop the fall risk models presented below.
For each resident included in the study, we assembled multiple records with contiguous starting and ending dates, so that a new resident record began on the day immediately after the previous record ended. Specifically, a new resident record began when at least one time-varying risk factor changed its previous status. Conceptually, then, we assembled data suitable for a repeated event survival model with time-varying covariates. This approach yielded 133 781 individual observations across the 5129 residents over the study observation period.
Approximately 37% of resident records were missing data for one or more risk factors of interest; item nonresponse for individual risk factors ranged from <0.5% to a maximum of 35%. We assume that covariates are missing at random but not missing completely at random, allowing us to perform multiple imputation via multivariate normal regression (5 replicate imputations) to impute missing observations. Imputation of missing values from MDS records relied solely on available MDS data, while imputation of missing values from EMR records relied on both other EMR data and the MDS data. This approach allows us to evaluate the predictive ability of the MDS as a stand-alone data source, to facilitate comparison with a more expansive model enriched by the addition of EMR data.
Risk factors for potential inclusion in our fall risk model were identified through review of the relevant literature. The nursing home fall risk and fracture literature identifies specific fall risk factors. These include: a prior fall in the last 6 months; full ambulation (i.e., the resident is not chair or bed-bound); wheelchair use; use of walking aids such as a cane or walker; an unsteady gait or imbalance; wandering; osteoporosis; anemia; epilepsy; use of antipsychotic, antianxiety, or antidepressant medications; and dementia.1,9,18–21
Individual risk factors affect a nursing home resident’s probability of falling in different ways. Mechanical risk factors directly impair a resident’s strength and physical ability to stand. An unsteady gait, caused by diabetes or another condition, is estimated as increasing the risk of falling by approximately 5 times.1 Walkers and canes can impede an individual’s stepping movement, which increases the likelihood of a fall.22 Poor vision affects an individual’s perception of balance,23 while imbalance is a symptom of gait disorders such as Parkinson’s disease.23 Anemia is associated with physical decline and increase in disability.24
Medication and psychological factors can introduce risk by altering a resident’s behavior. For instance, depression and anxiety have been noted to lead to a loss of confidence, ultimately increasing the risk of a fall.25 Cognitive impairment can affect a resident’s judgment when making decisions to walk without assistance, and plays a significant role in the regulation of balance and gait.26,27 Epilepsy and seizures impair awareness and levels of cognitive functioning.20 Medication can affect an individual’s balance and gait as well as cause dizziness and hypotension.28,29
Lastly, nursing home residents’ being introduced to a new environment may increase the risk of falls. Residents who are newly admitted to the nursing home are unfamiliar to nursing home staff, and this can impede the identification and management of fall risk factors shortly after an admission.28 In the same vein, a technical expert panel consulted by our study team suggested that changing a resident’s room can also introduce confusion and should be explored as a potential contributing risk factor.
One of our researchers, a nurse with long-term care expertise, worked with information technology specialists to review the MDS 3.0 instrument and EMR data specifications to identify measures corresponding to risk factors selected via the literature review and technical expert panel consultation. Table 1 lists measures from the MDS and/or EMR that were included in our fall risk models. Column 1 lists fall risk factor measures that were available in the MDS but not the EMR, Column 2 lists risk factors available in both the MDS and EMR, and Column 3 lists fall risk factors available in the EMR but not the MDS. Note that while the MDS 3.0 is standardized across nursing homes nationwide, the EMR system that the study’s nursing home chain uses is one of many available EMR systems; different EMRs may or may not include measures corresponding to the list below. Table 2 reports summary statistics for MDS and EMR measures. Supplementary Table 1 provides cross-tabulations of MDS and EMR measures for the same underlying risk factor, to demonstrate the high degree of concordance across sources. In general, for a given risk factor, EMR data more frequently indicate that a particular risk factor is present when MDS data do not than the reverse case. This is consistent with our hypothesis that EMR measures may be more sensitive to subtle or short-lived variations in status.
For estimation purposes, we randomly separate our data into testing and validation cohorts. Our testing sample includes 2527 residents (65 202 observations), while the remaining 2602 residents (68 579 observations) are allocated to the validation sample. As described in greater detail in the remainder of this section, the testing cohort is used to estimate model parameters, and the validation cohort is then used to assess the proportion of observed falls by predicted fall risk decile.
Within the testing cohort, we predict each resident’s probability of falling via a repeated events survival model by employing logistic regression. We include covariates for each risk factor, days since admission to the nursing home and days since admission squared, interactions between each risk factor and days since admission, and duration of time that each resident exhibits a particular risk profile. The length-of-stay interaction terms are intended to capture changes in the relationship of individual risk factors to fall risk as a function of time in the nursing home. For example, nursing home staff may make adjustments over time to mitigate risk of falling for a resident known to wander, so that wandering later in a resident’s stay is less likely to result in a fall than wandering that occurs shortly after admission.
Formally, we estimate the probability of falling using the log odds:
Here is the probability that resident i will fall during period t, and is a row vector of risk factors for individual i at time t. Days since nursing home admission is denoted , and is the interaction between individual risk factors and the time in the nursing home for individual i at time t; , , and are time-invariant column vectors of parameters. A robust covariance estimator accounts for the dependence across observations.
We then estimate the probability of falling during period t for each resident i in the validation cohort using the formula:
For each of the 4 model specifications described below, we then divide the validation cohort into deciles by projected fall risk, and determine the proportion of observed falls that occur within each decile. This allows us to compare how well each specification differentiates between residents with a high and low risk of falling. In a well-fitting model, the majority of observed falls will occur among those in the highest projected risk deciles, with few falls occurring among those in the lowest projected risk deciles. As a sensitivity test, we additionally examined distributions that defined projected risk groups by fixed cutpoints rather than model-specific deciles; we omit these findings for the sake of brevity, as they did not substantively change study conclusions.
In addition, we report the Akaike Information Criterion (AIC) as a goodness-of-fit summary statistic for each specification, with lower AIC values indicating better model fit.
To estimate the probability of a resident’s fall conditional on different sets of risk factors, we employ four different model specifications. Model 1 includes only variables from the MDS that have been identified in the literature as increasing a resident’s risk of falling. This “base” specification is used as a point of comparison for three alternative specifications incorporating EMR data to assess the extent to which use of EMR data can improve predictive power.
Model 2 takes the base MDS-only specification from Model 1 and adds variables on risk factors additionally available in the EMR. Comparing results from Model 2 to results from Model 1 thus allows us to assess the incremental contribution of these additional EMR-only risk factors to predictive power.
Model 3, in contrast, is intended to enable assessment of the incremental contribution of higher-frequency real-time data from the EMR. In particular, it substitutes all available EMR variables that measure the same risk factors as are included in the base MDS-only Model 1 specification, but does not include any additional EMR-only risk factors.
Finally, Model 4 takes the Model 3 specification with included “duplicate” EMR variables replacing MDS variables for identical risk factors, and adds the EMR-only risk factor variables that are included in the Model 2 specification. Model 4 can also be thought of as the “full” specification, as it makes maximal use of all available risk factor information from both data sources.
Estimated odds ratios for each of the 4 model specifications are reported in Supplemental Table 1; because our focus is on assessing goodness of fit across models, we do not focus on interpretation of individual odds ratios from the regressions in the main text.
For each of the 4 model specifications, Table 3 reports the proportion of observed falls for each estimated decile of projected resident fall risk, along with the AIC statistic. The 10th decile includes those with the highest projected fall risk; for all four specifications, the highest proportions of observed falls occur among those in the highest projected risk decile. The proportion of observed falls occurring within each subsequently lower projected decile then declines relatively smoothly for all four models, with individuals in the lowest projected risk decile accounting for only 2% of observed falls.
Proportion of Observed Falls in Validation Cohort by Projected Fall risk Decile (N = 68 579 Records for 2602 Residents)
MDS Assessments + EMR Only
MDS Assessments + EMR Duplicates
MDS Assessments + EMR Only + EMR Duplicates
Note: AIC = Akaike Information Criterion. Lower AIC values represent improved goodness of fit.
AIC statistics indicate that Models 3 and 4, which replace MDS risk factor measures with more-frequently-updated EMR measures, fit the observed data better than Models 1 and 2. Visual inspection of Table 1 results and the cumulative distribution of observed falls by risk decile (Figure 1) indicates that the models appear to perform similarly in the lower projected risk deciles, but that Models 1 and 2 underperform relative to Models 3 and 4 in higher risk deciles. The difference across models in the proportion of observed falls for those classified in the highest projected risk decile can be interpreted as the incremental improvement in the proportion of residents correctly classified as high-risk.
Cumulative distribution of observed falls in validation cohort by projected risk decile (N = 68 579 Records for 2602 Residents).
More concretely, in the base MDS-only model (Model 1), Table 1 shows that individuals in the highest projected risk decile account for 28.6% of observed falls. Model 2, which adds EMR variables for risk factors not represented in the MDS, yields broadly similar findings, with 28.6% of observed falls occurring in the top risk decile. In contrast, in Model 3, which replaces MDS variables with EMR variables for the same risk factors when available, 32.3% of observed falls occur among those in the highest risk decile, a 13% increase relative to the base MDS-only Model 1 specification. The full Model 4 specification, which includes both duplicative and additional EMR risk factors, offers no improvement relative to Model 3. Together, these results imply that replacement of MDS risk factor measures with more-frequently-updated EMR measures substantially improves identification of residents at highest risk for falls, but that the addition of risk factors available in the EMR yields little or no improvement.
Our results show a moderate improvement in identification of nursing home residents at high risk for falls when MDS assessment data are augmented with data from the EMR. In this application, the observed improvement appears to be almost entirely attributable to the increased frequency with which EMR data are updated, as opposed to the availability of additional risk factors in the EMR data.
Our analysis is subject to several important limitations. One set of limitations relates to study generalizability given our sample selection procedures. Because our sample was restricted to nursing homes within a single large for-profit California chain, it is unclear how results may generalize to other nursing homes. Furthermore, because study nursing homes within the chain were selected to meet requirements of a larger random assignment evaluation, our sample did not include nursing homes with substantial quality problems, brand-new EMR systems, or insufficient motivation to participate in a falls prevention program. Our sample may therefore disproportionately represent higher-quality nursing homes with more sophisticated use of EMR systems, further limiting generalizability of our findings.
Second, as noted above, 37% of residents were missing data on one or more MDS or EMR risk factors. Our application of multiple imputation relies on the assumption that data are missing at random (but not missing completely at random); that is, we assume that the probability that a selected risk factor is missing is unrelated to its value, conditional on other variables in the analysis. To the extent that patterns of “missingness” deviate from this assumption, estimated covariates may be subject to selection bias.
A third set of limitations relates to the particular EMR used in our study. As noted above, the study nursing home chain employed a single, centralized EMR system from a leading EMR vendor. As in most long-term care settings, the EMR system was customized for the chain’s specific clinical and administrative needs and procedures. As a result, our findings may not generalize to EMR systems developed by other vendors, or customized for other settings. One specific limitation of the chain’s EMR relates to the prescription drug data. The EMR prescription drug orders data used in this study were restricted to new orders only; renewals of standing prescription orders were not recorded in the EMR due to the chain’s operational procedures. Furthermore, these data included only orders from providers affiliated with the chain. For these reasons, the EMR prescription drug data likely captured only a fraction of all medications taken by a given resident. This limitation may explain why the addition of variables for prescription drug class risk factors from the EMR did not result in improvements in model prediction.
Despite these limitations, this study provides useful evidence on potential applications of EMR data for quality improvement. Because falls are so frequent and so costly, a back-of-the-envelope computation implies that even small improvements in identification of high-risk residents may result in large potential cost savings. Consider a hypothetical program to prevent nursing home falls that focuses on identification and pro-active intervention among those residents at highest risk, defined here as the top risk decile. As noted in the introduction, approximately 150 resident falls per year are expected to occur in an average 100-bed nursing home.3 If our hypothetical program were to rely on MDS assessment data alone to identify high-risk residents, our Model 1 results imply that 28.6% of those 150 falls, or approximately 43 falls in all, would occur among residents classified within the top projected risk decile. In comparison, if the program were additionally able to rely on data from resident EMRs, Model 3 results imply that 32.3%, or approximately 49 falls, would occur among residents classified within the top projected risk decile. If the program were 100% successful in preventing falls among targeted high-risk residents, assuming a direct cost per fall of $7307,4 incorporation of EMR data into risk-prediction procedures would then result in prevention of 6 additional falls, translating into $43 842 in cost savings per year. Even if the program prevented only ⅓ of falls among the high-risk cohort, annual cost savings would be $14 614. Particularly in nursing homes that have already adopted EMR systems, it seems likely that cost savings of this magnitude would readily justify the additional incremental cost of incorporating EMR information into targeted clinical decision support systems.
We have demonstrated that use of data from an EMR system can materially improve ability to identify those individuals at highest risk for falls. However, standard EMR systems currently provide no easy way to synthesize and summarize information on changing risk factors recorded in disparate parts of the EMR to support clinical decision-making. Recent work by the Agency for Healthcare Research and Quality has focused on developing specifications for easy-to-use reports for staff use, synthesized from EMR data on resident risk factors for falls and other preventable adverse events, including pressure ulcers and avoidable hospitalizations.28,30 Further development of such applications, focused both on falls and on other avoidable adverse events for which risk factors can be readily identified, should be a key priority as nursing homes expand their adoption of EMRs in coming years.
This study was funded by the Agency for Healthcare Research & Quality (AHRQ), Department of Health & Human Services (DHHS), under contract # HHSA290201000031I. The content of this article is solely the responsibility of the authors and does not represent the official views or recommendations of AHRQ or DHHS.
The authors have no competing interests to declare.
Contributors L.O. and W.S. were instrumental in obtaining the necessary data for this study. A.M., L.O., and W.E. developed the study idea; W.S. was extensively involved with technical specifications. A.M. led the technical analyses with substantial input from W.R. and L.O. A.M., L.O., and W.R. wrote the first manuscript draft, all authors reviewed manuscript drafts. A.M. and L.O. responded to referee comments.
. Use and characteristics of electronic health record systems among office-based physician practices, United States, 2001–2012. US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics; 2012. http://www.cdc.gov/nchs/data/databriefs/db111.pdf. Accessed January 28, 2015.
. Gait velocity versus the timed up and go test: which one to use for the prediction of falls and other adverse health outcomes in primary care? J Am Geriatr Soc. 2011;59:2191–2192; author reply 2192–2193.