January 18, 2019
Association of Adverse Effects of Medical Treatment With Mortality in the United StatesA Secondary Analysis of the Global Burden of Diseases, Injuries, and Risk Factors Study
Jacob E. Sunshine, MD, MS1,2; Nicholas Meo, MD4; Nicholas J. Kassebaum, MD2,3; et alMichael L. Collison, BA2; Ali H. Mokdad, PhD2; Mohsen Naghavi, MD, PhD2
Author Affiliations Article Information
JAMA Netw Open. 2019;2(1):e187041. doi:10.1001/jamanetworkopen.2018.7041
Question How has mortality associated with adverse effects of medical treatment in the United States changed over time, by state, age, and sex?
Findings In this cohort study, there was a decrease in the national age-standardized mortality rate associated with adverse effects of medical treatment in the United States between 1990 and 2016. Although no differences by sex were observed, increased mortality due to adverse effects of medical treatment was associated with advancing age, and geographic variability was noted.
Meaning Global Burden of Disease 2016 results suggest that mortality associated with the adverse effects of medical treatment has decreased modestly over the past 25 years, and although the degree of improvement varies by state, it appears that an increased burden of adverse effects of medical treatment on aging populations continues to affect the US health system.
Importance More than 20 years have passed since the first publication of estimates of the extent of medical harm occurring in hospitals in the United States. Since then, considerable resources have been allocated to improve patient safety, yet policymakers lack a clear gauge of the progress made.
Objectives To quantify the cause-specific mortality associated with adverse effects of medical treatment (AEMT) in the United States from 1990 to 2016 by age group, sex, and state of residence and to describe trends in types of harm and associations with other diseases and injuries.
Design, Setting, and Participants Cohort study using 1990-2016 data on mortality due to AEMT from the Global Burden of Diseases, Injuries, and Risk Factors (GBD) 2016 study, which assessed death certificates of US decedents.
Exposures Death with International Classification of Diseases (ICD)–coded registration.
Main Outcomes and Measures Mortality associated with AEMT. Secondary analyses were performed on all ICD codes in the death certificate’s causal chain to describe associations between AEMT and other diseases and injuries.
Results From 1990 to 2016, there were an estimated 123?603 deaths (95% uncertainty interval [UI], 100?856-163?814 deaths) with AEMT as the underlying cause. Despite an overall increase in the number of deaths due to AEMT over time, the national age-standardized mortality rate due to AEMT decreased by 21.4% (95% UI, 1.3%-32.2%) from 1.46 (95% UI, 1.09-1.76) deaths per 100?000 population in 1990 to 1.15 (95% UI, 1.00-1.60) deaths per 100?000 population in 2016. Men and women had similar rates of AEMT mortality, and those 70 years or older had mortality rates nearly 20-fold greater compared with those aged 15 to 49 years (mortality rate in 2016 for both sexes, 7.93 [95% UI, 7.23-11.45] per 100?000 population for those aged ≥70 years vs 0.38 [95% UI, 0.34-0.43] per 100?000 population for those aged 15-49 years). Per 100?000 population, California had the lowest age-standardized AEMT mortality rate at 0.84 deaths (95% UI, 0.57-1.47 deaths), whereas Mississippi had the highest mortality rate at 1.67 deaths (95% UI, 1.19-2.03 deaths). Surgical and perioperative events were the most common subtype of AEMT, accounting for 63.6% of all deaths for which an AEMT was identified as the underlying cause.
Conclusions and Relevance This study’s findings suggest a modest reduction in the mortality rate associated with AEMT in the United States from 1990 to 2016 while also observing increased mortality associated with advancing age and noted geographic variability. The annual GBD releases may allow for tracking of the burden of AEMT in the United States.
More than 20 years ago, the Harvard Medical Practice Study provided the first estimate of the extent of medical harm occurring in US hospitals.1 Building on mounting evidence from several studies, the Institute of Medicine (IOM) published their landmark report, To Err Is Human: Building a Safer Health System, in 1999, in which they estimated that approximately 44?000 to 98?000 deaths occur annually because of medical errors.2
Considerable resources have been allocated to improve patient safety since these reports on patient harm, with significant advances achieved in safety research, quality improvement initiatives, policy, health information technology, reimbursement strategies, and accreditation standards.2-10 In certain clinical domains, such as hospital-acquired infections and transitions of care between teams and patient care units, there have been several reports suggesting notable nationwide improvements in outcomes.11-13 It is unclear, however, if improvements in such proxy measures of overall patient safety have translated into mortality improvements across all types of adverse effects of medical treatment (AEMT) over time.
Although adverse event detection methods have continued to advance in recent years, a significant challenge remains in gauging the progress made at the state or national level. Previous studies of medical adverse events have been crucial to both jumpstarting the patient safety movement and to providing critical insights into medical harm, but commonly applied approaches are limited in several key respects. First, point estimates of medical harm using retrospective surveillance systems—several of which have reported higher estimates of annual mortality related to medical errors than the IOM report—are derived from resource-intensive medical record reviews.1,14-22 These tools are excellent for use at a health care organizational level but make comprehensive and consistently applied assessments on a state or national scale particularly challenging. Second, voluntary reporting systems may be used to monitor patient safety trends on a larger scale but are known to have selection bias and underreporting.23-26 Finally, administrative databases screened for adverse events may be limited to a range of conditions or have an overall low detection sensitivity.27-30
The Global Burden of Diseases, Injuries, and Risk Factors (GBD) study31 is the most comprehensive source of comparable information on the levels and trends of health loss across all disease and injuries throughout the world. This report uses the 2016 GBD study to present the scope and trend of mortality associated with AEMT in the United States from 1990 to 2016.
2016 GBD Study
The 2016 GBD study is a multinational collaborative project with an aim of providing regular and consistent estimates of health loss worldwide. Methods for GBD 2016 have been reported in full elsewhere.31 Briefly, data were obtained from deidentified death records from the National Center for Health Statistics32; records included information on sex, age, state of residence at time of death, and underlying cause of death. Causes were classified according to the International Classification of Diseases, Ninth Revision (ICD-9), for deaths prior to 1999 and the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) for subsequent deaths.33,34 Each death was categorized as resulting from a single underlying cause. All ICD codes were mapped to the GBD cause list, which is hierarchically organized, mutually exclusive, and collectively exhaustive.31 The complete lists of ICD-9 and ICD-10 codes mapped to AEMT in the 2016 GBD study cause classification system are in eTables 1 and 2 in the Supplement. This cohort study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines. This research received institutional review board approval from the University of Washington, Seattle. Because anonymized death records were analyzed retrospectively, informed consent was not required. We defined AEMT as a coded injury that arises from an individual’s medical management.
Cause-of-death ensemble modeling (CODEm), a standard analytic tool used in GBD cause-specific mortality analyses, was used to generate mortality rate and cause fraction (percentage of all-cause deaths due to a specific GBD cause) estimates for the years 1990 through 2016. A full technical description of CODEm based on the most up-to-date iteration of the GBD methodology31,35 is available in the eMethods and eTables 3 and 4 in the Supplement. Results are age standardized using the GBD world age population standard, which accounts for differences in population size and age structure.31,36
The GBD methodology also accounts for when ill-defined or implausible causes were coded as the underlying cause of death.37 Plausible underlying causes of death were assigned to each ill-defined or implausible cause of death according to proportions derived in 1 of 3 ways: (1) published literature or expert opinion, (2) regression models, and (3) initial proportions observed among targets. These codes are shown in eTable 4 in the Supplement.
All estimates in the primary analysis include an uncertainty interval (UI), which represents a range of values that reflects the certainty of an estimate. The UIs are generated by sampling 1000 values (called draws) for each estimate and summing the draws across age, cause, and location for all intermediate calculations. The UIs are defined by the 25th and 975th draw values, representing the 2.5th and 97.5th percentiles.
Two secondary analyses were completed as part of this report to supplement the GBD 2016 findings. First, the ICD codes encompassing AEMT were subdivided into categories to capture the nature of the medical harm based on aggregate analysis of all available death records from 1980 to 2014 (N = 80?414?952). The ICD classification framework adopted for this secondary analysis was modified from Houghland et al38and allocated AEMT deaths into 6 categories: (1) adverse drug events, (2) surgical and perioperative adverse events, (3) misadventure (events likely to represent medical error, such as accidental laceration or incorrect dosage), (4) adverse events associated with medical management, (5) adverse events associated with medical or surgical devices, and (6) other. The categorization of ICD codes into the 6 subgroups was done by consensus among 3 of us (J.E.S., N.M., and M.N.). All subcategorized codes are available in eTables 1 and 2 in the Supplement.
Cause-of-Death Chain Analysis
The final secondary analysis involved the analysis of the cause-of-death chains for all deaths from 1980 to 2014 to measure how frequently AEMT was (1) anywhere within a death certificate’s cause-of-death chain (ie, not underlying cause) and (2) which other contributing causes were most frequently found in the causal chain when AEMT was certified as the underlying cause. Because multiple causes can appear in the causal chain for any single death, the results of this analysis were not mutually exclusive. The secondary analysis involves actual counts and does not represent a modeled estimate; as such, the results are from 1980 through 2014 (the earliest to the most recent year in which data were available for this analysis).
From 1990 to 2016 in the United States, there were 123?603 deaths (95% UI, 100?856-163?814 deaths) in which AEMT represented the underlying cause of death. The absolute number of deaths in which AEMT was the underlying cause increased from 4180 (95% UI, 3087-4993) in 1990 to 5180 (95% UI, 4469-7436) in 2016. Most of this increase was due to population growth and aging, as demonstrated by a 21.4% decrease (95% UI, 1.3%-32.2%) in the national age-standardized AEMT mortality rate over the same period, from 1.46 (95% UI, 1.09-1.76) deaths per 100?000 population in 1990 to 1.15 (95% UI, 1.00-1.60) deaths per 100?000 population in 2016 (Figure 1A). When not exclusively measured as the underlying cause of death, AEMT appeared in the cause-of-death chain in 2.7% of all deaths from 1980 to 2014, which corresponds to AEMT being a contributing cause for an additional 20 deaths for each death when it is the underlying cause. Mortality associated with AEMT as either an underlying or contributing cause appeared in 2.8% of all deaths.
The AEMT mortality rates among men and women were similar (not shown), but there were considerable differences in AEMT mortality with respect to age at death. Per 100?000 population, older individuals had higher mortality rates associated with AEMT compared with younger populations: aged 15-49 years, 0.38 (95% UI, 0.34-0.43) deaths; aged 50-69 years, 1.68 (95% UI, 1.57-2.07) deaths; 70 years or older, 7.93 (95% UI, 7.23-11.45) deaths (Figure 1B). However, when looking at the AEMT mortality cause fraction, children, adolescents, and young adults had a proportionally higher mortality burden from AEMT than older adults (Figure 1C).
Interstate variability in AEMT mortality was observed (Table 1 and Figure 2A). In 2016, California had the lowest age-standardized AEMT mortality rate at 0.84 (95% UI, 0.57-1.47) deaths per 100?000 population, whereas Mississippi had the highest at 1.67 (95% UI, 1.19-2.03) deaths per 100?000 population (Table 1). From 1990 to 2016, mortality estimates in many states decreased. The largest percentage decrease in age-standardized AEMT mortality rate occurred in the District of Columbia (39.9%) followed by New York (33.0%), Maryland (32.2%), New Jersey (30.5%), and California (28.5%) (Table 1 and Figure 2B). In addition, Colorado (16.6%), Oregon (14%), and Virginia (18.8%) all had decreased AEMT mortality after the year 2000.
In the secondary analysis, in which AEMT was listed as the underlying cause of death, 8.9% were due to adverse drug events, 63.6% to surgical and perioperative adverse events, 8.5% to misadventure, 14% to adverse events associated with medical management, 4.5% to adverse events associated with medical or surgical devices, and 0.5% to other AEMT (eTable 6 in the Supplement). The ranking of the subtypes was stable over time (Figure 3A) but with increasing rates of adverse drug events and decreasing rates of misadventure and surgical and perioperative adverse events. Adverse events related to medical or surgical devices and other AEMT were nearly absent in the 1990s but have been responsible for a stable proportion of overall AEMT since the switch to ICD-10 coding of death certificates. Surgical and perioperative adverse events were the most common subtype of AEMT in almost all age groups and increased in importance with age (Figure 3B); misadventure was the largest subtype in neonates, and adverse drug events predominated in individuals aged 20 to 24 years. For full results for when AEMT was the underlying cause of death, see eTables 5 and 6 in the Supplement.
In the final secondary analysis, AEMT as a contributing cause (ie, not underlying cause) of death was examined. Among the 2.7% of all deaths in which AEMT was a contributing cause, surgical and perioperative adverse effects were the most common subtype of AEMT, followed by misadventure (Table 2). Adverse drug events were the biggest contributor in children, adolescents, and elderly individuals, whereas surgical and perioperative adverse events were the most important in neonates and infants and were of increasing importance in older adults. When AEMT was not listed as the underlying cause of death, external injuries were the most commonly associated with AEMT, appearing in the cause-of-death chain of more than 20% of such deaths (eTable 7 in the Supplement).
In a comprehensive study of US mortality associated with AEMT, deaths were observed in every state, in every year, and in each age group. From 1990 to 2016, the absolute number of annual deaths from AEMT increased marginally, whereas the age-standardized mortality rate decreased, suggesting the absolute increase may be largely due to population increases. Several states had significant decreases in mortality over time. Increases in mortality associated with AEMT were observed with advancing age and in a higher proportion among very young individuals. Mortality associated with AEMT as either an underlying or a contributing cause appeared in 2.8% of all US deaths.
Consistent with prior studies, an overall increase in AEMT was observed with advancing age.28,39,40 The vulnerability seen in elderly individuals has been well demonstrated in several areas, including adverse drug events and postoperative complications.41-43 The mortality cause fraction was also shown to be disproportionally higher in young individuals (Figure 1). For pediatric patients (particularly the very young and premature), the additional risk for AEMT may stem from myriad unique considerations in children that present additional safety challenges, such as the dosing and administration of medication, as well as the complexity inherent in often rare pediatric surgical conditions. Taken together, this may also reflect the increased prevalence of medical contact in both the very young and old populations, allowing for more opportunity for AEMT to occur. Although the overall number of events was low among the very young population (Figure 3B), the high cause fraction is concerning and should be an area of further investigation.
State-to-state variability in age-standardized death rates from AEMT was observed (Figure 2A). Estimates between top and bottom performers varied up to 2-fold, perhaps reflecting general regional differences in quality and safety of care that have been noted in other reports.44-46 Variation in health care utilization may be a contributing factor as well. Given that surgical and perioperative AEMT is the most common subtype of AEMT, the southeast region’s higher AEMT mortality estimates may be accounted for by the area’s higher-than-average surgical volume, representing more opportunities for harm to occur.47 The interstate variability in this report highlights the need for further understanding on what may drive regional differences. Other potential contributors include variability in state regulatory efforts (eg, mandatory adverse event reporting), information technology adoption, patient and clinician characteristics, staffing patterns including nurse-patient ratios, and the level to which health systems have adopted care standardization practices. Understanding variability in these domains across states represents an important opportunity for states to learn from one another and to assist in state and national efforts to reduce AEMT.
Patients whose underlying cause of death was traumatic injury were the most likely to have AEMT listed in the cause-of-death chain on their death certificate followed by intermediate causes, such as cardiac arrest and sepsis, cardiovascular diseases, and gastrointestinal conditions that frequently require surgery. Adverse effects of medical treatment are also commonly in the cause-of-death chain of patients whose underlying cause of death is heart disease, which is perhaps a reflection of the large number of patients with heart disease, the complex care these patients receive toward the end of life, and the possibility that they are likely poorly suited to tolerate and recover from an AEMT. These findings highlight the crucial importance of investing in safety and quality in settings where urgent, emergent, and complex procedures are performed, particularly for individuals with several comorbid conditions.
Strengths and Limitations
Measuring patient safety in a comprehensive manner on a national scale is challenging. Available approaches to detect and attribute mortality due to medical harm require trade-offs in generalizability, resource intensity, and detection sensitivity.48,49 The biggest strength of the GBD study’s approach is that it includes the use of a standardized, comprehensive, consistently applied framework for quantifying causes of death over time. It produces internally consistent results by ensuring that the sum of all specific causes of death is exactly equal to all-cause mortality. In addition, the GBD study is comprehensive in scope, covering 23 age groups from birth to 95 years or older in both sexes and in all 50 states using primary data from those locations.
The GBD approach for estimating mortality associated with AEMT also has limitations. First, ICD-coded death certificates have been shown to have varying degrees of reliability in identifying medical harm.50,51 They may have limited ability to distinguish between variation in completeness of death certificate reporting and variation in the occurrence of AEMT events. It is also probable that many deaths involving AEMT are not captured either because of motivated misreporting or unintentional omission. Single institutions may therefore find intensive medical record review to be helpful for quality improvement by more comprehensively assessing death certification practices and capturing temporal relationships and assigning causality between AEMT and death. Second, there is no venue within administrative data to capture the contribution of factors for which there are no ICD codes. For example, although the National Quality Forum lists a “death associated with patient elopement for more than four hours” as a serious reportable adverse event,52 elopement would not be captured on a death certificate. Contributions from latent failures of health systems, a major focus of patient safety, would likewise not be captured here. Third, the sensitive nature of medical harm may result in “motivated misreporting” or changes to documentation practices to avoid implication.53 Fourth, the GBD study’s cause classification system that assigns each death to only a single underlying cause means that some events associated with AEMT may be grouped elsewhere. Such groupings are dependent on which ICD code was assigned as the underlying cause. For example, adverse drug events from prescribed opioids leading to death would likely be assigned to the GBD study’s cause of “opioid abuse” (ICD-10 code, F11) or “accidental poisoning” (ICD-10 code, T40) based on the mechanism of death, whereas they are included with medical harm in many other studies based on the association with a prescription.54,55 Somewhat analogously, nosocomial infections (ICD-10 code, Y95) are often coassigned with a pathogen or type of infection when responsible for a death, and, because Y95 does not end up as the single underlying cause on such death certificates, they are not classified in the GBD study as AEMT.56
Patient safety is and will remain a major priority in the United States health care system, yet gauging progress to date on a national scale has been limited. The results presented here add to the body of work in evaluating AEMT and provide another lens through which to assess the burden of AEMT to further guide prevention efforts. The GBD methodology allows for a regular, comprehensive, and consistent assessment of AEMT within the United States. It may be a particularly powerful tool to assess high-level trends over time and place AEMT in context with other causes of mortality. Further work is needed to understand differences from GBD estimates to other AEMT detection approaches.
This study showed a modest reduction in the death rates from AEMT in the United States from 1990 to 2016 while also observing increased mortality risk with advancing age and certain geographic locations. The annual GBD study releases may allow for tracking of the burden and trend of AEMT over time. In conjunction with other detection systems, the GBD study may provide an increasingly robust assessment of the burden of AEMT across the United States.
Back to top
Accepted for Publication: November 20, 2018.
Published: January 18, 2019. doi:10.1001/jamanetworkopen.2018.7041
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2019 Sunshine JE et al. JAMA Network Open.
Corresponding Author: Mohsen Naghavi, MD, PhD, Institute for Health Metrics and Evaluation, University of Washington, 2301 Fifth Ave, Ste 600, Seattle, WA 98121 (email@example.com).
Author Contributions: Dr Naghavi had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Sunshine, Meo, Kassebaum, Naghavi.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Sunshine, Meo, Kassebaum, Collison, Naghavi.
Critical revision of the manuscript for important intellectual content: Sunshine, Meo, Kassebaum, Mokdad, Naghavi.
Statistical analysis: Kassebaum, Collison, Mokdad, Naghavi.
Obtained funding: Sunshine.
Administrative, technical, or material support: Meo, Kassebaum, Mokdad, Naghavi.
Supervision: Kassebaum, Mokdad, Naghavi.
Conflict of Interest Disclosures: Dr Sunshine reported being an equity stakeholder in Sound Life Sciences, Inc, a company unrelated to the submitted work. Dr Kassebaum reported receiving travel expenses and honoraria from Vifor Pharmaceuticals outside the submitted work. No other disclosures were reported.
Funding/Support: This study was funded in part by the Bill and Melinda Gates Foundation and the Foundation for Anesthesia Education and Research.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
1.Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients: results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324(6):370-376. doi:10.1056/NEJM199102073240604PubMedGoogle ScholarCrossref
2.Kohn LT, Corrigan JM, Donaldson MS. To Err Is Human: Building a Safer Health System. Washington, DC: National Academies Press; 1999.
3.Sorbero MES, Ricci KA, Lovejoy S, et al. Assessment of contributions to patient safety knowledge by the Agency for Healthcare Research and Quality-funded patient safety projects. Health Serv Res. 2009;44(2, pt 2):646-664. doi:10.1111/j.1475-6773.2008.00930.xPubMedGoogle ScholarCrossref
4.Berwick DM, Calkins DR, McCannon CJ, Hackbarth AD. The 100?000 Lives Campaign: setting a goal and a deadline for improving health care quality. JAMA. 2006;295(3):324-327. doi:10.1001/jama.295.3.324ArticlePubMedGoogle ScholarCrossref
5.Institute for Health Care Improvement. Protecting 5 million lives from harm. http://www.ihi.org/Engage/Initiatives/Completed/5MillionLivesCampaign/Pages/default.aspx. Accessed November 18, 2016.
6.Department of Health and Human Services. Patient Safety and Quality Improvement Act of 2005. https://www.hhs.gov/hipaa/for-professionals/patient-safety/statute-and-rule/index.html. Accessed December 9, 2018.
7.Classen DC, Munier W, Verzier N, et al. Measuring patient safety: the Medicare patient safety monitoring system (past, present, and future) [published online October 20, 2016]. J Patient Saf. doi:10.1097/PTS.0000000000000322PubMedGoogle Scholar
8.Jagsi R, Weinstein DF, Shapiro J, Kitch BT, Dorer D, Weissman JS. The Accreditation Council for Graduate Medical Education’s limits on residents’ work hours and patient safety: a study of resident experiences and perceptions before and after hours reductions. Arch Intern Med. 2008;168(5):493-500. doi:10.1001/archinternmed.2007.129ArticlePubMedGoogle ScholarCrossref
9.Furukawa MF, Eldridge N, Wang Y, Metersky M. Electronic health record adoption and rates of in-hospital adverse events [published online February 6, 2016]. J Patient Saf. doi:10.1097/PTS.0000000000000257PubMedGoogle Scholar
10.Medicare.gov. Linking quality to payment. https://www.medicare.gov/hospitalcompare/linking-quality-to-payment.html. Accessed April 13, 2017.
11.Centers for Disease Control and Prevention. 2016 National and State Healthcare-Associated Infections Progress Report. https://www.cdc.gov/hai/data/portal/progress-report.html. Updated October 25, 2018. Accessed December 16, 2018.
12.Wang Y, Eldridge N, Metersky ML, et al. National trends in patient safety for 4 common conditions, 2005-2011. N Engl J Med. 2014;370(4):341-351. doi:10.1056/NEJMsa1300991PubMedGoogle ScholarCrossref
13.Wasfy JH, Zigler CM, Choirat C, Wang Y, Dominici F, Yeh RW. Readmission rates after passage of the Hospital Readmissions Reduction Program: a pre-post analysis. Ann Intern Med. 2017;166(5):324-331. doi:10.7326/M16-0185PubMedGoogle ScholarCrossref
14.Landrigan CP, Parry GJ, Bones CB, Hackbarth AD, Goldmann DA, Sharek PJ. Temporal trends in rates of patient harm resulting from medical care. N Engl J Med. 2010;363(22):2124-2134. doi:10.1056/NEJMsa1004404PubMedGoogle ScholarCrossref
15.Classen DC, Resar R, Griffin F, et al. “Global trigger tool” shows that adverse events in hospitals may be 10 times greater than previously measured. Health Aff (Millwood). 2011;30(4):581-589. doi:10.1377/hlthaff.2011.0190PubMedGoogle ScholarCrossref
16.Hayward RA, Hofer TP. Estimating hospital deaths due to medical errors: preventability is in the eye of the reviewer. JAMA. 2001;286(4):415-420. doi:10.1001/jama.286.4.415ArticlePubMedGoogle ScholarCrossref
17.Levinson DR. Adverse events in hospitals: case study of incidence among Medicare beneficiaries in 2 selected counties. https://oig.hhs.gov/oei/reports/oei-06-08-00220.pdf. Accessed December 9, 2018.
18.Levinson DR. Adverse events in hospitals: national incidence among Medicare beneficiaries. https://oig.hhs.gov/oei/reports/oei-06-09-00090.pdf. Accessed December 9, 2018.
19.Bjørn B, Anhøj J, Østergaard M, Kodal AM, von Plessen C. Test-retest reliability of an experienced Global Trigger Tool Review Team. J Patient Saf. 2017. doi:10.1097/PTS.0000000000000433PubMedGoogle Scholar
20.Reed K, May R. Health Grades Patient Safety in American Hospitals Study. https://patientsafetymovement.org/wp-content/uploads/2016/02/Resources_Reports_Patient_Safety_in_American_Hospitals_Study.pdf. Accessed December 9, 2018.
21.Hibbert PD, Molloy CJ, Hooper TD, et al. The application of the Global Trigger Tool: a systematic review. Int J Qual Health Care. 2016;28(6):640-649.PubMedGoogle Scholar
22.Parry G, Cline A, Goldmann D. Deciphering harm measurement. JAMA. 2012;307(20):2155-2156. doi:10.1001/jama.2012.3649ArticlePubMedGoogle ScholarCrossref
23.Office of Inspector General. Hospital incident reporting systems do not capture most patient harm. https://oig.hhs.gov/oei/reports/oei-06-09-00091.asp. Accessed December 9, 2018.
24.Hutchinson A, Young TA, Cooper KL, et al. Trends in healthcare incident reporting and relationship to safety and quality data in acute hospitals: results from the National Reporting and Learning System. Qual Saf Health Care. 2009;18(1):5-10. doi:10.1136/qshc.2007.022400PubMedGoogle ScholarCrossref
25.Latif A, Rawat N, Pustavoitau A, Pronovost PJ, Pham JC. National study on the distribution, causes, and consequences of voluntarily reported medication errors between the ICU and non-ICU settings. Crit Care Med. 2013;41(2):389-398. doi:10.1097/CCM.0b013e318274156aPubMedGoogle ScholarCrossref
26.Cullen DJ, Bates DW, Small SD, Cooper JB, Nemeskal AR, Leape LL. The incident reporting system does not detect adverse drug events: a problem for quality improvement. Jt Comm J Qual Improv. 1995;21(10):541-548.PubMedGoogle Scholar
27.Thomas EJ, Studdert DM, Newhouse JP, et al. Costs of medical injuries in Utah and Colorado. Inquiry. 1999;36(3):255-264.PubMedGoogle Scholar
28.Miller MR, Elixhauser A, Zhan C, Meyer GS. Patient safety indicators: using administrative data to identify potential patient safety concerns. Health Serv Res. 2001;36(6, pt 2):110-132.PubMedGoogle Scholar
29.Winters BD, Bharmal A, Wilson RF, et al. Validity of the Agency for Health Care Research and Quality patient safety indicators and the Centers for Medicare and Medicaid hospital-acquired conditions: a systematic review and meta-analysis. Med Care. 2016;54(12):1105-1111. doi:10.1097/MLR.0000000000000550PubMedGoogle ScholarCrossref
30.Greenberg MD, Haviland AM, Yu H, Farley DO. Safety outcomes in the United States: trends and challenges in measurement. Health Serv Res. 2009;44(2, pt 2):739-755. doi:10.1111/j.1475-6773.2008.00926.xPubMedGoogle ScholarCrossref
31.Naghavi M, Abajobir AA, Abbafati C, et al; GBD 2016 Causes of Death Collaborators. Global, regional, and national age-sex specific mortality for 264 causes of death, 1980-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet. 2017;390(10100):1151-1210. doi:10.1016/S0140-6736(17)32152-9PubMedGoogle ScholarCrossref
32.National Vital Statistics System. Multiple Cause of Death Data File, 1980-2015. Hyattsville, MD: National Center for Health Statistics; 2015.
33.Centers for Disease Control and Prevention. International Classification of Diseases, Ninth Revision (ICD-9). https://www.cdc.gov/nchs/icd/icd9.htm. Accessed December 9, 2018.
34.World Health Organization. International Statistical Classification of Diseases, Tenth Revision (ICD-10). Geneva, Switzerland: World Health Organization; 1992.
35.Foreman KJ, Lozano R, Lopez AD, Murray CJ. Modeling causes of death: an integrated approach using CODEm. Popul Health Metr. 2012;10:1. doi:10.1186/1478-7954-10-1PubMedGoogle ScholarCrossref
36.Ahmad O, Boschi-Pinto C, Lopez A, Murray C, Lozanno R, Inoue M. Age standardisation of rates: a new WHO standard. https://www.who.int/healthinfo/paper31.pdf. Accessed December 9, 2018.
37.Naghavi M, Makela S, Foreman K, O’Brien J, Pourmalek F, Lozano R. Algorithms for enhancing public health utility of national causes-of-death data. Popul Health Metr. 2010;8(1):9. doi:10.1186/1478-7954-8-9PubMedGoogle ScholarCrossref
38.Houghland K, Nebeker J, Pickard S, et al. Using ICD-9-CM codes in hospital claims data to detect adverse events in patient safety surveillance. In: Henriksen K, Battles JB, Keyes MA, et al, eds. Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 1: Assessment). Rockville, MD: Agency for Healthcare Research and Quality; 2008. https://www.ncbi.nlm.nih.gov/books/NBK43647/. Accessed December 18, 2018.
39.Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II. N Engl J Med. 1991;324(6):377-384. doi:10.1056/NEJM199102073240605PubMedGoogle ScholarCrossref
40.Thomas EJ, Brennan TA. Incidence and types of preventable adverse events in elderly patients: population based review of medical records. BMJ. 2000;320(7237):741-744. doi:10.1136/bmj.320.7237.741PubMedGoogle ScholarCrossref
41.Gurwitz JH, Field TS, Harrold LR, et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA. 2003;289(9):1107-1116. doi:10.1001/jama.289.9.1107ArticlePubMedGoogle ScholarCrossref
42.Strausbaugh LJ. Emerging health care–associated infections in the geriatric population. Emerg Infect Dis. 2001;7(2):268-271. doi:10.3201/eid0702.700268PubMedGoogle ScholarCrossref
43.Turrentine FE, Wang H, Simpson VB, Jones RS. Surgical risk factors, morbidity, and mortality in elderly patients. J Am Coll Surg. 2006;203(6):865-877. doi:10.1016/j.jamcollsurg.2006.08.026PubMedGoogle ScholarCrossref
44.McKellar MR, Landrum MB, Gibson TB, Landon BE, Fendrick AM, Chernew ME. Geographic variation in quality of care for commercially insured patients. Health Serv Res. 2017;52(2):849-862. doi:10.1111/1475-6773.12501PubMedGoogle ScholarCrossref
45.Zhang Y, Baicker K, Newhouse JP. Geographic variation in the quality of prescribing. N Engl J Med. 2010;363(21):1985-1988. doi:10.1056/NEJMp1010220PubMedGoogle ScholarCrossref
46.Welch WP, Miller ME, Welch HG, Fisher ES, Wennberg JE. Geographic variation in expenditures for physicians’ services in the United States. N Engl J Med. 1993;328(9):621-627. doi:10.1056/NEJM199303043280906PubMedGoogle ScholarCrossref
47.Goodney PR, Dzebisashvili N, Goodman D, et al. Variation in the care of surgical conditions. http://archive.dartmouthatlas.org/downloads/atlases/Surgical_Atlas_2014.pdf. Accessed January 18, 2018.
48.Shojania KG, Dixon-Woods M. Estimating deaths due to medical error: the ongoing controversy and why it matters. BMJ Qual Saf. 2017;26(5):423-428.PubMedGoogle Scholar
49.Brennan TA. The Institute of Medicine report on medical errors—could it do harm? N Engl J Med. 2000;342(15):1123-1125. doi:10.1056/NEJM200004133421510PubMedGoogle ScholarCrossref
50.Lawthers AG, McCarthy EP, Davis RB, Peterson LE, Palmer RH, Iezzoni LI. Identification of in-hospital complications from claims data: is it valid? Med Care. 2000;38(8):785-795. doi:10.1097/00005650-200008000-00003PubMedGoogle ScholarCrossref
51.McCarthy EP, Iezzoni LI, Davis RB, et al. Does clinical evidence support ICD-9-CM diagnosis coding of complications? Med Care. 2000;38(8):868-876. doi:10.1097/00005650-200008000-00010PubMedGoogle ScholarCrossref
52.National Quality Forum (NQF). Serious Reportable Events in Healthcare—2011 Update: A Consensus Report. Washington, DC: NQF; 2011. https://www.qualityforum.org/Publications/2011/12/Serious_Reportable_Events_in_Healthcare_2011.aspx. Accessed December 20, 2018.
53.Gnambs T, Kaspar K. Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis. Behav Res Methods. 2015;47(4):1237-1259. doi:10.3758/s13428-014-0533-4PubMedGoogle ScholarCrossref
54.Whiteford HA, Degenhardt L, Rehm J, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet. 2013;382(9904):1575-1586. doi:10.1016/S0140-6736(13)61611-6PubMedGoogle ScholarCrossref
55.Rudd RA, Seth P, David F, Scholl L. Increases in drug and opioid-involved overdose deaths—United States, 2010-2015. MMWR Morb Mortal Wkly Rep. 2016;65(50-51):1445-1452. doi:10.15585/mmwr.mm655051e1PubMedGoogle ScholarCrossref
56.Govindan S, Shapiro L, Langa KM, Iwashyna TJ. Death certificates underestimate infections as proximal causes of death in the US. PLoS One. 2014;9(5):e97714. doi:10.1371/journal.pone.0097714PubMedGoogle ScholarCrossref