doi: 10.1161/CIRCEP.108.819300
Editorials |
Publicly Reporting Implantable Cardioverter Defibrillator Outcomes
Grading the Report Card
From the Mayo Clinic (S.C.H.), Rochester, Minn; and Yale University (J.C.), New Haven, Conn.
Correspondence to Stephen C. Hammill, MD, Mayo Clinic, Rochester, MN 55905. E-mail hammill.stephen{at}mayo.edu
Key Words: Editorials • registries • implantable defibrillator
The past decade has seen a dramatic increase in efforts by both payors and governmental agencies to measure and publicly report patient outcomes to ensure that patients receive appropriate, high-quality care. As a result, terms such as outcomes, quality, report cards, and benchmarking have entered the lexicon of clinical medicine, and both physicians and hospitals are increasingly being held accountable for the outcomes of their patients. This movement shows no sign of slowing down. In May 2008, the Centers for Medicare and Medicaid Services proposed increasing the total number of inpatient hospital measures from 40 to 73,1 and numerous physician measures are in various phases of development. In the current environment, it is likely that implantable cardioverter defibrillators (ICDs), which reduce the risk of sudden cardiac death,2 will soon emerge as a high-priority target for the development of performance measures, given that the procedure is expensive, performed on severely ill patients, and associated with significant risk of complications.
Article see p 240
Accordingly, the findings published by Al-Khatib et al3 in this issue of Circulation: Arrhythmia and Electrophysiology warrant consideration in the context of current efforts to publicly report patient outcomes using Medicare claims data. In this excellent article, the authors used data from Centers for Medicare and Medicaid Services standard analytic files to determine the 90-day postprocedure complication rates and 1-year mortality rates of 8581 Medicare beneficiaries who had an ICD implanted by one of 1959 implanting physicians. The authors note that the complication rate declined from 18.8% in 2002 to 14.2% in 2005 and further identified specific factors associated with an increased risk of complication such as chronic lung disease, dementia, and renal disease. Of interest, physician experience as measured by procedure volume was not significantly associated with the risk of complication, although an earlier article did report a significant inverse relationship between implantation volume and complications.4 Furthermore, the risk of complications was not higher when implanting ICDs with cardiac resynchronization capability, despite the use of the additional left ventricular lead. However, as correctly reported by the authors, their study may not have had enough power to address this question.
These findings are informative, but the most striking finding of this study was that patients whose ICD had been implanted by a thoracic surgeon were at a significantly higher risk of complication as compared with other physician types (electrophysiology cardiologist, 15.9%; nonelectrophysiology cardiologist, 15.3%; thoracic surgeon, 26.4%; other, 15.4%; P<0.001). These findings persisted after risk adjustment for measured differences in patient characteristics. Although the authors are appropriately circumspect in their discussion, their findings may have far-reaching implications.
The key question is whether the approach of Al-Khatib et al3 could be used as a basis for developing hospital- or physician-based measures of patient outcomes after ICD implantation. The identification of clinically important differences in complication rates by implanting physician specialty will certainly make this a tempting candidate measure, and accordingly, it is worth critically examining the methodological approach of Al-Khatib et al3 through the lens of measure development.
The American Heart Association and American College of Cardiology have issued a scientific statement that details attributes of performance measures.5 Briefly summarized, measures must have the potential to improve patient outcomes by being evidence based, easily interpretable, and actionable; be well designed to identify clearly defined populations, use valid risk adjustment methods, and provide reliable results; and allow implementation at a reasonable cost. These organizations have also published guidelines outlining the risk adjustment methodology necessary for publicly reported outcomes measures. Such models should (1) use a clear and explicit definition of the patient population; (2) include clinically sensible covariates for risk adjustment; (3) use high-quality and timely data; (4) exclude variables that could represent complication rather than comorbidities; (5) designate an appropriate outcome evaluated at a standardized period of time; (6) apply an appropriate statistical approach; and (7) disclose the methods in detail, including information about the performance of the measurement system.5,6
Although Al-Khatib et al3 almost certainly did not design their model for this purpose, their methods and results are generally aligned with these attributes. For example, they have a clearly defined study cohort based on established and temporary Healthcare Common Procedure Coding System codes developed by Medicare for reimbursement of ICD implantation. Patient outcomes were assessed over a standard 90-day period using an algorithm developed using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes to identify conditions that likely represent complications of ICD implantation. Comorbid conditions that would be associated with increased risk of complication were similarly identified through a coding algorithm using ICD-9-CM codes. The statistical methods for risk adjustment accounted for the clustering of similar patients within hospitals. Furthermore, the 16.2% overall complication rate is clinically significant, and the identification of differences in complication rates by physician specialty suggests it could be actionable. Accordingly, this approach could serve as a foundation for a new ICD-related performance measure.
However, can a risk-adjustment methodology based on administrative data provide fair comparisons across different groups, whether they be hospitals, physicians, or, in the current example, physician specialties? Given the findings of Al-Khatib et al,3 one can most certainly anticipate a unanimous cry of "no" from thoracic surgeons. They will maintain that their patients are sicker in ways that administrative data simply cannot capture, and their concerns are not without merit. Criticisms of administrative claims data are well established. Although administrative data are inexpensive and readily available, these databases were originally designed for claims submission and benefits coordination, not outcomes profiling.7 Administrative databases are vulnerable to inaccuracies due to imprecise or ambiguous definitions, and coding may not be supported by the clinical record. Fisher et al8 evaluated the medical records from 239 hospitals and found that the ICD-9-CM coding of a principle diagnosis had an overall error rate of approximately 22% when compared with structured medical record review. Additional problems with ICD-9-CM accuracy include (1) hospital coding that may be biased toward maximizing reimbursement; (2) incomplete coding of nonessential data; (3) errors caused by redundant codes; (4) errors caused by the staff responsible for entering the data; and (5) specific hospital idiosyncrasies.9 Distinguishing comorbidities from complications using ICD-9-CM codes can be challenging.7 Finally, administrative data lacks information about important clinical conditions, such as left ventricular dysfunction or heart failure severity, which are associated with risk of complications. As such, risk-adjustment models based on administrative data may not fully account for differences in case mix among institutions and physicians.6
And yet, these barriers are not insurmountable. Krumholz et al10 published a well-designed model for the calculation of hospital-level risk standardized mortality rates for myocardial infarction using administrative claims data. This study evaluated a sample of >4500 hospitals and reported that the estimates of hospital-specific risk adjusted mortality rates from the claims data–based model were strongly correlated with estimates from a model based on medical record data. Similar results have been reported for heart failure,11 and the resulting models have been endorsed by the National Quality Forum.6,12 Therefore, when done carefully, Medicare claims data can be used for reporting of outcomes. The question is not whether it can be done, but whether it can be done better.
The major rationale for using administrative data is feasibility. Obtaining chart-level data is both expense and resource intensive. However, prospective registries maintained by professional societies may be a source of data that can correct the inherent deficiencies of administrative data. Registry data are typically more detailed and accurate than administrative data, and these differences can have significant implications for public reporting. Shahian et al7 compared hospital cardiac surgery profiling results based on clinical data derived from the Society of Thoracic Surgery’s prospective clinical database with results from hospital discharge billing data collected from the Massachusetts Division of Health Care, Finance, and Policy. The analysis demonstrated that these methods yielded substantially different results. The authors concluded that cardiac surgery report cards that use audited and validated clinical data are superior to those derived from administrative data.7
Performance measures based on registry data may enhance the reliability of the data, allow for meaningful comparisons between physicians and hospitals, and improve physician acceptance. In this light, it is useful to consider the National ICD Registry operated by the National Cardiovascular Data Registry.13–15 The National ICD Registry is the sole repository for ICD implantation data for Medicare beneficiaries receiving an ICD for primary prevention and, as of August 2008, had collected data from 1478 hospitals in the United States, totaling >300 600 implantations. The procedures were performed by 3899 physicians of whom 57.5% completed an electrophysiology fellowship, 11.3% completed a thoracic or cardiac surgery residency, 16.5% completed the Heart Rhythm Society ICD implantation guidelines,16 and 14.6% reported no specific training.15 The registry captures detailed information regarding demographics, cardiac status, and the presence of comorbid conditions. The registry is accruing patients at a rate of >10 000 per month, with 88% of implants in the registry being performed in hospitals that submit data on all patients, regardless of device indication or payor status. This full reporting of data provides a comprehensive characterization of contemporary practice and may allow more robust inferences about performance across groups when compared with analyses limited to Centers for Medicare and Medicaid Services–eligible patients. In addition, quality improvement is already the major focus of the ICD Registry, with benchmarking reports provided to hospitals on a quarterly basis. Accordingly, creating performance measures using data from the ICD Registry is consistent with the overarching mission of the registry. Notably, the National ICD Registry has reported similar findings as those of Al-Khatib et al3 regarding the association of physician certification and in-hospital outcomes; thoracic surgeons had a higher short-term risk of procedural complication when compared with electrophysiologists (relative risk, 1.40; 95% CI, 1.11 to 1.76; P<0.004).17
A significant limitation of the current ICD registry is that follow-up is limited to the initial hospitalization for the device implantation. However, this may not be a fatal flaw. Al-Khatib et al3 noted that the majority of complications occurred during the index hospitalization, suggesting that performance measures based on in-hospital outcomes may be an acceptable, if not ideal, compromise. Alternative approaches might include expanding the scope of the registry to include 30- or 90-day outcomes, as has been done in the National Cardiovascular Data Registry’s carotid stent registry, or merging registry data with administrative data to create a hybrid model that capitalizes on the best qualities of both data sources. Whatever the specific solution, the existence of a large, national ICD registry provides an attractive alternative to the use of administrative data, which warrants further evaluation. Al-Khatib et al3 have identified a new, potentially important outcome that could serve as a basis for a new performance measure. It will be everyone’s responsibility—payors, providers, and regulators—to see that the resulting measures are implemented in a manner that is accurate, equitable, and accepted.
 |
Acknowledgments |
|---|
Disclosures
None.
|
Footnotes |
|---|
The opinions expressed in this article are not necessarily those of the editors or of the American Heart Association.
|
References |
|---|
 Top References |
|---|
1. Federal Register, Volume 73, No. 84, April 30, 2008.
2. Bardy GH, Lee KL, Mark DB, Poole JE, Packer DL, Boineau R, Domanski M, Troutman C, Anderson J, Johnson G, McNulty SE, Clapp-Channing N, Davidson-Ray LD, Fraulo ES, Fishbein DP, Luceri RM, Ip JH. Sudden Cardiac Death in Heart Failure Trial (SCD-HeFT) Investigators. Amiodarone or an implantable cardioverter-defibrillator for congestive heart failure. N Engl J Med. 2005; 352: 225–37.
3. Al-Khatib SM, Greiner MA, Peterson ED, Hernandez AF, Schulman KA, Curtis LH. Patient and implanting physician factors associated with mortality and complications following implantable cardioverter-defibrillator implantation 2002–2005. Circ Arrhythmia Electrophysiol. 2008; 1: 240–249.
4. Al-Khatib SM, Lucas FL, Jollis JG, Malenka DJ, Wennberg DE. Relation between patients’ outcomes and the volume of cardioverter defibrillator implantations performed at hospitals treating medicare beneficiaries. J Am Coll Cardiol. 2005; 46: 1536–1540.
5. Krumholz HM, Brindis RG, Brush JE, Cohen DJ, Epstein AJ, Furie K, Howard G, Peterson ED, Rathore SS, Smith SC Jr., Spertus JA, Wang Y, Normand SL. Standards for statistical models used for public reporting of health outcomes: an American Heart Association scientific statement from the quality of care and outcomes research interdisciplinary writing group: co-sponsored by the council on epidemiology and prevention and the stroke council: endorsed by the American College of Cardiology Foundation. Circulation. 2006; 113: 456–462.
6. Krumholz HM, Normand ST, Sperties JA, Shahian DM, Bradley EH. Measuring performance for treating heart attacks and heart failure: the base for outcomes measurement. Health Affairs. 2007; 26: 75–85.
7. Shahian DM, Silverstein T, Lovett AF, Wolf RE, Normand ST. Comparison of clinical and administrative data sources for hospital coronary artery bypass graft surgery report cards. Circulation. 2007; 115: 1518–1527.
8. Fisher ES, Whaley FS, Krushat WM, et al. The accuracy of Medicare’s Hospital Claims Data: progress has been made, but problems remain. Am J Public Health. 1992; 82: 243–248.
9. Birman-Deych E, Waterman AD, Yan Y, Nilasena DS, Radford MJ, Gage BF. Accuracy of ICD-9-CM Codes for identifying cardiovascular and stroke risk factors. Med Care. 2005; 43: 480–485.[CrossRef][Medline]
10. Krumholz HM, Wang Y, Mattera JA, Wang Y, Han LF, Ingber MJ, Roman S, Normand SL. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with an acute myocardial infarction. Circulation. 2006; 113: 1683–1692.
11. Krumholz HM, Wang Y, Mattera JA, Wang Y, Han LF, Ingber MJ, Roman S, Normand SL. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with heart failure. Circulation. 2006; 113: 1693–1701.
12. http://www.qualitynet.org/dcs/ContentServer?cid=1163010398556&page-name=QnetPublic%2FPage%2FQnetTier2&c=Page
13. Hammill SC, Stevenson LW, Kadish AH, Kremers MS, Heidenenreich PA, Lindsay BD, Mirro MJ, Radford MJ, Curtis JP, Lang CM, Harder JC, Brindis RG. National ICD Registry Annual Report 2006. Review of the registries first year, data collected, and future plans. Heart Rhythm. 2007; 4: 1260–1263.[CrossRef][Medline]
14. Hammill S, Phurrough S, Brindis R. The National ICD Registry: now and into the future. Heart Rhythm. 2006; 3: 470–473.[CrossRef][Medline]
15. Hammill SC, Kremers MS, Stevenson LW, Kadish AH, Heidenreich PA, Lindsay BD, Mirro MJ, Radford MJ, Wang Y, Curtis JP, Lang CM, Harder JC, Brindis RG. National ICD Registry Annual Report 2007. Review of the registry’s second year, data collected, and plans to add lead and pediatric ICD procedures. Heart Rhythm. 2008; 5: 1359–1363.[CrossRef][Medline]
16. Curtis AB, Ellenbogen KA, Hammill SC, Hayes DL, Reynolds DW, Wilber DJ, Cain ME. Clinical competency statement: training pathways for implantation of cardioverter-defibrillators and cardiac resynchronization devices. Heart Rhythm. 2004; 1: 371–375.[CrossRef][Medline]
17. Leubbert J, Hammill S, Wang Y, Rathore SS, Lampert R, Krumholz HM, Heidenreich P, Curtis JP. Association of implanting physician certification with complications following implantable cardioverter-defibrillator procedures: insights from the ICD Registry [abstract]. Circulation. In press.
Related Article
Circ Arrhythm Electrophysiol 2008 1: 240-249.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||