The Office of the National Coordinator for Health Information Technology (ONC) might be making too much out of a new study in the Annals of Internal Medicine? In a post on Health IT Buzz, the ONC Director of the Office of Economic Analysis, Evaluation, and Modeling Michael Furukawa and Economist Meghan Gabriel have asserted that the study conducted by RAND researchers “found strong evidence that health IT improves patient outcomes.”
To be clear, ONC-funded study is a review of studies related to health IT functionalities required by the EHR Incentive Programs for eligible professionals and hospitals. It is not itself a study of meaningful use so much as literature review.
The most concrete evidence in support of meaningful use pertains to the increase in the number of studies between 1995 and 2013. Before 2007, these studies increased annually by roughly 13 percent; between 2008 and 2012, the percentage doubled to around 25 percent. Of those studies, clinical decision support and computerized physician order entry “have been studied extensively” whereas functionalities including health information exchange (HIE) and EHR patient access “are not well-studied.”
And here’s the kicker from the study itself:
Although the health IT evaluation literature base is expanding rapidly, we are concerned that there has not been a commensurate increase in our understanding of the effect of health IT or how it can be used to improve health and health care. Study questions, research methods, and reporting of study details have not sufficiently adapted to meet the needs of clinicians, health care administrators, and health policymakers and are falling short of addressing the future needs of the health care system.
Of the conclusions the researchers were willing to draw, the big takeaway is that both CPOE and CDS have received positive reviews in the context of improving care delivery and decreasing the number of medication errors.
So what then does the study actually prove? “In sum, the health IT literature is expanding rapidly but failing to produce a commensurate amount of useful knowledge,” the authors explain. “Although most studies reported that health IT interventions had statistically and clinically significant benefits, sometimes these were not as large as the developers had expected, and there are also examples where benefits were not realized.”
As tends to be the case, the solution to the problem is more research with a focus on meaningful outcomes:
Now that health IT is being widely adopted, researchers should refocus their efforts to show how health IT can be used to realize value. The most common characteristics of published studies are still pre–post studies of a CDS or CPOE at a single site that report nothing beyond the basics in terms of context and implementation. Such studies should be discouraged. The most important improvement that can be made in health IT evaluations is increased measurement of and reporting of context, implementation, and context-sensitivity of effectiveness.
Meaningful use is likely making a difference in a positive way, but it needs to be proven with actual data, not simply a review of the literature. Source
