A thorough systematic review assessing methods for dealing with missing data in electronic health records (EHRs) was carried out by researchers from Peking University’s National Institute of Health Data Science and Peking University People’s Hospital’s Department of Clinical Epidemiology and Biostatistics. The study, which was published in Health Data Science, emphasizes how machine learning techniques are becoming more and more crucial than conventional statistical methods for handling missing data situations.
Because they allow for analysis of clinical trials, treatment effectiveness studies, and genetic association research, electronic health records have emerged as a key component of contemporary healthcare research. Missing data, however, continues to be a problem since it can introduce bias and compromise the validity of results. This study examined 46 research papers from 2010 to 2024, methodically contrasting the effectiveness of contemporary machine learning techniques like k-Nearest Neighbors (KNN) and Generative Adversarial Networks (GANs) with more conventional statistical techniques like Multiple Imputation by Chained Equations (MICE).
The results show that while addressing both longitudinal and cross-sectional datasets, machine learning techniques—in particular, GAN-based methods and context-aware time-series imputation (CATSI)—consistently performed better than conventional statistical approaches. While probabilistic principle component analysis (PCA) and MICE performed better for cross-sectional datasets, Med.KNN and CATSI performed better for longitudinal data.
The potential of machine learning techniques to solve missing data in EHRs is substantial. The necessity for uniform benchmarking analyses across various datasets and missingness circumstances is highlighted by the fact that no single method provides a solution that is generally applicable.
Associate Professor Dr. Huixin Liu of Peking University People’s Hospital
The opacity of machine learning models, the variability of EHR datasets, and the absence of common standards for evaluating technique success are some of the major issues the report also highlights. Future studies seek to create benchmarking datasets for thorough assessment and standardize the process for managing missing EHR data.
According to Dr. Shenda Hong, an assistant professor at Peking University’s National Institute of Health Data Science, “our ultimate goal is to create a universally accepted protocol for handling missing data in electronic health records, ensuring more reliable and reproducible findings across medical research,” she added.
By providing insights that can aid in bridging the gap between robust analysis and data paucity, this research represents a big step toward tackling one of the most critical difficulties in digital healthcare research.
