Research from the United Kingdom provides a glimpse into the challenges following the widespread adoption of EHR systems and capturing of patient data electronically. “The present use of EHR databases requires programing expertise and understanding of the nuances of the coding systems. Queries may take hours or even days to run, and once obtained the only way most researchers can view the results is to scroll through hundreds of records in tabular form,” the authors write
The research team of Tate et al. developed an online user-focused system called TrialViz to improve the searchability of patient data stored in the Clinical Practice Research Datalink (CPRD) database, which includes anonymized primary care patient records for 5.5 million registered patients. In particular, the researchers identified four key challenges.
The first challenge was the difficulty surrounding the ability to abstract meaningful data from the massive amount of data stored in the CPRD database. “One of the main challenges of working with UK GP data is the number and complexity of the queries that need to be carried out in order to sift out relevant information from the mass of (mainly irrelevant) data,” note Tate et al. “There may be thousands of records for each patient, and there may be numerous codes for each disease or type of symptom representing essentially the same thing.”
Another major challenge is the process time involved in running queries. For data to be explored in an interactive and robust manner, results of queries should be returned in real or near real time. Most SQL-based systems that exist at present are not capable in general of doing this. SQL is known to be erratic when working with large datasets, that is, if the parser decides to execute a query in an inefficient manner, large data volumes tend to exacerbate the delinquent behavior, resulting in query execution time of several hours or days rather than expected seconds.
The third challenge dealt with data quality — that is, assurances that data are of sufficient quality to support reliable searches. “In any application based on data, it is very important to ensure that the data are of high quality. For example, it is unethical to enroll patients into an experiment in which poor quality data collection could lead to invalid results. Poor quality data lead to poor searches with low efficiency for screening visits,” add the authors.
The fourth and final challenge centered on the end user and making the process of querying patient information approachable for users of various degrees of technical expertise. “An inherent part of any software application development is a strong user involvement at all stages of the development cycle, from the conception of requirements specification, through the prototype testing to the usability evaluation,” observes Tate et al.
The research team has provided a series of solutions for mitigating these challenges:
• In terms of data abstraction, it developed a web portal that gives users interactive access to the entire CPRD database and visualized search results to improve the legibility and usability of data.• To speed the processing times for queries, Tate et al. relied on the efficiency afforded by solid state drives (SSDs) as well as the use of a three-database infrastructure: one for the user interface, another for storing cached results, and the third containing all the CPRD patient data.• The researchers highlighted the importance of the UK National Health Service quality outcomes framework in improving the recording of measures across the nation.• Lastly, a robust program for evaluating users was used to assess the needs of end users.
