0
Academia and the Profession |

An Electronic Practice-Based Network for Observational Comparative Effectiveness Research FREE

Wilson D. Pace, MD; Maribel Cifuentes, RN, BSN; Robert J. Valuck, PhD; Elizabeth W. Staton, MSTC; Elias C. Brandt, BS; and David R. West, PhD
[+] Article and Author Information

From University of Colorado Denver School of Medicine and University of Colorado Denver School of Pharmacy, Aurora, Colorado, and American Academy of Family Physicians National Research Network, Leawood, Kansas.


Grant Support: By the Agency for Healthcare Research and Quality (contract HHSA 290200500371, task order 2).

Potential Financial Conflicts of Interest: None disclosed.

Requests for Single Reprints: Wilson D. Pace, MD, University of Colorado Denver, School of Medicine, Department of Family Medicine, Mailstop F496, Academic Office 1, 12631 East 17th Avenue, Aurora, CO 80045-0508; e-mail, wilson.pace@ucdenver.edu.

Current Author Addresses: Drs. Pace and West, Ms. Cifuentes, and Ms. Staton: University of Colorado Denver, School of Medicine, Department of Family Medicine, Mailstop F496, Academic Office 1, 12631 East 17th Avenue, Aurora, CO 80045-0508.

Dr. Valuck: University of Colorado Denver, School of Pharmacy, Mailstop C238, Academic Office 1, 12631 East 17th Avenue, Aurora, CO 80045-0508.

Mr. Brandt: American Academy of Family Physicians National Research Network, 11400 Tomahawk Creek Parkway, Leawood, KS 66211.


Ann Intern Med. 2009;151(5):338-340. doi:10.7326/0003-4819-151-5-200909010-00140
Text Size: A A A

A distributed health data network is a system that allows secure remote analysis of separate data sets, each comprising a different medical organization's or health plan's records. Distributed health data networks are currently being planned that could cover millions of people, permitting studies of comparative clinical effectiveness, best practices, diffusion of medical technologies, and quality of care. These networks could also support assessment of medical product safety and other public health needs. Distributed network technologies allow data holders to control all uses of their data, which overcomes many practical obstacles related to confidentiality, regulation, and proprietary interests. Some of the challenges and potential methods of operation of a multipurpose, multi-institutional distributed health data network are described.

Key Summary Points

The Distributed Ambulatory Research in Therapeutics Network (DARTNet) is a federated network linking health data from 8 organizations representing more than 500 clinicians and more than 400 000 patients.

Electronic health record and other clinical data are aggregated, standardized, and stored within each organization, then deidentified and made available for secure queries through the Web.

A full set of patient data never leaves the individual clinical sites.

DARTNet can be used for observational comparative effectiveness research, which may provide important information about safe and effective health care.

DARTNet tools can prompt clinicians to obtain specific information during a patient encounter.

Key clinical information traditionally missing in comparative effectiveness research is now available through DARTNet.

The Distributed Ambulatory Research in Therapeutics Network (DARTNet) is a federated network of electronic health data that was created to facilitate advanced observational comparative effectiveness research and examine outcomes associated with prescription medications and devices. A federated network links geographically and organizationally separate databases so that a single database query can return results from multiple databases while maintaining the privacy and confidentiality of patient data. DARTNet currently links patient-level clinical data from 8 organizations representing more than 500 clinicians and more than 400 000 patients. This article presents an overview of DARTNet, explains the contributions of DARTNet to the field of comparative effectiveness research, and summarizes the initial research study and lessons learned.

The Agency for Healthcare Research and Quality funded the creation of DARTNet through its Developing Evidence to Inform Decisions about Effectiveness Network. DARTNet was created to examine outcomes associated with prescription medications, devices, and various approaches to medical care.

Within each member organization, DARTNet assembles patient-level information (such as vital signs, social history, family history, and physical examination findings) from electronic health records, laboratory tests, imaging results, pharmacy utilization databases, and billing systems. All of this information is put into 1 database that is deidentified and made available for secure access through the Web. Each organization's aggregated, deidentified clinical database is linked to similar databases in other DARTNet member organizations; the relationship among these databases is the federated network.

The DARTNet research team can submit a precise request for information through a database query. The query runs simultaneously on each federated database, and the results provide highly detailed information about care and outcomes for hundreds of thousands of patients across the United States. The DARTNet technology currently works with 5 brands of ambulatory electronic health records, but it is capable of working with essentially any ambulatory electronic health record.

In addition to facilitating queries among the standardized and federated databases, the DARTNet system can prompt clinicians to obtain specific information during a patient encounter. This capability allows the research team to collect additional data beyond what would normally be recorded in electronic records. Thus, DARTNet is designed to combine the essential elements of observational research with elements of a practical clinical trial (1).

DARTNet is also designed to help support a learning community. By comparing clinical care provided across the network, high-performing practices and systems can be identified. One expectation of DARTNet membership is that high-performing sites will share their approaches to care with other members. The DARTNet staff is working to develop methods and systems to support this process. The learning community process is one of the main reasons that clinicians and practices wish to join DARTNet.

By allowing examination of routine care from many clinicians, DARTNet provides insight that would not otherwise be available. Much of the current evidence base for health care depends on the results of randomized trials; however, those trials do not adequately account for the variability seen in actual care (1). Furthermore, randomized trials are not a practical way to answer important questions about the many possible variations in chronic disease treatment (1). Patients, physicians, payers, purchasers, health care administrators, and public health policymakers need better information to compare the effectiveness of treatments and thus make sound health care decisions from the individual to the national level (1).

Traditionally, observational comparative effectiveness researchers examine data sets created for other purposes, such as insurance claims data. Cohorts of patients created from large populations are then compared. Statistical matching algorithms are applied to try to account for underlying clinical differences among the groups (23), and the resulting analysis provides comparisons of outcomes of treatment. Although this methodology is powerful, it has been criticized because it cannot account for important clinical information not available in claims databases (4).

Two brief examples demonstrate how the findings of a comparative effectiveness study might be skewed. First, a typical observational comparative effectiveness study of oral hypoglycemic medications cannot account for body mass index differences between cohorts. In addition, a comparative effectiveness examination of liver toxicity related to a particular medication would be much more robust if it could account for the use of alcohol or acetaminophen among cohort patients. DARTNet provides key missing clinical information of this type, which will advance the field of comparative effectiveness research.

To demonstrate the capabilities of DARTNet, we conducted a retrospective cohort study of patterns of use and comparative effectiveness and safety of oral diabetes medications for adults with type 2 diabetes. In phase 1 of the study, a limited set of data elements from a commercially available, integrated medical claims database (the Ingenix National Managed Care Impact database [Ingenix, Eden Prairie, Minnesota]) was used to examine the comparative effectiveness and safety of oral diabetes medications. This is a standard approach to observational comparative effectiveness research. A secondary aim was to identify limitations in claims databases for which DARTNet could supply useful and otherwise unavailable information.

Phase 2 was designed to replicate phase 1 of the study, this time using DARTNet data. For phase 2, we studied a smaller patient sample but examined a broader range of data elements. We looked at electronic health record data and tested point-of-care data collection. Phase 2 findings show that DARTNet identified similarly sized panels of diabetic patients and patients receiving various oral diabetes medications, permitting analyses of similar power to the claims-based study in phase 1. In additional, DARTNet added useful clinical data (such as body weight, height, self-reported alcohol intake, and self-reported hypoglycemic events) to the analysis of comparative effectiveness of oral diabetes medications.

Our experience in developing DARTNet indicates that the system has broad utility and power. This power derives from the system's ability to access and standardize an array of data (approximately 150 data elements at this time) from various electronic health records and other clinical databases. Nonetheless, we encountered difficulty locating particular types of data in the electronic health records. It can be challenging to standardize data from separate practices, even those using the same electronic health record (5). We also found that many data fields in electronic health records lack reasonable range checks. For instance, most electronic health records will accept a systolic blood pressure in the thousands of mm Hg. Correction of these errors, although infrequent, will require work from the developers of electronic health records.

DARTNet currently relies on a third-party clinical decision support system to extract and aggregate clinical data at each organization (6). The ongoing use of extracted data for clinical purposes serves as a continuous quality control feedback process: Clinicians who rely on the clinical decision support quickly recognize errors in the data and therefore routinely correct problems. Even with the use of powerful clinical decision support tools, we encountered difficulty in our attempts to view a discrete “episode of care” that spans multiple encounters. Many essential features linked to a given visit, including diagnoses, medication refills, and procedures, can be lost when the data are extracted for clinical decision support purposes. To improve its ability to examine episodes of care, DARTNet will include billing data to better distinguish encounters and procedures as well as focused point-of-care data collection.

Even with effective decision support tools at their disposal, practices varied greatly in clinical outcomes and performance. This could be construed as a data quality issue, but it also highlights the potential for the system to add value to members through their participation in a learning community that identifies top-performing members, disseminates best practices, and provides facilitation to enhance clinical care at the practice level.

The DARTNet prototype was connected to both small offices and large group practices specifically to demonstrate that it can include a wide array of organizations with varying informatics configurations, support levels, and ability to manipulate their information management environment. The next steps for DARTNet will be to expand its technical capabilities, expand the patient base, define the final corporate and organizational structure of the network, define the selection process for research and quality improvement projects, and further develop its capacity as a learning community (7).

In conclusion, DARTNet is capable of bidirectional electronic communication with practices that use electronic health records, and it holds great potential for becoming a valued tool for observational comparative effectiveness research while improving the quality of care provided by its members. Successfully combining the concepts of point-of-care data collection with secondary data analysis of large populations of patients should substantially advance the ability to understand the effects of various medical interventions in routine practice. A comprehensive online report about DARTNet will become available on the Agency for Healthcare Research and Quality's Web site (www.ahrq.gov).

Tunis SR, Stryer DB, Clancy CM.  Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003; 290:1624-32. PubMed
CrossRef
 
Rubin DB.  Estimating causal effects from large data sets using propensity scores. Ann Intern Med. 1997; 127:757-63. PubMed
 
Rubin DB.  Bias reduction using Mahalanobis-metric matching. Rubin DB Matched Sampling for Causal Effects. London: Cambridge Univ Pr; 2006.
 
Schneeweiss S, Avorn J.  A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. 2005; 58:323-37. PubMed
 
The Health Industry Forum.  “Road Testing” Electronic Health Records for Effectiveness Research. Conference Report. Accessed athttp://healthforum.brandeis.edu/meetings/materials/2008-4-June/Conference%20Report%20HIF%20EMR%20Event%206-4-08%20Final.pdf. on 4 June 2008.
 
Clinical Integration Networks of America.  The CINA Protocol Engine. Accessed atwww.cina-us.com/The-CINA-Protocol-Engine.htmon 16 June 2009.
 
Mold JW, Peterson KA.  Primary care practice-based research networks: working at the interface between research and quality improvement. Ann Fam Med. 2005; 3:Suppl 1S12-20. PubMed
 

Figures

Tables

References

Tunis SR, Stryer DB, Clancy CM.  Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003; 290:1624-32. PubMed
CrossRef
 
Rubin DB.  Estimating causal effects from large data sets using propensity scores. Ann Intern Med. 1997; 127:757-63. PubMed
 
Rubin DB.  Bias reduction using Mahalanobis-metric matching. Rubin DB Matched Sampling for Causal Effects. London: Cambridge Univ Pr; 2006.
 
Schneeweiss S, Avorn J.  A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. 2005; 58:323-37. PubMed
 
The Health Industry Forum.  “Road Testing” Electronic Health Records for Effectiveness Research. Conference Report. Accessed athttp://healthforum.brandeis.edu/meetings/materials/2008-4-June/Conference%20Report%20HIF%20EMR%20Event%206-4-08%20Final.pdf. on 4 June 2008.
 
Clinical Integration Networks of America.  The CINA Protocol Engine. Accessed atwww.cina-us.com/The-CINA-Protocol-Engine.htmon 16 June 2009.
 
Mold JW, Peterson KA.  Primary care practice-based research networks: working at the interface between research and quality improvement. Ann Fam Med. 2005; 3:Suppl 1S12-20. PubMed
 

Letters

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Comments

Submit a Comment
Submit a Comment

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

Toolkit

Want to Subscribe?

Learn more about subscription options

Advertisement
Related Articles
Journal Club
Topic Collections
PubMed Articles
Health services and policy research in hepatology. Curr Opin Gastroenterol Published online Mar 21, 2014.;

Want to Subscribe?

Learn more about subscription options

Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.
(Required)
(Required)