Scott P. Turner, BS, 1 Sean T. Pompea, MFA, 1 Kelly L. Williams, MBA, MSN, RN, 3 David A. Kraemer, Jr., BS, 1Evan T. Sholle, MS, 1 Cindy Chen, MA, 1 Curtis L. Cole, MD, 1 , 2 Rainu Kaushal, MD, MPH, 3 andThomas R. Campion, Jr., PhD 1, 3 , 4
1 Information Technologies & Services Department, Weill Cornell Medicine, New York, NY;
2 Department of Medicine, Weill Cornell Medicine, New York, NY;
3 Department of Healthcare Policy & Research, Weill Cornell Medicine, New York, NY;
4 Department of Pediatrics, Weill Cornell Medicine, New York, NY;
The NIH All of Us Research Program, a national effort to collect biospecimens and health data for over one million participants from across the United States, requires participating healthcare provider organizations (HPOs) to use informatics tools maintained by the NIH to manage participant consent, biospecimen processing, physical measurements, and other workflows. HPOs also maintain distinct workflows for handling overlapping tasks within their individual aegis, which do not necessarily achieve seamless interoperability with NIH-maintained cloud-based systems. At our HPO, we implemented informatics to address gaps in enrollment workflows and hardware, clinical workflow integration, patient engagement, laboratory support, and study team reporting. In this case report we detail our approach to inform efforts at other institutions for the NIH All of Us Research Program and other studies.
To enable precision medicine, the National Institutes of Health (NIH) All of Us (AoU) Research Program aims to collect data and biospecimens from one million or more people in the United States. Toward this goal, NIH has funded nearly 50 healthcare provider organizations (HPOs)—including regional medical centers (RMCs), federally qualified health centers (FQHCs), and Veterans Affairs Medical Centers (VAMCs)—to engage patients for AoU participation (1). Of RMCs, the New York City Precision Medicine Consortium consists of Columbia University Medical Center, NYC Health + Hospitals/Harlem, NewYork-Presbyterian, and Weill Cornell Medicine (WCM). Over a span of five years, the New York City Precision Medicine Consortium has a goal of enrolling 90,000 participants, about half of which WCM aims to enroll. Similar consortium-based efforts exist across the country.
To meet enrollment goals, HPOs must obtain from all participants consent to participate, survey responses, physical measurements, biospecimens (i.e., blood), and electronic health record (EHR) data. To complete these tasks, AoU requires participants and study team members to use informatics tools hosted in secure NIH-controlled cloud environments. For consent and surveys, participants use the publicly accessible Participant Portal website. To document physical measurements and biospecimens, HPO study team members use the HealthPro web application, which also provides individual- and HPO-level study participation lookup and export. For EHR data, HPO informatics staff transform local clinical data to the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM) and transmit payloads to a secure data resource center.
Although AoU has provided informatics tools to enable enrollment at HPOs, gaps exist between AoU and HPO systems and workflows, and optimal approaches for supporting HPO study teams with informatics are unknown. The objective of this case report is to illustrate how our HPO implemented informatics to address gaps in enrollment workflows and hardware, clinical workflow integration, patient engagement, laboratory support, and study team reporting to support AoU enrollment activities.Go to:
Weill Cornell Medicine is an academic medical center located on the Upper East Side of Manhattan in New York City. Over 1400 WCM physicians conduct approximately two million visits a year as part of the Weill Cornell Physician Organization. Outpatient care and professional billing for the physician organization is documented in the EpicCare Ambulatory EHR system, and the WCM Information Technologies & Services group uses multiple systems to manage research IT needs, including a clinical trials management system (CTMS) for tracking enrollment and participation in clinical trials (2), an electronic data capture system, REDCap (3), for storing and managing research data, and additional informatics solutions for extracting and transforming EHR data for analytic use (4), including an instance of the OMOP CDM comprising data for all patients seen at the institution.
Figure 1 shows the flow of information across systems to support AoU at our HPO. Whereas the top swim lane contains three systems overseen by NIH for AoU, the bottom-three swim lanes contain systems overseen by our HPO for general clinical and research operations, some of which we configured specifically to support AoU. While NIH focused on the three AoU systems—Participant Portal, HealthPro, and Data Resource Center (DRC) Raw Data Repository (RDR)—our HPO expended effort across the three systems and all other systems to support AoU. Critically, integration of AoU and HPO workflows required association of unique AoU participant and local HPO patient identifiers. To support AoU at our HPO, we addressed gaps including enrollment systems and hardware, clinical workflow integration, patient engagement, laboratory support, and study team reporting.
Enrollment workflows and hardware
To support AoU enrollment, we augmented NIH Participant Portal and HealthPro applications with three novel, AoU-specific local HPO workflows using REDCap—participant identity management, participant relationship management, and compensation management—and also provisioned hardware and support services.
Participant identity management
Managing the association of each participant’s unique identifier from AoU systems with a unique identifier from HPO systems was necessary for AoU-required EHR data extraction as well as local AoU-supporting HPO workflows. As shown in Figure 1, after a participant provided consent in the Participant Portal and paired with the consortium, data about the participant, including participant-entered demographics and a unique system-generated Precision Medicine Initiative Identifier (PMI ID), became automatically available in HealthPro for access by HPO study team members. Although the PMI ID of each participant was unique, it lacked a corresponding identifier in the HPO, such as a medical record number (MRN).
To associate each PMI ID with an HPO MRN, a study team member copied-and-pasted the PMI ID and demographics (i.e., first name, last name, date of birth) of a participant from HealthPro to a record in a REDCap project specific to AoU enrollment. Additionally, a study team member accessed the HPO EHR to determine whether the participant was an HPO patient. If the participant existed in the EHR, the study team member copied-and-pasted the patient’s MRN from the EHR to a corresponding field in a REDCap record. If the participant did not exist in the EHR, the study team member registered the participant as a patient to create an MRN, which the study team member then copied-and-pasted to REDCap.
Based on entry of the MRN, REDCap’s dynamic data pull (DDP) plugin retrieved demographics from the EHR for adjudication by a study team member and, pending confirmation, saving in designated REDCap fields (5). Use of DDP eliminated double data entry and risk of error for transcribing demographics from the EHR to REDCap. As shown in Figure 2, REDCap presented individual participant demographics from HealthPro and the EHR so study team members could confirm values matched across the two systems in order to verify identity.
Participant relationship management
To recruit patients at our HPO to participate in AoU, study team members frequently screened upcoming clinical appointment schedules in the EHR and contacted patients in advance of office visits. All contacts occurred with approval from patients’ physicians. To manage relationships with potential participants, study team members logged contacts, such as phone calls and in-person conversations, in a repeatable REDCap form (Figure 3) corresponding to an existing participant identity management record.
If a patient indicated he or she did not wish to participate in AoU, a study team member recorded the preference, which subsequent reports presented to inform the study team. Similar to the pre-consent recruitment contact form, study team members used a post-consent contact form for documenting interactions with participants after they had consented to participate but had not yet completed all AoU requirements (i.e., survey, physical measurements, biospecimens).
Upon completion of general consent, EHR consent, physical measurement, survey, and biospecimen requirements, AoU participants became eligible to receive a $25 gift card as compensation. To track compensation, study team members used REDCap to document who provided the gift card and on what date for each participant. To confirm receipt of the gift card, each participant signed a field in a REDCap survey form using an iPad screen and a stylus or finger. To access the REDCap survey form specific to a participant, a study team member scanned a unique survey QR code configured within REDCap.
Hardware and support services
To enable participants and study team members to access Participant Portal, HealthPro, REDCap, and the EHR at our HPO, we deployed new hardware and reused existing institutional infrastructure. For participants to complete consent and survey tasks using the Participant Portal and compensation verification using REDCap during WCM clinical visits, we purchased eight iPads, which had restricted access settings configured by ITS security to encourage usage only for AoU. For study team members to access HealthPro and REDCap, we purchased eleven laptops. All iPads and laptops were managed by ITS Security and had permission to use the HPO Wi-Fi network. After discovering a gap in Wi-Fi connectivity in clinical space used for AoU enrollment, NYP IT staff expanded wireless network capacity. All laptops required mapping to institutionally-managed printers as well as Zebra label printers provisioned by NIH for biospecimen label creation. Using an institutionally-hosted wiki, we documented standard operating procedures (SOPs) describing device and system provisioning, de-provisioning, and troubleshooting as well as workflows involving one or more systems, such as participant identity management using HealthPro, REDCap, and the EHR. We instructed study team members and ITS service desk personnel to use the wiki materials for onboarding and troubleshooting.
Clinical workflow integration
To inform clinician awareness and billing compliance for AoU, we established automated integration between REDCap and the CTMS, which served as the gateway to the EHR for research enrollment records at our HPO. A novel middleware sent AoU enrollment and withdrawal data associated with HPO MRNs from REDCap to the CTMS as study team members updated records. From the CTMS, enrollment records flowed to the EHR where they appeared in the Research Studies tab, the standard location for clinicians to determine whether a patient was participating in a study. Creation of research studies records also activated billing compliance workflows in the financial module of the EHR, which was standard at the institution for all patients participating in studies.
To engage HPO patients and the general public for AoU enrollment at our HPO, we implemented online self-scheduling for potential participants in clinic space allocated by the WCM Clinical and Translational Science Center exclusively for AoU. In the EHR patient portal, a message appeared upon logon to existing patients about AoU with a link to learn more and schedule an appointment. Via an open public website linked to the EHR, non-HPO patients were able to reserve a time for AoU participation (Figure 4).
Self-service features enabled potential participants to schedule an appointment for collection of physical measurements and biospecimens as well as the opportunity to ask questions of study team members while completing consents and surveys.
Although laboratory staff used HealthPro for documenting of biospecimen collection and processing, HealthPro lacked fields for staff to record specimen-specific elements, such as lab delivery time and discarding, for supporting AoU requirements including specimen finalization within four hours. To address this gap, we configured an existing laboratory information management system (LIMS) to capture AoU-specific data elements, including PMI ID and a biobank order ID. Capture of AoU-specific ID values supported subsequent reporting activities combining data from the LIMS and HealthPro.
Study team reporting
To support study team activities, we generated multiple reports that combined data from HealthPro and REDCap. While HealthPro contained NIH-defined data elements, such as consent and study participation status (e.g., survey completion, biospecimen collection, physical measurement completion) and REDCap contained local HPO-defined data elements, such as enrollment locations and participant contact information, both systems stored each participant’s PMI ID as a common key. To reduce double data entry and risk of error, we developed an approach whereby a study team member downloaded a CSV file from HealthPro containing bulk participant data and automatically integrated the data with REDCap on the basis of common PMI ID values. Of note, the CSV contained data from NYC Consortium partners (i.e., Columbia, Harlem) not pertinent to WCM-specific HPO activities. By combining national HealthPro data with local HPO REDCap data, we enabled comprehensive understanding of AoU participation at our HPO not possible with use of HealthPro or REDCap alone, including participant compensation and participant contact as well as enrollment by clinic site, method (i.e., in-person, phone), and race and ethnicity. Accessible securely via the web-based Microsoft SQL Server Reporting Services, reports enabled HPO executives to understand local performance as well as study team members to support enrollment workflows. As shown in Figure 5, the reporting site enabled visualization of trends derived from HealthPro and REDCap data.
Using NIH systems combined with existing and novel local informatics infrastructure, we augmented five areas-enrollment workflows and hardware, clinical workflow integration, patient engagement, laboratory support, and study team reporting–to support the NIH All of Us Research Program at our HPO. To date, our approach to enrollment workflows and hardware has supported more than 2,300 participants at our HPO. Clinical workflow integration has enabled display of AoU enrollment status for all participants in the EHR, which has supported at least one reported adverse event potentially caused by AoU study participation. For patient engagement, potential participants have used online self-scheduling to arrange 258 appointments, including cancellations and no-shows, yielding 12 enrollments. Laboratory support has enabled processing of more than 2,400 biospecimens in compliance with quality control standards. Through integration of HealthPro and REDCap data, study team members have accessed reports about 100 times per month.
To support AoU enrollment at our HPO, we addressed five gaps—enrollment workflows and hardware, clinical workflow integration, patient engagement, laboratory support, and study team reporting—using existing and novel local HPO informatics infrastructure. To date, more than 2,300 participants have enrolled at our HPO assisted by our informatics approach. To the best of our knowledge, the literature does not describe how HPOs can support AoU with local informatics efforts, and this case report can inform efforts at other institutions in integrating clinical and research workflows.
Our methodology scaled well to support the needs of the institution and may be of use to other institutions participating in similar efforts. We expect that the efforts detailed herein will scale appropriately as WCM continues towards its target of enrolling 40,000 patients to the protocol, and other institutions participating in AoU or other similar large-scale registry studies may find that the workflow described herein addresses challenges unique to the nature of the study. Of particular note is the large scale of the work required: a number of different local IT groups covering different specialties provided expertise, and the complexity of integrating NIH systems into institutional workflows suggests that support from NIH and similar funders is required to ensure smooth procession of these workflows.
Multiple limitations of the methodology emerged in considering its implementation and impact. It is difficult to overemphasize the effort required to maintain a robust identity management approach, first in collecting data from patients and again in verifying data prior to the submission of EHR data. Despite implementing multiple heuristics and computational techniques (e.g. Jaro-Winkler algorithm), we still had to conduct manual verification of about 50 of 1000 participants for our initial EHR data submission. Use of self-scheduling was substantially lower than expected, suggesting that patients were either unwilling to participate without clinician encouragement, publicity efforts did not succeed in making them aware of the potential to self-schedule visits to enroll in the protocol, or other factors affected usage.
The primary implications of the approach are its scalability and its generalizability. The system has performed admirably to date, as we have added more users, reports, and hardware to support the expansion of the study. Additionally, components we have developed for AoU, especially for clinical systems integration with REDCap, may be useful for other studies. Future work will include implementation of dedicated participant relationship management tools, including the NIH-sponsored Mission Control and HPO-specific Salesforce, to support recruitment and retention efforts. Additionally, future work will evaluate the return on investment of the approach as well as factors affecting participation status of individuals.
In building a robust, scalable, and modular IT infrastructure to support the NIH All of Us Research Program, we developed unique solutions that address the challenges of integrating external systems with local workflows to facilitate patient enrollment throughout the participant lifecycle. It is our hope that the lessons learned in implementing this system are of use to other institutions considering embarking on similar endeavors.
Research reported in this publication was supported by the National Institutes of Health (NIH) Office of the Director (UG3 OD023163-01) and NIH National Center for Advancing Translational Sciences (UL1TR000457). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
2. Campion TR, Blau VL, Brown SW, Izcovich D, Cole CL. Implementing a clinical research management system: one institution’s successful approach following previous failures. AMIA Jt Summits Transl Sci Proc. 2014 Apr 7;2014:12–17. [PMC free article] [PubMed] [Google Scholar]
3. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support.J Biomed Inform. 2009 Apr;42(2):377–381. [PMC free article] [PubMed] [Google Scholar]
4. Sholle E, Kabariti J, Johnson S, Leonard J, Pathak J, Varughese V. (In press.); 2017. Secondary Use of Patients’ Electronic Records (SUPER): An Approach for Meeting Specific Data Needs of Clinical and Translational Researchers. AMIA Annu Symp Proc. pp. 1581–1588. [PMC free article] [PubMed] [Google Scholar]5. Campion TR, Sholle ET, Davila MA. Generalizable middleware to support use of redcap dynamic data pull for integrating clinical and research data. AMIA Jt Summits Transl Sci Proc. 2017 Jul 26;2017:76–81.[PMC free article] [PubMed] [Google Scholar]