Therapeutic Innovation & Regulatory Science https://doi.org/10.1007/s43441-020-00213-4
ORIGINAL RESEARCH
Evolving Clinical Data Strategies and Tactics in Response to Digital Transformation Beth Harper, BSOT, MBA1 · Michael Wilkinson, MPH2 · Raj Indupuri, MBA3 · Sheila Rocchio, MBA3 · Ken Getz, MBA2 Received: 30 April 2020 / Accepted: 24 August 2020 © The Drug Information Association, Inc 2020
Abstract Background Contending with a continuously expanding volume and variety of clinical data poses challenges and opportunities for the industry and clinical data management organizations. Methods Tufts CSDD conducted an online survey aimed at further quantifying and understanding the magnitude and impact that expanded data volume, sources and diversity are having on clinical trials. The survey was distributed between October and December 2019. Responses from a total of 149 individuals were included in the final analysis. Results The survey found that companies use or pilot from one to six different data sources with the majority of respondents using or piloting 3–4 different sources of data in their clinical trials. The results showed that average times to database lock have increased an average 5 days compared to a 2017 study, possibly as a result of managing an even larger number of data sources. Finally, three key mitigation strategies surfaced as techniques respondents used to tackle expanding data volume, sources, and diversity: the creation of a formalized data strategy, investment in new analytics tools and more sophisticated data technology infrastructures, and the development of new data science disciplines. Conclusion Without further investments into infrastructure and developments of additional mitigation techniques in this area, database lock cycle times are likely to continue to increase as more and more data supporting a clinical trial are coming from nontraditional, CRF sources. Further research must be done into organizations who are handling these challenges appropriately. Keywords Clinical data · Clinical technology · Clinical data management · Data strategy · Data governance · Data science
Background and Introduction Volume, velocity, and variety, often referred to as the three V’s of big data, are well known concepts in the world of consumer data management [1]. From the millions of Facebook posts, Instagram images, or tweets that social media companies manage to the wide variety of apps and connected devices that generate millions of data points, we rarely give any thought to how these organizations ingest, process, file, and retrieve the incomprehensible amounts of data that are generated at faster rates each day. * Michael Wilkinson Michael.Wilkinson@tufts.edu 1
Clinical Performance Partners, Atlanta, GA, USA
2
Tufts CSDD, 75 Kneeland Street‑ Floor 11, Boston, MA, USA
3
eClinical Solutions, Mansfield, MA, USA
While the volume of clinical trial data may not compete with the scale of volume that Facebook manages, it has nonetheless grown 183% over the last decade [2]. Across nine different types of e-clinical trial data, the majority of companies surveyed in a 2017 study anticipated that their use of these data sources would double or triple within a 3-year period [3]. In fact, all of the nine data sources were predicted to be used by more than 50% of those surveyed in 3 years’ time. That study also found that average cycle time durations needed to achieve database lock were longer and more variable than those observed 10 years prior. Additionally, a recent white paper published by the Society for Clinical Data Management expanded the three V’s to five V’s, adding veracity and value into the mix [4]. This paper advocated that major shifts were needed to re-think clinical data management approaches and develop fit-forpurpose strategies. The paper posed these thought-provoking questions: Where and how, both logically and physically, should the increasingly variable clinical data sources be
13
Vol.:(0123456789)
Therapeutic Innovation & Regulatory Science
orchestrated to shorten the gap between data acquisition to data consumption? Across industries, the definition of formal Data Strategies has become a strategic imperative defining how an organization deals with its most important asset, its data. Life Sciences organizations have long taken conservative approaches to data strategy focusing on ways to maximize control and minimize risk [5]. As data sources and assets grow in life sciences and research, new opportunities are created across the digital health continuum that may shift the data strategy balance from a control focus to one that is more flexible and provides a greater ability to create new customer and end user experiences. The purpose of the Tufts eClinical Solutions Data Strategies and Transformation study was to further quantify and understand the magnitude and impact that expanded data volume, sources and diversity are having on clinical trials. Specifically, the study was designed to inform how life sciences organizations are managing the growing volume and variety of clinical research data to support drug development activities and prepare for new capabilities like artificial intelligence. In addition, this study also looked to identify and share best practices with regard to:
survey comprised 29 questions divided into six sections. These sections included data sources and cycle times, data management, data analytics, data strategy, data sciences and demographics questions. As an incentive to stimulate response, participants were offered a summary report of the study findings and the opportunity to enter a drawing for one of twenty $25 Amazon gift cards. The survey was distributed to both proprietary and commercially available contact lists. It was fielded over a 12-week period from October 4 to December 30, 2019. Eligible study participants were individuals with either direct responsibility for clinical data management activity or a frequent user of clinical systems and employed at a company’s organization (including biopharmaceutical companies), a contract research organization (CRO), or consultant organization that has initiated at least one clinical trial each year. Logic programmed in the study survey was designed to accept responses only from those who met these eligibility criteria. Participation in the survey was terminated if respondents indicated that they had no responsibility for data management, were an occasional user of, or never used clinical systems.
• The types of ways external/non-CRF data are being
Survey Themes and Data Verification
incorporated and leveraged in clinical trials • How external data sources are influencing data management best practices • The data platforms, analytics tools, capabilities and competencies that companies are putting into place to manage the growing number of external/non-CRF data sources.
Methods Study Design and Participants The Tufts CSDD, in collaboration with and funded by eClinical Solutions LLC, designed a survey to investigate data transformation and management strategies in clinical research. Respondents targeted included senior level professionals working in the area of data management, data sciences, biostatistics, information technology, and clinical research operations. The online survey was designed and distributed using Qualtrics survey software. The final
Figure 1. Data Management Workflow or Process Steps used in Survey.
13
A data lifecycle framework (Fig. 1) were introduced at the beginning of the survey to orient the respondents to key aspects of the data management process that were evaluated throughout the survey. A standardized set of data sources was also incorporated along with definitions for each source as provided by Transcelerate [6]. The data sources included non-CRF data, direct data capture, devices and apps, electronic health records, omics with biomarker data, medical images and an option to specify other types of data.
Data Analysis Eligible responses were first cleaned in Microsoft Excel and then imported in to the statistical analysis program SAS. Analyses performed and reported on included descriptive statistics, frequency comparisons, coefficients of variation, analysis of mean response values, subgroup stratification, and significance testing. Subgroups included in the final
Therapeutic Innovation & Regulatory Science
analysis were company size (small, mid-size and large), the current experience with a data strategy (formally implemented, currently examining or no plans to implement a data strategy), the average number of data sources routinely used in their clinical trials (1–2, 3–4 or 5–6 sources), and those using a data lake or hub. Significance tests for variables including at least one categorical variable and one numerical variable were conducted using t-tests and multiple regression. Chi-square tests were performed to evaluate the null hypothesis between categorical variables. Significance was defined as p < 0.05.
Results The survey yielded 239 responses that were completed in full. Ten respondents indicated that their organization did not initiate clinical trials, 64 respondents had inadequate clinical systems experience, 15 were from duplicate companies, and one did not explicitly provide consent. This resulted in a final sample of 149 valid responses for analysis. A broad cross section of functional groups were represented by respondents including Clinical Operations (40%) Data Management (23.5%), Biostatistics (7%), Data Sciences (5%), Research and Development Information Technology (5%). Other functional groups (19.5%) comprised Clinical Development, Quality Management, Regulatory and Executive Management functions. Most of the respondents indicated they were one of several individuals responsible for clinical data (48%), with 33% of the respondents indicating they were the primary individual responsible for clinical data. The remaining nineteen percent were frequent users of clinical data systems.
Several subgroups were analyzed including company size, data strategy experience, average number of data sources used, and use of a data lake/hub to integrate and manage their data. The number of clinical trials initiated annually was used as a proxy for company size. Those companies that initiated between 1 and 5 trials per year were considered small (N = 47) and averaged 2.9 trials each year; medium companies initiated between 6 and 50 trials per year (N = 44) and averaged 23.8 trials; large companies initiated between 51 and 350 trials per year (N = 25) and averaged 112.4 trials. These subgroups were based on literature and prior Tufts CSDD surveys experience.
Data Sources With regard to the type of data being captured in clinical trials all companies reported using all types of data sources (Fig. 2). Non-CRF data and direct data capture were found to be used by the majority of respondents. About two-thirds of respondents reported using devices and apps and medical images with less than half of the respondents using electronic health records (EHR) or omics data. Large companies were found to be using five or all six of these data types almost five times more frequently than medium companies (79.2% vs 16.7%) and about 75% more often than small companies (79.2% vs 4.7%). The survey found that companies use or pilot from one to six different data sources with the majority of respondents using or piloting 3–4 different sources of data in their clinical trials (Fig. 3).
Figure 2. Clinical Data Sources Used or Piloted in Evaluations.
13
Therapeutic Innovation & Regulatory Science
Figure 3. Number of Data Sources used or Piloted in Clinical Trials
Increasing Timelines Associated with Larger Number of Data Sources As previously noted, the 2017 study found that more diverse clinical data sources were associated with longer and more variable database lock cycle times. The results of this study were consistent with that finding and showed that the trend has continued with an average 5-day increase in time to database lock, possibly as a result of managing an even larger number of data sources (Fig. 4). Large pharmaceutical companies are experiencing the greatest delays, showing a 32% increase in time from Last Patient Last Visit (LPLV) to Database Lock (DBL) since the 2017 study. From a cycle time perspective, those using 3–4 sources have some advantages compared to the other groups (Fig. 5).
Figure 4. Database Lock Cycle Times—Overall and by Company Size. *Study examining current and evolving EDC and clinical data usage practices across the drug development enterprise. Survey con-
13
Unsurprisingly, organizations who are managing 5–6 data sources face longer times to achieve database lock by approximately 9 days compared to the average overall cycle time. With regard to which data source were contributing the most to the database lock delays, over half (51%) of the respondents indicated that non-CRF data were the culprit by a wide margin.
Challenges Associated with Data Access While database lock cycle times are a downstream indicator of the impact of managing more and more diverse data, moving upstream, this study found that respondents face challenges in all aspects of data management. While analyzing
tained 22 questions, was open for 10 weeks, and garnered 257 completed responses from mostly sponsor organizations (75%)
Therapeutic Innovation & Regulatory Science
Figure 5. Database Lock Cycle Times—Number of Data Sources.
and consuming the data were rated as least challenging of all activities, still over fifty percent of respondents still rated this as somewhat or extremely difficult. All other activities were rated as challenging by over 60% of respondents with data curation seen as the most difficult aspect. Not all data are equal when it comes to performing the required data management functions from importation, integration, organization, transformation, curation, and analysis. Perceptions of the ease of access and integration varied widely across data sources with direct data capture being rated as the easiest to access and integrate while omics data and EHR data being the most difficult.
Labor‑Intensive Data Lifecycle Areas Beyond accessing and integrating the data, respondents found that nearly all aspects of the data lifecycle were either extremely or somewhat time-consuming in the era of disparate and varied sources of clinical data. The most timeconsuming and labor-intensive aspect of the process related to initiating relationships with the myriad data providers. Data analysis were deemed to be the least time and resource
Table 1. Systems and Tools Being Used to Integrate and Organize Data.
Percent Report Using Non-CRF Data (N = 116) Direct Data Capture (N = 108) Devices and Apps (N = 99) Medical Images (N = 95) Electronic Health Records (N = 79) Omics Data (N = 78)
intensive of all the activities; however, some 62.9% of the respondents still found this activity to be challenging. There were not significant differences noted across the continuum of other data management activities with between 74 to 83% of the respondents ranking all of these activities as being somewhat or extremely time-consuming and labor-intensive.
Systems and Tools to Integrate and Organize Data When asked about systems and tools to integrate and organize data both within and across trials, respondents are moving to a centralized infrastructure using a clinical data hub or repository (49.1%) followed using SAS for integration (46.6%) with EDC being used just 20.7% of the time (Table 1). These results revealed that regardless of data source, clinical data hubs or repositories are being used more often compared to EDC, SAS-based infrastructure, or other systems. The one exception was that EDC was used most often for integrating and organizing direct data capture. It should be noted that respondents could select all of the tools that applied, signifying that no single tool or system
EDC (%)
SAS-Based Infrastructure (%)
Clinical Data Hub/Repository (%)
Other (%)
20.7 50.9 24.2 8.4 20.3 3.9
46.6 36.1 42.4 27.4 31.7 33.3
49.1 33.3 54.5 53.7 44.3 41.0
8.6 5.6 4.0 23.2 16.5 32.1
13
is universally being used to support this aspect of the data management process. In terms of consuming, reviewing, and analyzing data, SAS was used most of often for all data sources except medical images, where other types of tools were used. Again, respondents were asked to select all the systems that applied. Finally with regard to data conversion and analysis activities, SAS is still the primary tool used by far, with some 75% of respondents indicating that they use SAS as the primary tool to convert raw data to analysis-ready data, as well as for exploring and analyzing data.
Addressing the Challenges Regardless of the company’s size, all respondents agreed that it was important to address these challenges. Potential solutions were rated as important by the majority of respondents. Finding support for dealing with more external data sources was rated as somewhat or very important by 90.1% of the respondents with streamlining processes to improve cycle times, supporting a patient-centered data acquisition strategy and better opportunities to use real-world data all being ranked as important solutions by over eighty percent of the respondents. Recognizing the importance of solving these challenges, the study found that companies are addressing these in several areas and some solutions are having a greater impact than others. Formalizing a data strategy, governance processes, tools and technology infrastructure for data management and developing data science competencies are the key areas where some noteworthy differences were identified.
Figure 6. Respondent by Use of Formalized Data Strategies. *N = 112.
13
Therapeutic Innovation & Regulatory Science
Use and Impact of a Data Strategy The survey found that one-third of the respondents have implemented a formal data strategy that defines how different data sources will be collected, organized, and analyzed for use in clinical development (Fig. 6). Nearly half of the respondents were either examining or planning a data strategy initiative in the coming months; these were grouped into the “currently examining” group for further subgroup analyses. A smaller segment of the respondents (13.4%) had no plans to develop a data strategy and 6.4% were not sure. This group was removed from further subgroup analyses. Perhaps not surprising, large companies are more likely to have implemented a formal data strategy as are those organizations using a greater number of data sources. Nearly 53% of large companies and 53% of organizations using 5–6 data sources have implemented a formal data strategy compared to the average of 33% across all companies. The need for more sophisticated tools and processes or a formalized data strategy may simply not be strong enough for those organizations only using one or two external data sources. This theme is reinforced throughout the remaining data strategy subgroup analysis graphs. The most dramatic cycle time differences were noted for those who have versus those who have not implemented a formal strategy. Those who have implemented the data strategy achieve database lock times 10 days faster than those who have not (Fig. 7). In this case, the organizations examining, researching, or who didn’t have a strategy were grouped together under the umbrella of not having a strategy.
Therapeutic Innovation & Regulatory Science
Figure 7. Database Lock Cycle Times—Data Strategy.
Data Strategy, Analytics and Artificial Intelligence Data strategy and analytical capabilities appear to go handin-hand for life sciences companies surveyed. Companies who have implemented a formal data strategy perceive their analytical capabilities and competencies to be more fully developed and mature. Over 50% of the companies who have implemented a formal data strategy rate their capabilities as more mature across all types of analytical capabilities, with augmented data discovery as the sole exception. In comparison, fifty percent of those without a strategy believe their descriptive analytics (where descriptive is defined analytics that show what has happened) capabilities to be fully developed with a significant drop off across all other analytical capabilities. Organizations who have implemented a formal data strategy are significantly more mature in terms of researching or implementing Artificial Intelligence (AI) including Machine Learning and Deep Learning capabilities. While these are still emerging technologies, with less than 20% of companies reporting that they have fully or somewhat mature capabilities in these areas, there is a significant difference seen between those with and without a data strategy. The organizations who have implemented a data strategy are also much more likely to be establishing a new data science function or specific data sciences roles and competencies. Nearly half (47.2%) of the organizations who have implemented a data strategy have also created a data science function whereas organizations with no plans to develop a data strategy also are not focused on creating specific data science competencies. While organizations who had implemented a formal data strategy felt they were more mature in a range of data science and AI capabilities, they also believe it is more important to solve a wide range of data management challenges.
Companies who have implemented a formal data strategy see greater importance in solving these challenges compared to those who have no data strategy plans. Regardless of whether the companies felt it was important to solve certain data management challenges, more than half of all respondents perceive data operations as somewhat or extremely difficult. Companies who have implemented a data strategy, however, perceive the majority of these activities as less difficult than their counterparts. Having developed a data strategy, these organizations may have overcome some of the challenges.
Opportunities for Automation In terms of whether automation of the various data pipeline, management and analytics activities would have an impact, the results were more varied. The majority of respondents felt that almost all activities, except for setting up relationships with data providers had the potential to be automated. With regard to the suitability of automation for some of the key data management activities, there was a significant difference between those who have and have not implemented a formal data strategy. Companies who have implemented a strategy see greater value in automating data review, cleaning, and analysis compared to those without a strategy.
Data Strategy Benefits and Obstacles Regardless of where organizations are on their evolution of implementing a formalized data strategy, the majority of all respondents believed there were a range of benefits to creating the strategy. These benefits to be realized from the strategy included: • Ability to fully leverage clinical data assets
13
• Increasing visibility of data to key stakeholders for faster
decision-making • Decreasing cycle time through increased collaboration and shared analytics • Saving costs by resource optimization and reduced rework • Faster acquisition and benefits from new real-world data sources that add to development
All of the potential benefits were rated as somewhat or very beneficial by over 85% of the respondents. When asked about the obstacles to creating a data strategy, perceived barriers included finding the time and resources, the costs of creating the strategy, selecting the right technology, not having executive sponsorship and others. Companies who have implemented a formal data strategy perceived the barriers to creating a formal strategy as less difficult and burdensome. Having implemented the strategy they may have already overcome what their counterparts perceive as barriers. For those not yet developing a formal strategy, the cost of replacing their existing technology was seen as the biggest barrier.
Data Governance Approaches Formalized data governance were seen as a potential mitigation strategy however the survey found wide variability with regard to the existence or sophistication level of data governance. Large companies were more likely to have a formal and strong governance process in place for managing data flow, security and accessibility, while small companies were least likely to have a formal process in place. In terms of which functional group owns or manages the governance process, the survey found that data governance activities are managed via a hybrid of clinical operations and data management functional groups for large and medium companies. The data management function is more likely to lead the data governance activities at small companies.
Data Science Disciplines With all the opportunities to leverage technology as a means of mitigating some of the challenges associated with a greater volume and diversity of data, this may necessitate new capabilities and competencies in the area of data science. The results indicate that large and medium size sponsors are more likely to be piloting or implementing a new data sciences competency than smaller sponsors. Further, most small sponsors are more likely to have no plans for creating a data science competency. As noted above, organizations who have implemented a formal data strategy are also more likely to establish a new data science function or specific data science roles.
13
Therapeutic Innovation & Regulatory Science
Regardless of sponsor size, the results suggest that most sponsors are incorporating data science capabilities into existing data management function although large sponsors are more likely to have decentralized data science roles across the organization.
Organizational Maturity While sponsors are adopting a variety of approaches to mitigate the challenges associated with a growing volume and diversity of data, in general, they still perceive themselves as relatively immature in many of these areas. Factors associated with perceptions of greater maturity were those who have implemented formalized data strategies, are using data hub/lakes, those with a more formal governance in place, and those who have a data science function/competency in place. With regard to analytics abilities, organizations with a formal data governance perceived themselves to be more mature across the entire data lifecycle enabling efficient decision-making. The same patterns were seen with regard to those organizations who have a data science function or established data competencies in place. This same pattern was seen across all of the analytics critical capabilities assessed in the survey. From having analytics dashboards to interactive visual exploration capabilities to augmented data discovery, and the ability to publish, share and collaborate on analytic content, organizations with a strong data governance in place and those with a data sciences function rated themselves as being more mature and capable in all these areas compared to their counterparts. Lastly, the same pattern occurred with perceptions of data science and AI maturity.
Discussion Data volume and diversity continue to grow dramatically with the unintended consequence of disrupting clinical trial performance and efficiency [7]. This study was conducted to inform understanding of how companies are responding to digital disruption.
External Data are Growing The results confirm findings and trends reported in an earlier Tufts CSDD study and show that companies and CROs are managing even greater data volume and diversity at this time. One-third of the companies indicate that they are routinely using or piloting up to 5–6 data sources in their clinical trials with half of companies (49.7%) using 3–4 data sources. The vast majority of survey respondents (87%) are incorporating non-CRF data into their trials. Nearly
Therapeutic Innovation & Regulatory Science
two-thirds of the companies are using direct data capture, devices, apps, and medical images as other data sources, with approximately one-third using electronic health records (43.4%) and omics data (35.5%).
Data Strategy Matters and is Impacting Results and Capabilities Across all data management activities, companies report a variety of challenges. Study results show that companies and CROs have implemented four primary solutions: the establishment of formal organization-wide data strategies, implementation of data governance processes, investment in more sophisticated analytic tools and technologies, and the development of data science competencies. Significant subgroup differences were observed. Companies who have implemented a formal data strategy are more likely to have implemented a data hub/lake as part of their strategy—both of which are associated with shorter relative data management cycle times. Those organizations that have implemented a formal data strategy and a data lake/hub also rate their analytics abilities and critical capabilities to be more mature and fully developed. This group also finds that non-CRF data contributes to fewer database lock delays than those not using a data lake/hub. Companies and CROs with a formal data strategy find managing operational activities to be less difficult. And while there are many perceived barriers to implementing a formal data strategy, most respondents agreed that this results in a number of benefits. However, those most likely to have implemented a data strategy are mostly large companies using a wide variety of data sources, so it is possible that the benefits seen in this area are from organizations with greater resources and more concrete practices. Large companies are more likely to have a formal data governance structure in place than small or medium sized companies. These companies rate themselves much more mature across a whole host of data management and analytics competencies. For the most part, the data governance process appears to be a joint effort across clinical operations and data management functions. Whether it is the data strategy or governance process that leads to more fully developed capabilities is unclear. However, the patterns suggest some relationship across these factors.
Technology Infrastructure to Manage Data is Not Keeping Pace with the Data Many companies are still using somewhat outdated tools (Excel, data capture systems) to manage the data, although the use of clinical data hubs/lakes appear to be emerging as a more common way to combine different data sources. There was universal agreement with regard to the benefits
of data lakes/hubs, offering self-service analytics capabilities to end users, automating many of the data management activities, and others, but there is not yet widespread use of these and other more cutting-edge technologies. In fact, less than twenty percent of respondents felt they had developed capabilities in the area of Artificial Intelligence. As the tools and technologies evolve and become more sophisticated, so will the skills and competencies of those interacting with the modern infrastructure. Developing data science competencies has also been identified by others as a key to success in navigating and managing digital transformation within their organizations [8–10]. Companies that have built data science competency deem their analytical capabilities to be more fully developed and mature. Interestingly, companies with a formal data strategy are more likely to report that they are also developing data science competencies.
Conclusion and Future Research The results of this study reinforce findings from the 2017 Tufts CSDD study and suggest that without mitigation strategies, database lock cycle times, as a key performance metric, are likely to increase. Companies—particularly those managing a larger number of clinical data sources supporting a given clinical trial—are taking specific steps to mitigate these risks and improve their capabilities and competencies. Key practices include establishing a formal data strategy, implementing a data hub/lake, developing data science capabilities, and investing in more sophisticated data technology infrastructure. Opportunities for future research include the impact new tools and practices including expanded automation, advanced analytics and machine learning will have on helping life sciences companies manage and analyze these new data streams for increased knowledge along with datadriven decision-making to bring medicines faster to patients. Funding
The Tufts Center for the Study of Drug Development received a Grant from eClinical Solutions to perform this research.
Compliance with Ethical Standards Conflict of interest
No potential conflicts were declared.
References 1. Gewirtz, D. Volume, velocity, and variety: Understanding the three V’s of big data. ZD Net. https://www.zdnet.com/article/
13
Therapeutic Innovation & Regulatory Science
2. 3. 4.
5. 6.
volume-velocity-and-variety-understanding-the-three-vs-of-bigdata/. Accessed February 17, 2020. Getz, K. Anticipating the Impact of the Patient Engagement Movement on Clinical Operations. Presentation at CROWN conference. Slide #9. January 2020. Wilkinson M, Young R, Harper B, Machion G, Getz K. Baseline assessment of the evolving 2019 eClinical Landscape. Therap Innov Regul Sci. 2018;53(1):869–76. Society for Clinical Data Management White Paper. The Evolution of Clinical Data Management to Clinical Data Science (Part 2: The technology enablers). https://scdm.org/wp-content/uploa ds/2020/03/SCDM-Reflection-Paper-Evolution-to-Clinical-toData-Science-Part-2.pdf. Accessed March 25, 2020. DalleMule, L, Davenport, T. What’s Your Data Strategy. Harvard Business Review. https://hbr.org/2017/05/whats-your-data-strat egy. Accessed April 15, 2020. Kellar E, Bornstein SM, Caban A, et al. Optimizing the use of electronic data sources in clinical trials: the landscape, part 1. Therap Innov Regul Sci. 2016;50(6):682–96.
13
7. Limaye N, Limaye R. Why Pharma is rethinking its data integration strategy. Technology Networks. https: //www.techno logyn etwo rks.com/informatics/articles/why-pharma-is-rethinking-its-dataintegration-strategy-the-criticality-of-data-integration-307918. Accessed March 2, 2020 8. Insights on Digital Health Technology Survey. 2016. Validic eBook. https://pages.validic.com/rs/521-GHL511/images/Digit al_Health_Survey_Results_Pharma_2016.pdf Accessed March 8, 2020 9. Getz K. Surmounting e-clinical data volume and diversity. Applied Clinical Trials. https://www.appliedclinicaltrialsonli ne.com/surmo u ntin g -eclin i cal-data-volum e -and-diver s ity. Accessed March 8, 2020. 10. Matranga A. Developing a clear vision in the face of the technology deluge. Informa Connect. https://informaconnect.com/clear -vision-clinical-technology-deluge/. Accessed March 8, 2020.