1. INTRODUCTION
This short paper will discuss real-world evidence (RWE) and the increasing interest in its use for late-stage clinical research. It will provide a clear overview of the scientific value of RWE and the potential of patient-reported outcome measures (PROMs) as a source of real-world data (RWD), as well as possible pitfalls in collecting reliable data through innovative digital solutions. Some successful use cases of PROMs in secondary care will be discussed, and the potential value of ePROMs as a source of RWD will be debated.
In order to better understand this paper, some key concepts and definitions are briefly explained under section 1.1 Definitions.
1.1. DEFINITIONS
Patient-reported outcome measures (PROMs): PROMs are the tools used to measure patient-reported outcomes (PROs). PROs can be defined as any report of the status of a patient’s health condition or health behavior that comes directly from the patient. The key domains of PROs are health-related quality of life, symptoms and symptom burden (e.g., pain, fatigue), and health behaviors (e.g., smoking, exercise).1 Besides PROMs, patient-reported experience measurements (PREMs) are increasingly used to measure performance of healthcare and patient-care.2 This paper will focus on digital tools used to collect PROs, also referred to as ePROs.
Real-world evidence (RWE): The analysis of real-world data (RWD) regarding usefulness and effectiveness of drugs and/or lifestyle changes that are derived from multiple sources outside of clinical research settings, such as electronic health records, claims and billing data, product and disease registries, and data gathered through personal devices and health applications.3 This paper will primarily focus on data collected through personal devices and health applications.
eHealth: Healthcare services provided through electronic and internet services. It is both the digital provision of healthcare information as well as the use of digital technology to improve healthcare quality and processes. This paper will focus on the latter, discussing the possibilities and difficulties of using healthcare applications for patient treatment and education.
Primary, secondary, and tertiary healthcare: Healthcare services are divided into different categories. Primary care embodies day-to-day care given by healthcare providers. Typically, these providers act as a first contact and principal point of continuing care for patients within a healthcare system. Next to general practitioners (GP), home nursing, pharmacists, psychologists, physiotherapists, and many more are considered primary care providers. Secondary care refers to health services provided by specialized health professionals, usually after a referral from a primary care provider. Finally, tertiary care can be described as highly specialized care, often involving complex treatments or procedures for severe or life-threatening situations.
2. REAL-WORLD EVIDENCE AND DIGITAL HEALTHCARE
2.1. THE VALUE OF SECONDARY DATA FOR CLINICAL RESEARCH
In clinical research, randomized control trials (RCTs) are the golden standard for assuring efficacy and safety of a treatment.4 RCTs are used to prove superiority of a new treatment over an existing treatment or a placebo. Together with meta-analyses, they are considered to be one of the highest levels of scientific evidence. Performing a meta-analysis of several RCTs is the highest. Many trials take place in a highly controlled environment to guarantee patient safety in case of severe adverse events (AE).
Historically, the RCT setting has been created to minimize the risk of disturbing events. In the creation of these settings, limited attention was given to learning opportunities that can arise from secondary analysis of data gathered from everyday practice. Back then, all secondary data was hand-written, and its analysis would be very unpractical. This unintentionally led to a divide between learning (research) and using (everyday practice).5
Nowadays, medical information is largely stored digitally. This digitalization means that the infrastructure to start learning from secondary data is present, but this possibility is often untouched or undeveloped. In recent years, there has been an increased interest in real-world data in clinical research by regulatory organs such as the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA).6 Although RWE can provide valuable insights in effectiveness and generalizability of practical interventions, it is unlikely to replace RCTs since these remain the gold standard for measuring efficacy.7
In general, RWE has strong limitations in early stage clinical research, but it might be a powerful tool for performing post-approval surveillance and effectiveness validation. In addition to this, RWE might be useful for capturing rare adverse effects and drug-drug or drug-disease interactions that RCTs miss due to its relatively small sample size and sometimes limited follow-up period.8 Another advantage is the relatively easy way in which RWE can be obtained through the use of digital technologies which allows for certain gaps in the factual record to be quickly filled.9 Finally, RWD can also be used to estimate the effect of certain confounders and comedication on the effect and effectiveness of a drug.
2.2. DIGITAL HEALTHCARE SOLUTIONS AND SECONDARY DATA
The digital explosion since the beginning of this century has spawned an incredible amount of healthcare applications that all generate health-specific data.10 Over the last few years, the use of ePROM-questionnaires has become a well-known tool for collecting patient-specific health data in a secondary care setting. There has been evidence showing that electronic patient-reported outcomes (ePROs) improve both quality of life and survival for cancer patients receiving immune checkpoint inhibitor therapy through increased follow-up and predictive outcome modelling.11,12 Other studies in the field of oncology have shown an increase in survival probability when ePROMs are used for symptom monitoring in routine care.13
In the field of chronic kidney disease (CKD), researchers have developed a system using ePROMs to assist the management of patients with advanced CKD. The system used remote self-reporting on an online platform which was shared real-time with the clinical team through the electronic patient record. The study supported the potential of such software in improving patient outcomes and reducing health service costs.14 In the field of cardiology, studies suggest that the collection of digital data through ePROs can reduce the follow-up burden on medical staff while improving individual symptom management for patients through better engagement and patient education.15 Several other recent studies discuss the potentially positive effects of using ePROs and their reliability in the monitoring of chronic diseases.16,17
However promising, the use of most available digital solutions is limited to a hospital-patient context and highly treatment-specific. Although research groups are providing evidence for the utility of digital methods in patient health care, large-scale implementation of ePROMs and utilization of their data remains absent.
2.3. PITFALLS OF REAL-WORLD EVIDENCE
The use of RWE in research has a lot of potential, but it is important to keep in mind that RWE should not and cannot replace an RCT as a source of clinical evidence. It can, however, form a very strong source of complementary evidence when combined properly with RCT results, as seen in pharmacovigilance.9 There are still many technical difficulties that have to be overcome. First, many sources of RWE do not collect their data with a specific research question in mind. Also, much of the data collected from non-traditional sources requires cleaning before it can be used for research purposes.18 When a lack of methodological savvy results in a poorly designed study or analytic design, the risk increases that RWE will result in more noise than valuable data.3
Connecting the dots between different stakeholders of the health care process will be challenging in a heavily fragmented healthcare system. Another future issue of RWD that may arise concerns is the ownership of the generated data. While ownership is very clear-cut in an RCT, this becomes less obvious for secondary use of data as RWD.19
Besides this, there are also several risks of bias. First, there is a selection bias. Unlike RCTs, where selection bias stems from the drafting of patients that are deemed eligible for the clinical trial, RWE is usually generated from non-randomized groups. This makes it nearly impossible to know whether the effectiveness of an outcome resulted from differences in types of intervention or differences in patient selection. Another bias forthcoming from relying on observational studies, is performance bias. This bias arises since observed patients may adhere to treatment plans differently, such as varying degrees of success in taking medications at the right time and dosage. There is also attrition bias, referring to patients withdrawing from certain treatments due to concern about e.g., reimbursement or side effects. Finally, there is the general concern of publication bias. This is the risk of industry-funded trials only reporting the positive outcomes. Since the cost of RWE is a lot lower compared to RCTs, this increases the risk of only reporting the positive aspects of a trial or large dataset and withholding the negative outcomes.20 Besides the biases mentioned above, another risk in collecting RWD is confounding, also known as indication bias. When a patient takes other medications, this can have a confounding effect on the trial outcome. A statistical method which is currently used to minimize some of these biases is the use of propensity score methods.21 In order to valorize RWE as a powerful source of clinical data, improvements and innovations must be made.
3. OBTAINING REAL RESEARCH VALUE FROM REAL-WORLD DATA
The use of PROMs in a secondary care context has been proven to be beneficial for both patients and healthcare professionals.22 As discussed above, the most outspoken benefits of gathering data directly provided by the patient are an increase in quality of life and a decrease in disease burden.12,16,17,22 Next to a wide array of use cases in patient-centered care, such as patient-empowerment and improving health literacy, the data generated through ePROM-questionnaires can be anonymized, aggregated and analyzed to create deeper insights in diseases and their care pathways.15 In doing so, this RWD could help bridging the knowledge gap between clinical research and everyday use.
Although RWD distilled from ePROMs is prone to bias, some progress has been made. One recent study suggests that the use of electronic alerts has a positive effect on adherence to heart failure therapy in outpatient care.23 If these findings can be extrapolated to reminders for questionnaires, electronic reminders could help challenging performance and attrition issues in the collection of RWD.
Finally, by better documenting and understanding subtle health changes reported by the patient, insights could be generated to serve as a means of prevention. It is well-documented that hospital readmissions are accompanied by larger health care costs.24 However, exactly quantifying the health care capital that could be saved by shifting the focus to early detection of adverse events and prevention remains an unmet challenge. When constructed correctly, tools to collect near real-time data through ePROMs could be used to assess the true value of increased follow-up and prevention.
4. REFERENCES
1. Andrew P, Puddle S, Spencer M, Shaabany M, Fisher G. Patient Reported Outcome Measures Programme. Inst Innov Improv. 2021;(September):1–6. (From: CMS Measures Management System (MMS) Hub)
2. Desomer A, Van den Heede K, Triemstra M, Paget J, De Boer D, Kohn L, et al. Het gebruik van patiëntuitkomsten en -ervaringen (PROMs/PREMs) voor klinische en beleidsdoeleinden. Health Services Research (HSR). Brussel. Federaal Kenniscentrum voor de Gezondheidszorg (KCE). 2018. KCE Reports 303A. . (From: KCE Reports)
3. Sherman RE, Anderson SA, Dal Pan GJ, Gray GW, Gross T, Hunter NL, et al. Real-World Evidence — What Is It and What Can It Tell Us? N Engl J Med. 2016;375(23):2293–7. (From DOI: 10.1056/NEJMsb1609216)
4. Kabisch M, Ruckes C, Seibert-Grafe M, Blettner M. Randomized Control Trials. Dtsch Arztebl. 2011;108(39):663–8. (From DOI: 10.3238/arztebl.2011.0663)
5. Eichler HG, Bloechl-Daum B, Broich K, Kyrle PA, Oderkirk J, Rasi G, et al. Data Rich, Information Poor: Can We Use Electronic Health Records to Create a Learning Healthcare System for Pharmaceuticals? Clin Pharmacol Ther. 2018;105(4). (From DOI: 10.1002/cpt.1226)
6. Arlett P, Kjaer J, Broich K, Cooke E. Real-World Evidence in EU Medicines Regulation: Enabling Use and Establishing Value. 2022;111. (From DOI: 10.1002/cpt.2479) Available from: www.cpt-journal.com
7. Beaulieu-Jones BK, Finlayson SG, Yuan W, Altman RB, Kohane IS, Prasad V, et al. Examining the Use of Real-World Evidence in the Regulatory Process. Clin Pharmacol Ther. 2020;107(4):843–52. (From DOI: 10.1002/cpt.1658)
8. Tatonetti NP, Ye PP, Daneshjou R, Altman RB. Data-driven prediction of drug effects and interactions. Sci Transl Med. 2012;4(125). (From DOI: 10.1126/scitranslmed.3003377)
9. Evans K. Real World Evidence: Can We Really Expect It to Have Much Influence? Drugs - Real World Outcomes. 2019;6(2):43–5. (From DOI: 10.1007/s40801-019-0155-3)
10. Kuehn BM. Is there an app to solve app overload? JAMA - J Am Med Assoc. 2015;313(14):1405–7. (From DOI: 10.1001/jama.2015.2381)
11. Iivanainen S, Alanko T, Peltola K, Konkola T, Ekström J, Virtanen H, et al. ePROs in the follow-up of cancer patients treated with immune checkpoint inhibitors: a retrospective study. J Cancer Res Clin Oncol. 2019;145(3):765–74. (From DOI: 10.1007/s00432-018-02835-6)
12. Iivanainen S, Ekstrom J, Virtanen H, Kataja V V., Koivunen JP. Electronic patient-reported outcomes and machine learning in predicting immune-related adverse events of immune checkpoint inhibitor therapies. BMC Med Inform Decis Mak. 2021;21(1):205. (From DOI: 10.1186/s12911-021-01564-0)
13. Basch E, Deal AM, Dueck AC, Scher HI, Kris MG, Hudis C, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA - J Am Med Assoc. 2017;318(2):197–8. (From DOI: 10.1001/jama.2017.7156)
14. Kyte D, Anderson N, Auti R, Aiyegbusi OL, Bishop J, Bissell A, et al. Development of an electronic patient-reported outcome measure (ePROM) system to aid the management of patients with advanced chronic kidney disease. J Patient-Reported Outcomes. 2020;4(1). (From DOI: 10.1186/s41687-020-00223-8)
15. Masterson Creber R, Spadaccio C, Dimagli A, Myers A, Taylor B, Fremes S. Patient-Reported Outcomes in Cardiovascular Trials. Can J Cardiol. 2021;37(9):1340–52. (From DOI: 10.1016/j.cjca.2021.04.006)
16. Bell K, Dykas C, Muckian B, Williams B, Rainey H, Comberg M, et al. Patient‐Reported Outcome Information Collected from Lupus Patients Using a Mobile Application: Compliance and Validation . ACR Open Rheumatol. 2021;0(0):1–11. (From DOI: 10.1002/acr2.11370)
17. Generalova O, Roy M, Hall E, Shah SA, Cunanan K, Fardeen T, et al. Implementation of a cloud-based electronic patient-reported outcome (ePRO) platform in patients with advanced cancer. J Patient-Reported Outcomes. 2021;5(1). (From DOI: 10.1186/s41687-021-00358-2)
18. Davis B et al. Getting real with real-world evidence: Deloitte’s Real-World Evidence Benchmark Survey shows life sciences companies have room for improvement. Deloitte Insights. 2016; (From: Deloitte Insights)
19. Asche C V., Seal B, Kahler KH, Oehrlein EM, Baumgartner MG. Evaluation of Healthcare Interventions and Big Data: Review of Associated Data Issues. Pharmacoeconomics. 2017;35(8):759–65. (From DOI: 10.1007/s40273-017-0513-5)
20. Krause JH, Saver RS. Real-world evidence in the real world: Beyond the FDA. Am J Law Med. 2018;44(2–3):161–79. (From DOI: 10.1177/0098858818789423)
21. Johnson SR, Tomlinson GA, Hawker GA, Granton JT, Feldman BM. Propensity Score Methods for Bias Reduction in Observational Studies of Treatment Effect. Rheum Dis Clin North Am. 2018;44(2):203–13. (From DOI: 10.1016/j.rdc.2018.01.002)
22. Lisa S. Rotenstein, M.D., M.B.A., Robert S. Huckman, Ph.D., and Neil W. Wagle, M.D. MB. Making Patients and Doctors Happier — The Potential of Patient-Reported Outcomes. NEJM. 2017;337(14):1309–12. (From DOI: 10.1056/NEJMp1707537)
23. Ghazi L, Yamamoto Y, Riello RJ, Coronel-Moreno C, Martin M, O’Connor KD, et al. Electronic Alerts to Improve Heart Failure Therapy in Outpatient Practice. J Am Coll Cardiol. 2022;79(22):2203–13. (From DOI: 10.1016/j.jacc.2022.03.338)
24. Kwok CS, Abramov D, Parwani P, Ghosh RK, Kittleson M, Ahmad FZ, et al. Cost of inpatient heart failure care and 30-day readmissions in the United States. Int J Cardiol. 2021 Apr;329:115–22. (From DOI: 10.1016/j.ijcard.2020.12.020)