Creating a Learning Healthcare Care System in Oncology:

Creating a Learning Healthcare Care System in Oncology: ASCO’s CancerLinQ Presentation for the LHS Collaboratory University of Michigan October 5, 2017 1

I Plan to Address the Following: Why is a Learning Healthcare System Important? Why is it Necessary in Oncology? What Could it Accomplish? What Are the Risks, Obstacles, and Progress to Date?

A Definition:

Embedding Research into Practice We’ve lived in a world where research was in one side of the house and clinical care was on the other. We now have an opportunity to link the two. Classical research involving classic clinical trials will continue. But we can also aggregate data from our routine clinical care and gain valuable insights from massive numbers of patients. We can now do critically important research on our clinical care!

Why is a Learning Healthcare System Important? Why is it Necessary in Oncology? What Could it Accomplish? What Are the Risks, Obstacles, and Progress to Date?

A Key Example:

Compared to clinical trial subjects, the patients we see every day tend to be: less healthy older 25 % of clinical trial patients are 1 vs 61 % 40% of of real-world patients are 65 65 1 kidney cancer patients were not healthy enough to qualify for the trials that supported the approval of their treatments2 and more diverse 90% 23% vs of patients in NCI trials are white 3 of the US POPULATION is nonwhite3 1. Lewis JH, et al. Participation of patients 65 years of age or older in cancer clinical trials. J Clin Oncol. 2003;21:1383-1389. http://jco.ascopubs.org/content/21/7/1383.full.pdf. 2. Mitchell AP, et al. Clinical trial subjects compared to "real world" patients: generalizability of renal cell carcinoma trials. J Clin Oncol. 2014;32(suppl):6510. 3. Taking action to diversify clinical cancer research. National Cancer Institute Web site. http://www.cancer.gov/ncicancerbulletin/051810/page7. Accessed July 23, 2014.

The Result is we are essentially treating a huge number of patients “off label”, using clinical trial results from studies they never would have been eligible for. We can and will change this.

9

10

Two Key CancerLinQ Decisions We decided to: 1. Take as much of the clinical record as we could get, not a specific, pre-defined set of data elements. Cancer is actually 100s of diseases. It’s hard to specify data elements for each one. The science of oncology is changing very rapidly. For example, when we started this project in 2010 the was no immunotherapy. 2. We decided to gather the data in a centralized location. We knew we would need to commercialize the data in order to sustain the project. 11

Why is a Learning Healthcare System Important? Why is it Necessary in Oncology? What Could it Accomplish? What Are the Risks, Obstacles, and Progress to Date?

a. Generate New Knowledge It’s about more than understanding how patients not eligible for a trial respond to trial therapy The fact is: No amount of clinical trial effort, and no amount of money and resources, can lead to enough classic clinical research -randomized clinical trials -- to answer all the questions in oncology that need to be answered today. We must have a better, faster, and cheaper way of generating new knowledge. That is what CancerLinQ is designed to do. 13

Our Ability to Run Trials is Disappearing

b. Promote Better Quality Care 15

c. Promote More Cost-Effective Care

d. Data at Your Fingertips 17

Data at your Fingertips “Patients Like Mine” “For many [if not most] clinical questions today the only relevant data available to aid decision making are observational.” C.A. Longurst, Health Affairs, July, 2014 “In cases [where randomization is not feasible] it seems more reasonable to try to estimate the effects of treatments from nonrandomized studies than to ignore these data and dream of the ideal experiment ” D.B. Rubin, J. Educ. Psych, 1974 (as quoted by Longhurst)

e. Improving Clinical Workflow EMRs are Severely Constrained: Built for Billing and Compliance Do not Handle Complex Radiology or Laboratory Tests Very Well Large Parts of the Data are Neither Computable nor Searchable

Much of Healthcare Data is Still Exchanged Today via:

How is Data Transferred From an Outside Lab to an EMR?

CancerLinQ Patient Timeline 22

f. Make sense of common diseases that are now quite complex Example: Lung Cancer Formerly: 3 histologic types, 4 stages, two genders 24 unique patterns Today: There are at least 15-20 molecular drivers known 350-500 patterns Tomorrow: There will be more We have taken a seemingly uniform disease and chopped it up into many, many small pieces. No single institution can effectively cover the full spectrum of even a common malignancy like lung cancer. We must be able to pool data from clinical sites in order to continue to make progress. 23

Why is a Learning Healthcare System Important? Why is it Necessary in Oncology? What Could it Accomplish? What Are the Risks, Obstacles, and Progress to Date?

There is no Shortage of Risks and Obstacles It won’t work. Data security will be insufficient resulting in a data breach. Observational data will yield unreliable insights. The data will be hard to get. The data quality will be insufficient to meet our goals.

We partnered with SAP to Mitigate our Risk #1 Enterprise software 22.2B SAP revenue worldwide 261,000 customers in 190 countries 68,000 employees worldwide 74% world’s transaction revenue 26

CancerLinQ data architecture Collaborator specific private clouds Third party data 27

Our Secure Environment for PHI

Where Is Your Physician Viewing Your Medical Record?

Using Observational Data Many in medicine are skeptical of attributing causation to an association derived from an observational dataset. We have been led down a false paths many times (e.g.: postmenopausal estrogen use). We believe that sophisticated mathematical and computational approaches can allow us to “see” and understand more of what’s in the data, much in the way the microscope enhanced our ability to “see” details that were missed with the naked eye. What is required is careful validation.

Using Observational Data Surely, no one would propose a clinical trial to test whether smoking causes cancer. We are very confident in that relationship because it has been validated in many ways.

CancerLinQ progress to date 101 30 practices/ cancer centers implementations in progress 32 48 active sites practices providing data (27 fully live) 12 2,000 500K active source systems represented oncologists cancer patient records Participation from Case Western, Vanderbilt, U. Va., U. Arkansas, U. Maryland, Rush, Lahey Clinic, Intermountain Health, Stanford, and more coming. 32

Quality Performance Indicators 33

CancerLinQ is Community

SEERLinQ National Cancer Institute: Surveillance, Epidemiology and End Results Program (SEER) SEER is funded by NCI to support research on diagnosis, treatment and outcomes SEER is one of the primary sources of data on cancer incidence and survival in the nation, representing 30% of U.S. population Dataset shared with CLQ includes: 1.2M patients with breast or colorectal cancer (2004-2014) Present SEER data in CancerLinQ Insights (CLQi) and provide meaningful views and visualizations to support population-level analyses by our CLQ practices Example of CancerLinQ’s use of SEER data: Breast cancer cases by stage and year

WE HAVE ENCOUNTERED NUMEROUS CHALLENGES It’s far easier to talk about a learning healthcare system than it is to actually create one.

You Know this is Going to be Difficult But when it hits you

EMR Data is “Not Fit for Purpose” – These systems are extremely costly. – They have fostered unprecedented levels of user frustration and dissatisfaction. – Their output is relatively poor at advancing the clinical care of patients (it is quite good at providing documentation for purposes of submitting an insurance claim). – The structured data is so incomplete we actually need to hire highly trained humans to read the electronic record!

User Dissatisfaction “ we have lost the hearts and minds of clinicians. We overwhelmed them with confusing layers of regulations. We tried to drive cultural change with legislation. We expected interoperability without first building the enabling tools. In a sense, we gave clinicians suboptimal cars, didn’t build roads, and then blamed them for not driving.” Halamka and Tripathi NEJM Sept. 7, 2017

Some of The Data Challenges of the Learning Health System How Accurate are the Data? 1. Correctness: % invalid codes, invalid dates, out-of-range numeric values [dob 1/1/1800] 2. Measurement error: attribution of data elements to incorrect codes [ferritin 2 ] 3. Consistency: amount of variation on repeat measurement [pt weight in kg 72, 70, 71, 73, 69, 70, 98, 72, 70]

4. Completeness: % of missing values Practice-specific rates of structured data: ER, PR, HER2 as % of breast ca patients (n 127K patients, 30 practices) ER PR HER2

Management of rules/codification a major activity at CLQ due to variation between/within EHRs Distinct Lab Names in EMRs per Codified Lab Name (n 30 practices) 80 70 60 50 76 67 62 59 56 55 55 50 48 45 40 30 33 32 31 26 20 23 22 19 19 10 0 # n # it ls % % il bi cr hi il e te o p h lo t yt ph cy p g c a rt o o o o o o r m u m si n on ut on He Ne M Eo He M Ne Pl W el at te hi B et od lo l Ce nt ou C l si Eo il ph no R % ed B od lo l ls Ce es yt c o ph e yt oc ym L h p nt ym ce L r e Pe ut ol s Ab t un Co W RD CV M C CH M Ba il ph so % Ba ls hi p so 15 CH M

Challenges & Remediation Plans Many critical oncologic elements are only found in unstructured documents – e.g., cancer stage, biomarkers, adverse events, disease progression – Incentivize practice workflow changes re data capture – Human curation of pathology reports, imaging, clinic notes ( )

Challenges & Remediation Plans Data not captured in the medical oncology EHR (e.g., hospitalizations, radiation therapy, oral therapy) – Supplement w/ tumor registry data, claims, prescription refill data – Build new data feeds to incorporate other specialists involved in care – Human curation

Additional LHS challenges Workflow issues – CLQ is not the transactional system; so how best to deliver clinical decision support at POC? – We are investigating how to embed a CLQ window in the EHR interface ( , proprietary vendor systems)

Additional LHS challenges Enormous organizational complexity and cost to deliver a software-based, interactive LHS – Product managers, implementation teams, customer support and care teams, security/privacy, science/clinical SMEs, informaticists, engineers/developers, administrative support, change management, data governance, strategic alliances, commercial engagements – Software (custom user interface, back-end admin tools, reporting solution that is customizable and responsive – all w/ near-zero latency) – Hardware (e.g., secure servers to support PHI, backups, fail-safe systems)

What Have We Learned This is extremely difficult to pull off – Almost every EMR hook up is unique – The structured electronic data are sadly deficient – Getting data out of unstructured notes is costly timeconsuming This is not for the feint of heart – We have expended tens of millions of dollars so far and we are not close to done Happily, the oncology community embraces this idea – The uptake into practices is terrific – The collaboration from sister organizations is heartening

We Will Solve These Problems EMRs have to be become better, and they will. But we must pay a lot of attention to this issue. Data quality will improve. Extracting data from unstructured notes will get easier. The value of the data to physicians and patients will continue to rise as we improve our system. We are undaunted in our pursuit of a LHS in oncology. We have embarked on a long journey.

The future of oncology is tied to “Big Data”. ASCO is creating the Learning Oncology Care System: CancerLinQ. This is an ambitious and critically important endeavor. We are undaunted in our commitment to transform the way we learn to care for cancer patients worldwide. 49