1. Introduction
The promise of precision oncology is to deliver key information and have gained based on molecular specific alterations to individual cancer patients in the hope of finding a tailored treatment solution for their disease. Initial large scale attempts to profile patients in the hope of directing them to molecularly matched clinical trials have highlighted that only a small proportion of patients have actionable alterations which are suitable for enrollment into a trial. In 2,000 patients profiled at M.D. Anderson, of whom 789 (39%) had at least one alteration in potentially actionable genes, only 83 (4.2%) were enrolled in a matched clinical trial [1]. The use of a comprehensive assay, MSKIMPACT, to prospectively sequence 10,000 patients with diverse cancers at Memorial Sloan Kettering found that, despite 37% of tested tumors harboring a clinically relevant alteration, only 527 (11%) of the first 5,009 patients profiled were enrolled onto genomically matched clinical trials [2]. The initial experiences have left some oncologists to express frustration at the relatively few patients to have benefited from precision oncology, and there is some debate on whether this is due to the approach in general or reflects the methodology used, tumor types or availability of appropriate drugs [3,4]. Tumor sequencing is already used as standard of care to guide routine treatment decisions in many cancers, but the basic tenet of precision medicine is that this needs to have broad application across all solid cancers [5]. This has lead to the development of large basket trials such as the National Cancer Institute’s Molecular Analysis for Therapy Choice (NCI-MATCH) and Targeted Agent and Profiling Utilization Registry (TAPUR) to look at the outcomes of matched therapy. For oncologists who are actively integrating tumor profiling into their patient’s care today, it can be challenging to understand the differences in clinical utility and benefit between numerous available molecular profiling services. The often-considerable costs at the patient’s expense require justification. Limited data exist about the clinical outcomes achieved with different profiling solutions. Molecular tumor profiling utilizes a panel of biomarkers that may influence therapy selection for an individual patient. This may involve single or combined technologies including Next-Generation Sequencing (NGS), Immunohistochemistry (IHC), Polymerase Chain Reaction (PCR) or In Situ Hybridization (ISH). A handful of biomarkers have strong associations with treatment outcomes and gained regulatory approval as “Companion Diagnostics”. Others have a less validated association better suited for patients for whom no well-documented standard of care exists. Limited data exist on the use of predictive biomarkers and the respective therapies for which utility was established in one cancer type, in different cancer types. A few examples, like the use of trastuzumab for NSCLC with activating HER2 mutations, have been included in clinical guidelines based on limited clinical data, but as knowledge increases, we will see other examples emerge from research in clinical trials or serendipitous case reports.
The validity of such predictive biomarker results is crucial since oncologists often prescribe drugs with major side effects based on the results. It is inadvisable to blindly trust the results of a molecular profile, as many factors may influence therapeutic response.
All services now have adapted to work with formalin fixed paraffin embedded tissue specimens, which is the routine method of pathological analysis used by diagnostic laboratories worldwide. Despite this, it is remarkable that it is a relatively uncontrolled process in a clinical setting (e.g. devitalization and fixation time variations, differing reagents and processing schedules etc.), underlining the need for comprehensive and rigorous quality control to understand which biomarkers can be analyzed reproducibly and more importantly which cannot (e.g. phosphorylated proteins). The leading professional molecular profile services include several layers of quality checks and controls, which are almost impossible to implement in a smaller service with less volume and expert resources.
The development of less invasive technologies such as profiling circulating free DNA or tumor cells might be initially more appealing to patients, however, these technologies are still highly experimental and certainly not validated for clinical decision-making. We have therefore excluded these methodologies from this review.
Key differences between molecular profiling services lie in the range and scope of platforms used to assess biomarkers. Whereas currently approved companion diagnostics would require the use of several technologies for their assessment (e.g. IHC, ISH & NGS for HER2 analysis) those are neither established nor validated in some laboratories. Importantly, while NGS is able to detect single nucleotide variants (SNVs), insertions and deletions (InDels), gene amplifications, and some fusions, it is not able to detect or reliably predict transcriptional regulation or changes in protein expression. This becomes a limitation if protein expression is related to the drug effect.
The published medical scientific data about drug biomarker associations is extensive and requires frequent updating of biomolecular panels to ensure optimal relevance. This is best accomplished by an expert multidisciplinary team that shares its understanding with the oncologist to support therapy decisions, either via direct communication or by developing algorithms that generate a concise qualitative and quantitative report with potential therapeutic options, that is shared with the treating oncologist. The interpretation of the profiling results for the treating oncologist is essential in determining the ultimate treatment decision and thus optimize outcome for each particular patient.
It is imperative that selection of cancer therapy in patients at late stages is derived from an evidence base and can demonstrate a positive clinical benefit versus toxic side effects ratio to justify continued investment from Governmental or insurance medical care providers [6,7].
The true value of prospective tumor profiling is measured by its ability to influence treatment decisions and improve outcomes for patients. In this paper, we aim to compare and contrast the clinical utility of 4 commercial approaches, along with those from leading academic institutions to show that not all approaches to precision medicine are the same in terms of quality and cost-effectiveness. This research can be the basis for inclusion of newer decision algorithms in cancer patients.
2. Impact on Treatment Choice
We compared the treatment choice impact of 4 commercial services in this paper – Caris Molecular Intelligence® from Caris Life Sciences, FoundationOne® from Foundation medicine, PCDx™ from Paradigmdx, and OncoDEEP™ from OncoDNA to independent academic clinical trials by analyzing published clinical data [8-55]. As the services are offered for all solid tumors, and the actual treatment choice itself reflects the real-world situation and not theoretical actionable alterations, all prospective and retrospective studies were considered without differentiation on country, prior treatments or tumor type. CMI has the highest impact on treatment choice (77%), followed by PCDx™(26%), and FoundationOne®(19%) (Figure 1A and Table 1). The impact on treatment choice of profiling services that have been developed by academic institutions in house is similar to that of FoundationOne®. For OncoDEEP™, we were unable to find any published references that demonstrated impact on treatment choice or corresponding clinical benefit in a patient population, so no clinical utility for this profiling service can be documented.
3. Clinical Benefit
Clinical benefit can be defined as a positive impact on patient health as defined by the treating physician and varies between the clinical studies of incorporating profiling approaches into clinical practice. Achieving clinical benefit from molecular profiling-guided treatments depends on several critical factors that all need to be considered: the inclusion of appropriate predictive biomarkers in the panel, the frequency of each biomarker in the respective patient population, measurement of all biomarkers in the panel with the necessary precision, the correlation of each biomarker to clinical outcome, the proportion of associations based on clinical evidence rather than preclinical or animal models and the guidance provided to the treating oncologistin support of selecting a treatment for an individual patient. The available profiling services differ in content for each of these factors and documented treatment outcomes are the most suitable overall measure of success incomparing different services. We reviewed published clinical studies and case series of patients who underwent molecular profiling using Caris Molecular Intelligence®(534 patients) [8–18], FoundationOne®(2,675 patients) [19–44], PCDx™(168 patients) [45], and academic approaches developed at leading academic institutions (5,576 patients) [46-54] (Table 1).
Clinical benefit was reported in a number of different ways. In the physician-led studies, disease stabilization and improvement of profiling-guided outcomes compared to prior outcomes were considered to define clinical benefit while more conventional methods of assessing response were generally used in the academic studies. The clinical benefit in those patients in whom a treatment was selected based upon the molecular profile is summarized in Figure 1b. The benefit is highest with Caris Molecular Intelligence® (50%), followed by PCDx™ (43%), FoundationOne®(34%) and academic approaches (33%). It was not possible to evaluate the clinical benefit of OncoDEEP™ as there has only been a single case report published to date [55].
4. Clinical Utility
Clinical utility is a measurement of both the influence that molecular profiling has on oncologist’s treatment decisions and the clinical benefit of the resulting treatments. A high utility means treatment decisions for the majority of patients have been revised in line with the profiling results and that the chosen treatments led to improved clinical outcomes. The impact on treatment choice is directly dependent on the panel of biomarkers tested, the frequency of those biomarkers in the population and the level of evidence presented to the oncologist in support of a change in treatment decision.
In precision medicine in cancer, clinical utility is defined as the generation of clinical useful and relevant information that can change the course of the disease for a patient, resulting in improved outcomes [56-58]. The individual patient’s likelihood of clinical benefit within the whole profiled patient group is the most critical measure for a patient and their oncologist when setting expectations of what a molecular profile can offer. Furthermore, clear communication of this information leads to realistic expectations and can build the patient’s confidence and mental strength which in themselves are reported to lead to improved treatment outcomes in patients with metastatic solid tumors [59], a perhaps unexpected significant benefit of undertaking molecular profiling. If clinical benefit is restricted to a limited subset of patients within a tumor group (e.g. those with a specific mutation or only one ‘targetable anomaly’), the expectations of successful outcomes for an individual patient should be similarly reduced. Based on the demonstrated impact on decision choice and the clinical benefit observed in guided treatments, Caris Molecular Intelligence® shows clinical utility in 34% of all patients profiled, PCDx™ in 11%, and FoundationOne® and academic studies only in 6% and 5% respectively (Figure 1c). A representative graph comparing decision choice impact and clinical benefit from all approaches is shown in Figure 2.
There is a significant difference in clinical utility between Caris Molecular Intelligence® and other commercial providers. In order to determine the underlying reason for this difference, the actual therapies chosen to treat individual patients that were reported in some of the clinical studies was analyzed (Table 2). It is apparent that after profiling with Caris Molecular Intelligence®, a large number of patients were treated with cytotoxic chemotherapies (alone or in combination) whereas the fraction of targeted therapies was highest for FoundationOne®. This can be explained with the panel of biomarkers used in the respective molecular profiling, since FoundationOne® is limited to genomic profiling. The same biomarkers would also be measured with Caris Molecular Intelligence® and in addition, Caris Molecular Intelligence® offers a number of biomarkers measured with IHC and ISH which are predictive of outcomes from treatment with cytotoxic chemotherapies. The information provided by NGS is mainly relevant to novel targeted drugs and many of such known targets are uncommon. In contrast, the protein biomarkers measured with Caris Molecular Intelligence® in addition are usually more frequently informative, leading to a higher clinical utility (more treatment decisions can be based on the results) and a higher clinical benefit. Based on the sparsity of relevant targets and a lack of availability of drugs targeting appropriate alterations, the clinical utility of comprehensive genomic profiling is limited to about 5% of all tumors profiled.
4.1 Technical details, quality and turnaround time
Different technical details such as the technology platforms used, the biomarker panels, attention to pre-analytical quality of tissues and original diagnosis, and the validation of individual assays can explain the differences in clinical utility in the profiled patients from the services we compared. Caris Molecular Intelligence® has in-house pathological analysis and review and any issues regarding diagnosis and sample selection can be discussed among experts before committing to the full profiling.
With regard to the choice of NGS, Illumina is viewed as the industry standard, and has achieved FDA approval for medical use e.g. in diagnosing cystic fibrosis. Moreover the leading Academic institutions have chosen it (e.g. MSKCC, Dana Farber). An overview of the development and validation of a clinical NGS test is given by Frampton and his team [60]. Further guidance on the best practice and necessary steps for integration of NGS in clinical diagnostic molecular pathology laboratories has been described by the IQN Path ASBL group [61]. Caris Molecular Intelligence® and FoundationOne® use Illumina but PCDx™ and OncoDEEP™ use Ion Torrent which has not been approved for medical diagnostics and is intended for academic research. This is related to the fact that only hotspots on a single strand of DNA are sequenced in only one direction, which leads to a high error rate, in particular for homologous insertions and deletions where the error rate is doubled compared to Illumina[62]. The Ion Torrent platform does have advantages in turnaround time due to the shorter time needed to sequence data from hotspots only as shown in a direct comparison of the PCDx™ and FoundationOne® platforms when the median turnaround time was 9 days earlier for PCDx™ compared to FoundationOne® [63]. The multiplatform approach of PCDx™ also revealed more associations with commercially available drugs and less clinical trial associations than the single platform approach of FoundationOne®. A recent case report highlights the discordance between the two NGS platforms and the possible challenges in managing a patient resulting from it [64]. Reliable detection of sensitizing mutations associated with commercially available drugs is critical and a major problem is that many of the platforms may lack rigorous analytic validation. For example, a comparison of EGFR activating mutations sensitive to erlotinib or gefitinib between FoundationOne® and local testing found that 12 of 71 (17%) cases identified with FoundationOne® were missed by prior local testing [65].
The different companies also vary in their approach they take towards protein testing using IHC. Ideally each IHC test must adhere to the strict quality requirements from CAP, CLIA and NYSDOH and meticulous testing of all reagents for use in assays would be mandatory as described by O’Hurley and colleagues [66]. Caris Life Sciences has developed laboratory-developed tests (LDT) with comprehensive testing of all reagents used for its entire panel of IHCs (except where an IVD test kit is commercially available).While the NGS-section of the OncoDNA laboratory is ISO15189 accredited all other tests are performed without meeting internationally accepted quality standards for clinical use. OncoDEEP™ has also included phosphorylated proteins in their panel but the phosphorylation of proteins is known to be highly sensitive to pre-analytical processing producing results that may be misleading for a treatment decision [67].The technical details of the services are summarized in Table 3.
The turnaround time of commercially available profiling services is advertised between 4 and 14 working days depending on service provider; the time for sample shipment must be added to this. For a detailed comparison, please see Table 3.
4.2 Technical details, quality and turnaround time
To date, the majority of traditional cost effectiveness models have been designed to show a cost benefit in adding one drug or regimen over an existing comparator. In looking at the cost effectiveness of precision medicine approaches, new models must be considered. Previous modeling work from Shen and colleagues based on the Cancer Genome Atlas (TCGA) estimated the cost of finding a theoretically actionable mutation at between $6,254 and $22,907 and a pharmaceutically actionable mutation at between $8,035 and $55,556 depending on tumor type [68]. The true cost of profiling should not be considered as a reflection of the unit price but rather the amount that needs to be invested to bring clinical benefit to a patient. The key factors that must be taken into consideration are how often a molecular profile leads to a guided treatment being given to a patient, how often that treatment leads to clinical benefit, and the overall clinical utility. If we assume that all drugs selected by profiling are of comparable benefit, then it is imperative that impact on decision choice must be used to differentiate between approaches as it gives the easiest estimate of return on investment, be it in individual patients or on a larger scale from medical care providers.
List prices for the respective services were based on telephone enquiry or accessed online [68,69]. A summary of the cost effectiveness of the commercial and academic approaches based on the unit price of testing and the clinical utility is presented in Table 4. It is clear that approaches that have low clinical utility in the overall profiled population are unsustainable. The cost of profiling per patient tested with clinical benefit using Caris Molecular Intelligence® at $19,118 is less than a quarter of the equivalent cost of $96,667 using FoundationOne®’s NGS only approach and half that of the other approaches. This cost does not include any consideration for the recommended therapies, which would further extend the gap in cost given the high price of targeted therapies compared to conventional cytotoxic agents. This also means that approaches that do not show any clinical utility must be treated with some skepticism, as the cost effectiveness cannot be estimated. We estimated the true cost for an in-house approach that must include not only full costs (as opposed to the commonly adopted “marginal costs”) of the materials and staff time, but also the ongoing investments in space, equipment, and ongoing training. The predicted number of patients profiled per year is also critical as a low throughput is likely not cost-effective.
5. Conclusion
Patients, medical care providers, pharmaceutical companies, researchers and oncologists have different requirements to utilize molecular tumor profiling for effectively treating advanced cancers. Patients demand a service that gives them a high likelihood of individual benefit (=clinical utility) at a justifiable price; often, they are not able or willing to pay excessive amounts for a molecular profiling with a low expectation of success. Medical care providers need to evaluate cost effectiveness of molecular profiling for the entire cohort of their insured patients, with a method that allows comparison with other innovations in health care. They need to provide care packages which include molecular profiling and are commercially viable and seek to avoid large expenditures on ineffective therapies.
A secondary advantage for pharmaceutical companies is the acceleration of clinical studies because patients with rare alterations can be reliably identified with molecular profiling and factored into clinical studies (perhaps explaining non-responders to a novel agent) or identifying specific cohorts for early development of therapies. Profiling data is also of high interest for the identification and selection of targets and the design of new drugs.
Oncologists need to navigate between all these needs and find solutions that satisfy all major stakeholders in the treatment of their patients. We showed that molecular profiling with NGS only is very well suited to search for patients that meet the inclusion criteria for clinical trials of novel targeted drugs. It is therefore no surprise that academic institutions and drug companies mainly support broader use of NGS alone, be it through their in house developed profiling or through Foundation Medicine.
Molecular profiling reveals more insights; however, the information gained is often telling us about potential oncogenic drivers of the individual cancer rather than informing a treatment decision. When it comes to the selection of commercially available drugs, NGS has limited utility because most frequently proteins are identified as predictive biomarkers for conventional cytotoxic drugs, which remain the backbone of treatment in oncology. If the benefit for individual patients is the main goal it is relevant that the impact on decision choice of NGS is only around 20% and clinical utility is no higher than 5%.Given the diversity between the biomarker panels and the quality and reproducibility of the tests offered, it is important that a molecular profiling service has a track record and provides solid evidence of proven clinical utility and benefit. This creates a basis for a clinical evaluation and comparison. Offerings with no or little clinical data cannot be recommended for use in the clinical setting(although they might be suitable for research evaluation).
In the US, every commercial laboratory needs to be certified by CAP and CLIA who provide stringent guidance and have instituted regular auditing procedures. Quality requirements are even higher in some states, for example in New York, where the Department of Health enforces strict criteria for reproducibility and constant results in the population over time. In Europe, quality has traditionally been demonstrated by ISO 15189 accreditation but is more focused on processes and not on the precision of specific laboratory tests or on pre-analytical requirement specifications. We have reviewed the quality thresholds met by Caris Life Sciences, Foundation Medicine, Paradigm, and OncoDNA. Only Caris Life Sciences and Foundation Medicine are allowed to offer their entire services in New York, Paradigmdx holds CLIA certification and OncoDNA is not even approved to sell in the US at all. In house services are only held accountable to local standards and only in the US one can safely assume that acceptable quality criteria are met. The new Diagnostics Regulation (IVDR), which was endorsed by the European Parliament on April 5, 2017, may address this issue in future [70].
The validation of IHC tests for which no standardized antibodies are commercially available is complex and comprises extensive testing of the reagents. Only very few academic institutions have research projects in which this is included and Caris Life Sciences is the only commercial laboratory that has implemented this approach to quality assurance. It also emphasizes the necessity of having inhouse expertise in pathology and analysis to corroborate the IHC findings with appropriate and validated controls. All these extra steps may simply not be economical for smaller companies and academic institutions, but the additional complexity is translating into better clinical outcomes and therefore well worth the expense.
When we compared the clinical utility achieved with the different platforms, it was impressive to see how uniform the results were between the offerings including NGS only. In a large number of patients, both FoundationOne® and NGS in-house testing has proven decision choice impact in about 20% of the patients and brings clinical utility to 5% of the patients. This is no surprise as the frequency of driver mutations and the number of associated drugs is limited. Adding on more tests with different platforms, mainly IHC, adds more clinical utility. Both PCDx™ and Caris Molecular Intelligence® have included a number of IHCs in their offerings. PCDx™ achieves a twofold and Caris Molecular Intelligence® achieves a six fold higher clinical utility rate. Caris Molecular Intelligence® has the highest clinical utility and benefit because it includes the most clinically relevant biomarkers.
The high clinical utility from Caris Molecular Intelligence® translates into the best cost-effectiveness. While both FoundationOne® and inhouse NGS profiling share the same clinical utility we assumed a lower cost for in-house testing, well aware that this may not include the true cost of the testing. Under these assumptions, FoundationOne® would be the most expensive approach, in-house testing and PCDx™ would cost half of that, and Caris Molecular Intelligence® could be delivered at a quarter of the costs. A detailed analysis of the cost effectiveness is not only essential for medical care providers but also every patient who needs to pay for their profiling should be informed of these aspects by the treating physician. A very important but non-measurable benefit from a high clinical decision impact is the level of confidence for patients and their treating oncologists in having chosen a drug with a higher likelihood of benefit than a randomly selected drug.
In conclusion, molecular profiling has become an essential element in the selection of treatments for advanced or metastatic solid tumors.
Competing Interests
WMG is a co-founder of and Chief Scientific Officer with OncoMark Limited. The other authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript.