Written by: Jeff Elton, Ph.D., Mike Rossi Ph.D.,
and Kamala Maddali, DVM Ph.D.

There is a confluence of technically enabled data access, data federation, and advanced computational sciences that will reshape Translational Sciences in oncology.  This federated data at scale is often termed multi-modal data (MMD) and is the binding of diagnostic radiological, NGS/ transcriptomic, digital pathology, other laboratory, EMR-derived clinical, medical claims-derived clinical, and Social Determinants of Health (SDoH). The alignment and integration of these data support novel integrative analyses and change how scientific and clinical hypotheses are framed and explored.

Historically discovery and translational sciences would have explored model systems including immortal cell lines, animal models, etc. in pursuit a deep understanding of the process of diseases in different cancers and to elucidate complex disease pathways in service of finding new drug targets that could avoid past patterns of resistance or the built in redundancies of cell processes that often thwart different therapeutic approaches. It is now possible to directly align translational research to patient strata based on genomic and transcriptomic factors.  Multimodality machine and deep learning approaches will enhance and accelerate the discovery of biomarkers guiding treatment approaches.

Whereas single uni-modal or bi-modal analyses historically had value, it was also clear there were limitations to their explanatory power.  Confounders were frequent, which in turn, led to long-term and relatively slow experimental approaches that were uncertain in their ultimate utility.  Today, ConcertAI using its broad clinical data access through a network of research and clinical care partners, is able to bring together electronic medical record, medical claims, and radiological images from millions of patients.  We are joining forces with some of the leading genomic, diagnostic, drug discovery and data science companies in the industry.  For example, through our Caris Life Sciences partnership, we are bringing together for the first time a view of patients’ cancers at population scale with a genomic, transcriptomic, and clinical depth exceeding all past programs – private or public.  We are doing this with the intent of accelerating medicines in discovery or development of the world’s leading biopharma innovators and assuring that academic investigators have access to data, technology and expertise beyond what single research centers are capable of generating locally.

Multi-modal data is further driving new domains and capabilities in data management and data preparation. Data Fusion methods are evolving rapidly to extract and combine complementary contextual information across modalities in support of specific model development approaches and for enhanced interpretability.  In translational research, findings in one modality may have multiple interpretations, but findings of fused multi-modal data analyzed in a semi-supervised or weakly-supervised approach can eliminate some of the noise and confounders of single-modal data to improve the robustness, accuracy, and the ultimate utility of predictions.  With the latest generation of Generative AI (e.g., GPT-4) data fusion approaches that determine the modalities exposed and complementary elements holds the promise of accelerated insights.  New methodological approaches will also stand in assurance of zero or limited hallucinations or spurious relationships with no disease process associations.  Still, the outcomes of all of these analyses will need a rigorous validation process, elucidating the self-defined model structures and assuring their predictive power holds up in real-world clinical settings.

Speaking to individual patients, a key aspect of genomic testing results is the understanding that the data gathered from a tissue or liquid biopsy specimen is a timepoint that must be interpreted within the context of other information including prior therapies, clinical history, and other testing results. Liquid biopsy is providing the means to perform more serial testing and minimal residual disease (MRD) methodologies are allowing physicians more opportunities to test pre- and post-treatment with advanced limits of detection. It is critical that this information be supported by imaging data and other clinical features captured in EMR, claims and unstructured physician notes. When these disparate types of information are centrally located and structured to facilitate multipartite questions, it becomes clear how AI can be incorporated in routine cancer care and the value this data has in developing multi-modal biomarkers of new or improved combination therapies.

We now know that monotherapy has limited applications in most malignancies. The future of combined therapies are two or more antibodies or compounds with different mechanisms of action attempting to exploit different co-dependent biological vulnerabilities. Combining immunotherapy targeting PD-1, PD-L1, LAG3, etc. with antibody drug conjugates (ADCs) or proteolysis-targeting chimeras (PROTAC) targeting specific proteins essential for tumorigenesis and disease progression requires algorithm-based biomarkers derived from multi-modal variables. Centralizing these multi-modal variables allows for opportunities to develop external control arms for clinical trials rooted in more practical and applicable clinical scenarios.

Cancer is an evolving disease and the best strategies for treatment require real time and real-world data. For example, many successful first and second line therapies, eventually fail. For most patients, moving from one line of therapy to the next often requires evidence of disease progression and in third or later lines of therapies the choices are often limited. At ConcertAI and working with our Caris colleagues, we are establishing real-world multimodal datasets that provide insights that allows our clients to better predict disease progression and develop therapies that anticipate real world clinical scenarios to maximize clinical trials success.

Multi-modality is not just relevant to retrospective research.  ConcertAI and its partners are enabling collection of these data after the approval of new medicines to assure ongoing insights and refined approaches to stratification and treatment in assuring of outcomes improvements year-over-year beyond the initial benefit seen in the pivotal trial. We are integrating multi-modal principles into Digital Trial Solution being deployed in scores of leading community research centers around the country. Because our solutions have a high level of automation in the conduct of registrational clinical studies we can also expand the scope of data collected without placing an undue burden on either the research site or the patient.

In the end we do this all for one reason, furthering the options and the ultimate elimination of cancers placing the highest burden on patients.  This includes changing aspects of patient diagnosis, stratification, to define those treatment options most likely to deliver meaningful therapeutic benefit.  With ASCO 2023 approaching, we hope you will take the time to visit us and attend sessions we are holding on this topic.