Multimodal Analysis

Spatial omics

A wealth of new spatial omics technologies is rapidly arising, providing information on biomolecular targets within their native spatial context. While powerful, each of these individual omics imaging technologies only captures one aspect of biology, be it the proteome, lipidome, glycome, genome, metabolome or transcriptome. Therefore, in order to gain a complete picture, multiple imaging modalities must be combined to gain a deeper understanding of the underlying system. One of Aspect’s goals is to support such multimodal workflows by providing streamlined ways to manage, combine and analyze data originating from a multitude of modalities.

Spatial image registration

Often the starting point in a multimodal analysis is the spatial image registration between the different modalities. This ensures that spatial locations in one image modality are properly mapped to those in the other(s), so that information can be readily compared between the two.
Mass spectrometry imaging is commonly coupled with high resolution microscopy, where expert pathologist annotations can guide data analysis (e.g. using Aspect’s Annotation Studio). With increasing spatial resolution of MALDI mass spectrometry imaging data, accurate registration with the microscopy data is becoming imperative to properly transfer information from one modality to the other. Likewise, other forms of mass spectrometry imaging (e.g. SIMS and CyTOF), and other imaging modalities combined with MSI such as Raman spectroscopy often sport very high spatial resolutions.
To support these multimodal workflows, Aspect has developed tools that enable the accurate image registration needed at these high spatial resolutions. A sneak peak of our current pipeline, which is available for prototyping and collaborative projects at this time, can be found below. In the near future, this pipeline will be available in our platform for general use.
If you have a project where accurate spatial registration is key, and you want to collaborate, or you want to be kept up to date, be sure to get in touch with us. If you are looking for a solution to view and annotate microscopy whole-slide images, make sure to check out Annotation Studio.

Registration of high-dimensional molecular data

To register high-dimensional molecular imaging data, we first have to create visual representations that highlight the spatial structures that are present, so we can place landmarks based on those. Our pipeline enables you to flexibly configure one or several such summary images, based on combinations of ion images and results of dimensionality reduction pipelines.
In this example, UMAP is used to summarize the information of a full MSI dataset in a single image. This approach allows for highlighting of the salient features and anatomical structures in the MSI data, enabling visual inspection and comparison between high resolution microscopy and MSI data.
Registration of microsocpy and MSI data
H&E stained whole-slide image (40x zoom)
MSI data, pre-processed with UMAP dimensionality reduction

Fine grained registration control

Using our cloud-based interactive viewer, images can be explored at full gigapixel resolution. This allows the user to finely select landmark points between multiple modalities to guide the registration process.
Annotation while zoomed out in microscopy
Partial annotation (zoomed out)
Annotation while zoomed in in microscopy
Partial annotation (zoomed in)

Accurate non-rigid registration

When registering imaging modalities measured on different tissue sections (e.g. neighboring tissue sections), these sections will often differ due to tissue shrinkage, deformation during handling or cutting artefacts, complicating the registration process.
Most image registration tools for MSI support only basic rigid image registration, which discard such soft tissue deformation effects. These effects, however, become increasingly important when measuring at high spatial resolutions, and can lead to incorrect multimodal analysis results when ignored.
Our pipeline features advanced non-rigid registration algorithms, which can handle and accommodate for these soft tissue deformation effects. The result is a highly accurate registration between modalities, fully under control of the user.
Overlay of MSI and microscopy data after registration
Overlay after registration
Overlay of MSI and microscopy data after registration (zoomed)
Overlay after registration (zoomed in)
Check out the video below to see the pipeline in action:

Multimodal integration

When registering imaging modalities measured on different tissue sections (e.g. neighboring tissue sections), these sections will often differ due to tissue shrinkage, deformation during handling or cutting artefacts, complicating the registration process.
Aspect is highly versed in multimodal imaging analysis, and can aid in developing pipelines for bioinformatics analyses for your use case. A few example cases are listed below.
overview of full data analysis pipeline including registration

Example: multimodal classification

In MSI research, MSI experiments are often coupled with microscopy images, on which an expert pathologist defines regions of interest (e.g. tumour vs healthy tissue regions). These labels are then transferred to the matching regions in the MSI data, where they can be used as labels for machine learning and classification. In a recent collaboration, Aspect developed a high-performant classification model for diagnosis of melanoma in such a setting. Aspect is currently working on tools to facilitate creation of such classification models, contact us if you would like to be kept informed on future developments.

Example: data fusion

The ultimate goal of a multimodal study is to gain deeper insights by combining the strengths of different imaging modalities and, in doing so, gaining a more holistic view of the system under study and improving our biological insights. The sheer amounts of data resulting from such multimodal studies can often be daunting, however, and drawing sensible conclusions is far from trivial.
We have experience with multimodal studies, including the integration of mass spectrometry imaging with the Allen Mouse Brain atlas and various MRI-based atlases, and using these frameworks to combine and analyze data from a multitude of different imaging modalities.