SOLID: a novel similarity metric for mono-modal and multi-modal deformable image registration

Medical image registration is an integral part of various clinical applications including image guidance, motion tracking, therapy assessment and diagnosis. We present a robust approach for mono-modal and multi-modal medical image registration. To this end, we propose the novel shape operator based local image distance (SOLID) which estimates the similarity of images by comparing their second-order curvature information. Our similarity metric is rigorously tailored to be suitable for comparing images from different medical imaging modalities or image contrasts. A critical element of our method is the extraction of local features using higher-order shape information, enabling the accurate identification and registration of smaller structures. In order to assess the efficacy of the proposed similarity metric, we have implemented a variational image registration algorithm that relies on the principle of matching the curvature information of the given images. The performance of the proposed algorithm has been evaluated against various alternative state-of-the-art variational registration algorithms. Our experiments involve mono-modal as well as multi-modal and cross-contrast co-registration tasks in a broad variety of anatomical regions. Compared to the evaluated alternative registration methods, the results indicate a very favorable accuracy, precision and robustness of the proposed SOLID method in various highly challenging registration tasks.


Introduction
The recent trend towards personalized precision medicine requires increasingly a broad spectrum of imaging modalities for diagnostics, therapy-guidance as well as response assessment.Precise and robust fusion of image information of ultrasound (US), computed tomography (CT), positron emission tomography (PET), x-ray or magnetic resonance imaging (MRI) becomes therefore of considerable importance for a growing number of clinical work-flows.
As a consequence, state-of-the-art medical image registration has to encompass not only mono-modal but also increasingly multi-modal image registration tasks: In mono-modal registration, we usually seek to align images of the same anatomy that have been obtained at different times (multi-temporal data) but with the same imaging modality and contrast.In this case, a straightforward comparison of the anatomical structures or even the gray-level intensities of the given images is possible.A prime example of such an algorithm is the optical flow algorithm suggested by Horn and Schunck (1981), which relies on the assumption of intensity conservation.A more exhaustive overview of these types of algorithms can be found in Maintz and Viergever (1998), Mani and Arivazhagan (2013) or Song et al (2017).In comparison, multi-modal image registration seeks to establish spatial correspondence between images obtained with different medical imaging modalities and/or imaging contrasts and represents therefore a significantly more challenging task.
One of the first proposed solution strategies for multi-modal image registration is mutual information (MI).Proposed by both Viola and Wells (1997) and Maes et al (1997) independently, MI-based multi-modal image registration was well received in the medical imaging community and triggered a considerable amount of followup research underlining the substantial potential, as summarized by Pluim et al (2003).An alternative approach is to convert all images into modality-independent scalar representations and to subsequently apply monomodal registration algorithms to the pre-processed datasets.Several modality-independent representations such as the local phase by Mellor and Brady (2005), local entropy by Wachinger and Navab (2012) or the gradient orientation by Haber and Modersitzki (2006) have been suggested.
A more recent approach for multi-modal DIR, which has shown considerable potential is the modality independent neighborhood descriptor (MIND) by Heinrich et al (2012), which is based on a vectorial representation of the input images, encoding the relationship of each voxel to the surrounding voxel perimeter as suggested by Buades et al (2005).One of the principal requirements is that the method hypothesizes that each anatomical structure in one image has its exact counterpart in form of a similar descriptor in the other, which is frequently in multi-modal datasets only partially fulfilled.Another family of conceptually different DIR algorithms relies on edge alignment and has been originally proposed in the field of video image registration and subsequently translated to medical imaging as suggested by Pluim et al (2000), Sun et al (2004) and Sutour et al (2015).In this approach, the underlying assumption is that boundaries and details of anatomical structures display in both types of images contrast with respect to the surrounding tissue.One of the first data-fidelity metrics to successfully implement this principle for multi-modal registration is the normalized gradient fields (NGF) (Haber and Modersitzki 2006).A more recent variation of this type of algorithm has been EVolution, which implemented an edge-based detector in a variational algorithm by de Senneville et al (2016).
Finally, a recently emerging school of thought which deviates significantly from the aforementioned ideas rests on the employment of neural networks for the task of image registration (Fu et al 2020).With convolutional neural networks being the most frequently utilized architecture so far, deep learning approaches have attracted significant research interest due to their unexplored potential.In principle, such algorithms learn a function whose input is a pair of images and whose output is the sought after deformation vector field.Bypassing the necessity of repeating an optimization scheme for each image pair, learning-based approaches allow for lower computation times.There have been both supervised (Krebs et al 2017, Yang et al 2017) and unsupervised (de Vos et al 2019) approaches presented.Despite their apparent disparity, it has been pointed out that the traditional and deep learning approaches should not be regarded as incompatible since the insights obtained from traditional algorithms and their theoretical foundation have the potential of being applied in a deep learning setup (Dalca et al 2019).In particular, recent alternative approaches to convolutional neural networks incorporated these concepts directly as a similarity metric into variational algorithms, such as for example suggested by Simonovsky et al (2016) or Krebs et al (2019) for multi-modal image registration.In this study, we return to the more traditional approach and propose a deformable image registration algorithm, which combines the variational method with a novel similarity metric, treating the input images as hypersurfaces and comparing the geometric information encoded in the shape operator as described below.
As already mentioned, various image similarity metrics have relied on gradient information to codify structural information.Gradient direction has been utilized to align images of the same or different modalities.Normalized gradient fields and EVolution are examples of registration algorithms employing this idea.In a different spirit, the vector field similarity (VFS) technique attempts to create anatomically accurate structural representations (Jaouen et al 2021).Gradient based methods, despite their overall success, can be suboptimal in anatomies dense in gradient-based features since contiguous anatomical landmarks can be mismatched leading to registration errors, particularly in the presence of large initial displacements.This reduced discriminative ability of gradient fields suggests that a metric measuring higher order information of the shapes appearing in medical images could be beneficial.Our contributions in this work can be summarized as follows: • We introduce SOLID, a novel image similarity metric that uses second order information of the given images.
• We incorporate SOLID in a variational image registration algorithm.
• We evaluate the performance of the proposed algorithm by comparing it to other state-of-the-art methods in a series of challenging mono-modal and multi-modal tasks.

Theory
In this section, we describe our data-similarity term and the optimization framework used in the proposed registration algorithm.To avoid any potential confusion coming from the introduction of various mathematical objects, we also provide a notation table 1 at the end of the section.Our method employs the notion of shape operator to measure the similarity of given images.We think of a 2D image as a surface in  3 by using the image intensity   I: 2  as a height function so that z = I(x, y).In this case, the curvature of the surface is dictated by the intensity patterns that appear the image and in particular by first and second order intensity information.In order to encode this information, we use the shape operator which is a local measure of the surface's curvature.Although the rest of our work focuses on 3D images, this intuitive 2D picture is useful.Given a 3D image with intensity function I(x, y, z), the first quantity that needs to be computed is the metric tensor or first fundamental form which is defined as follows: x z y z x y The first fundamental form encodes the first order information of a given image.The second order information is captured by the second fundamental form h ij : Our strategy encompasses the alignment of the second fundamental forms of the given 3D input images.In order to achieve this, we need to define an appropriate inner product for the (2,0)-tensors h ij .Given a vector space of m × n matrices, a standard choice for an inner product is the Frobenius product defined by where A T denotes the transpose of A and • is the standard matrix multiplication.The Frobenius inner product can be naturally generalized to (2,0)-tensors in the following way: where S g h S g h , ¢ are called the shape operators associated to h ij and h mn ¢ respectively.
The summation indices run from one to the dimension of the images (in our case 3).With those definitions being in place, we can introduce our data-fidelity term.Given a reference and moving image with corresponding second fundamental forms h ref and h mov respectively (calculated as in equation ( 2)), we define the following scalar object called the shape operator based local image distance (SOLID) where the inner products are computed as in 3 and the integrations are carried out in a neighborhood of size 2ω + 1 around the the pixel with coordinates (x,y).A value of ω = 1 is usually the optimal choice for images with high SNR.For images heavily contaminated by noise, ω = 2 provides in general more robust results.The parameter γ is introduced on computational grounds to avoid division by zero.For our purposes, we chose where the outer brackets denote averaging over the entire domain Ω and g ˜is a numerical parameter to be specified.The precise value of g ˜does not affect the efficacy of the resulting registration algorithm.We elaborate on this in our experiment 3.5.
From equation (4) follows that the proposed similarity metric is conformally invariant, namely )for arbitrary  , k l Î .This is highly desirable since in multi- modal scenarios the magnitudes of the derivatives of the reference and moving images are in general not comparable.Expressed in a more intuitive way, our ultimate goal is to align the second fundamental form tensors h ref and h mov up to an overall factor, but not to match them directly.The aforementioned theoretical arguments are local so that they are valid within individual neighborhoods.This is crucial, since assuming that the curvatures of the two (aligned) images are a multiple of each other globally would be a very strict assumption that would not hold in many cases, in particular in complex multi-modal images.
A caveat of the definition in equation ( 4) is the ambiguity in the inverse metric g ij used to compute the inner product due to the fact that the Frobenius product in equation (3) for tensors is introduced on the same surface but h ref and h mov are defined on different surfaces.However, since our goal is to align the two surfaces, we hypothesize that a common inverse metric for this computation is sufficiently precise, in particular considering that this becomes an increasingly valid approximation during the registration process as the surfaces are pulled closer to each other.In our implementation, we used the inverse metric associated to the reference image.This choice offers some computational efficacy since this object remains unchanged during the registration.The geometric interpretation of this assumption is that we view h mov as a tensor field on the surface associated to the reference image.
From the definition 4 it follows that when either the reference or moving second fundamental form is identically zero at some point, then SOLID attains its minimum value.Such points convey no information for the registration and it is solely the regularization that determines the deformation then.In this sense, homologous anatomical structures must be present in both images in order for the data fidelity to have a nontrivial contribution to the estimated deformation vector field.In this way, SOLID is designed to be robust with respect to absent correspondences in the co-registered images.
We now move on to describe our registration algorithm that incorporates the SOLID similarity metric.Due to the under-determined nature of the registration problem, the introduction of a regularization term is imperative.For that reason, we use a smoothness regularization and our full functional reads where the integrations are carried over the entire image domain Ω.Here, β is the regularization weight and T(u i ) denotes the transformation generated by the vector field (u 1 (x, y, z), u 2 (x, y, z), u 3 (x, y, z)) applied on h mov .It is important to note that we transform the components of h ij mov as if they were scalars without taking their tensorial properties into account.In principle, given a diffeomorphism f: S → S the pullback of the (2,0)-tensor h ij mov is given by where f α are the coordinates of f.Since in the end we are computing a scalar (the data fidelity term in equation ( 4)) the transformation matrices from all the covariant and contravariant indices will cancel out.However, owing to the hybrid nature of the data fidelity term in equation (4), h mov is the only term that transforms while the inverse metric and h ref remain fixed, both being computed on the reference image.Therefore, if we apply the complete transformation in equation ( 7), the data fidelity term will not transform as a scalar which is highly undesirable.This justifies the transformation of the components h ij mov as independent scalars.In summary, under the outlined assumptions, our registration problem reduces to the minimization of the functional in equation (6) with respect to the vector field u i .Requiring that 0  d = results in the following Euler-Lagrange equations where Δ is the Laplace operator.We complete the set of equations by imposing Neumann boundary conditions.
To solve equations (8) we employ the following iterative scheme introduced in Fischer and Modersitzki (2004) and effectively used in Larrey-Ruiz et al (2007): where k is the iteration index, τ is the time step, I is the identity operator, A is the matrix representation of the Laplace operator and f i k is the force term 4 Phys.Med.Biol.69 (2024) 015020 P Tzitzimpasis et al Following Larrey-Ruiz et al (2007), a discrete cosine transform (DCT) is implemented to diagonalize the Laplacian matrix A. In order to enable the estimation of larger scale motion and a faster convergence, we implemented a coarse-to-fine scheme.At each resolution level, the components of the inverse metric g ij and the fundamental forms h ref and h mov are down-sampled by a factor of 2. The number of resolution levels is determined by the requirement that no dimension becomes smaller than 16 pixels.Finally, the time step τ is automatically determined at the first iteration of each resolution level as follows This choice ensures that the term f k i t in (9) is bounded, therefore avoiding potential numerical instabilities.

Experiments
We  (Heinrich et al 2012) whose data-fidelity metric relies on the concept of self-similarity.In order to evaluate differences originating strictly from the data fidelity term, we opted to implement EVolution and MIND using the same numerical scheme as SOLID.In the following, these algorithms will be denoted as EVolution (DCT) and MIND (DCT) to stress the registration framework used.In order to enable comparison with other established methods, we have used publicly available datasets where possible.Finally, the procedure followed to calibrate the parameters of the tested algorithms can be found as supplementary material of this article.

Multi-modal (MR-CT) alignment of abdominal images
This experiment evaluates the registration of intra-patient abdominal MR and CT scans.This task is highly challenging due to the structural differences in MR and CT scans.On top of that, the human abdomen comprises a variety of organs whose position and shape are affected by respiration and other anatomical changes.However, MR to CT registration is highly relevant in clinical practice, for example in radiotherapy treatment planning.We used the 8 training image pairs from the Learn2Reg challenge (Clark et al 2013).The images have a spatial resolution of 192 × 160 × 192 and isotropic voxel size of 2 mm.For more information we refer the reader to the website. 3We measured the DSC overlap score on the liver, kidneys and spleen using the segmentations provided.We make the following remarks regarding the use of the dataset: • In case 1, the left kidney was not taken into account, due to being present only in one of the scans.
• In case 2, both kidneys and the liver are only partially visible in the MR scan, making it less useful for validation purposes.Therefore, this case was used as a 'training' dataset that we utilized to tune the parameters of the tested algorithms (see figure S1 of supplementary material).

Cross-contrast abdominal MRI images with variable FOV
In this experiment we evaluate the SOLID algorithm on the task of T1w/T2w abdominal MR image registration.
A notable difficulty of this dataset arises from the highly variable field of view between the two images.This poses a great challenge for image registration algorithms but can occur frequently in clinical imaging.Furthermore, the images were acquired at different time points, and are subject to additional misalignment due to respiratory and peristaltic motion.The liver and kidneys of all images have been segmented by medical experts and the DSC score before and after registration is used to evaluate the registration accuracy.Parameter selection for this experiment was done using one of the image pairs (see figure S2 of supplementary material).

Inter-subject alignment of mammillary bodies in neonatal brains
The mammillary bodies are small brainstem nuclei located at the posteroinferior aspect of the hypothalamus.
Inter-subject registration and accurate contour propagation of these structures can be useful in population statistics analyses (see for example Vann et al (2022)).In order to evaluate the ability of SOLID to match similar anatomical features across images of different individuals, we used 5 T1w MR brain scans of neonatal subjects.
The scans were obtained from the 'Developing human connectome project' (dHCP) database4 (Hughes et al 2017, Bastiani et al 2019, Fitzgibbon et al 2020).Due to the small size of the structures, we evaluated the registration accuracy using the Hausdorff 95 measure.Our dataset contains 5 subjects resulting in 20 registrations and 40 evaluation data points (2 mammillary bodies per registration).The original image resolution was 512 × 512 × 512 with a voxel size of 0.28 × 0.28 × 0.2 mm 3 .We resampled the images to a resolution of 256 × 256 × 256.The high resolution together with the small size of the mammillary bodies makes this dataset suitable for accuracy assessment.The parameters of the algorithms tested were optimized using a dataset of adult human brains of the same contrast (see figure S3 of supplementary material).

Pre-operative to post-operative brain registration
Registration of images with evolving pathologies and for therapy response assessment is a demanding task due to significant changes of the tissue.The existence of features that are not present in both images prohibits the use of intensity-matching algorithms unless appropriate steps are implemented, such as tumor masking where the similarity metric is not computed in the tumor region (Brett et al 2001).The brain tumor sequence registration (BraTS-Reg) challenge (Baheti et al 2021) comprises of pre-operative baseline and post-operative follow-up multi-parametric MRI scans of glioma patients.Ground truth annotations of 6-50 landmarks per patient are provided for a total of 140 patients which constitutes the challenge's training dataset.The images provided by the challenge organizers have been pre-processed with a rigid registration to a common template, resampling to a common isotropic resolution (1 × 1 × 1 mm 3 ) and skull-stripping.Out of the four MRI contrasts that are available, we chose to work with the contrast-enhanced T1-weighted (T1-CE) due to the richness in features and clinical relevance.One case (case 54) was discarded from our experiment because the images were corrupted.Algorithm configuration was done using a random single case (see figure S4 of supplementary material).In accordance with the BraTs-Reg challenge evaluation pipeline, we evaluated the tested algorithms using the median absolute error (MAE) and robustness criteria.We also included the MAE 30, namely the 30th percentile of largest MAE across all cases.Finally, for a more holistic evaluation, we report the cumulative distributions of the landmark L 1 distance.

Dependence of the results on the γ parameter
In order to assess the robustness of our proposed algorithm with respect to the choice of the g ˜parameter, we coregistered an MR/synthetic CT pair of pelvic images for a wide range of g ˜values.Being artificially generated, the CT is pre-aligned with the MR scan.The size of the images was 256 × 256 × 256 with a voxel size of 1.7 × 1.7 × 0.35 mm 3 .Since we are interested in robustness and not anatomical plausibility, we applied a synthetically generated, B-spline based deformation to the moving image and performed a series of registrations for various values of g ˜.In this way, the ground truth deformation is known and we can evaluate the target registration error over the entire anatomy.

Multi-modal (MR-CT) alignment of abdominal images
The results of the cross-modality experiment are shown in table 2. SOLID achieved a consistently optimal performance in all three organs of interest.Elastix achieved (on average) the same DSC score in the liver but performed considerably worse in the kidneys.We noticed that the most significant differences were observed in cases with small initial overlap where the large scale initial displacement makes it particularly challenging for the algorithms to match the corresponding anatomical structures.For further insight, figure 1 shows coronal slices from three cases before and after registration using the different algorithms.It can be observed that in those cases, SOLID delivers more accurate contour overlap, particularly on the lung-liver interface.

Cross-contrast abdominal MRI images with variable FOV
The DSC scores before and after registration are reported in table 3 and indicate the robustness of the proposed algorithm in this challenging task.As can be seen from the examples in figure 2, the initial misalignment is predominantly due to the variable FOV, a known and longstanding challenge in the field of image registration.
The DSC scores together with the visual information from figure 2 illustrate the high anatomical accuracy of SOLID in spite of the large initial misalignment.The gradient-based EVolution scored the second best average DSC score but was prone to significant registration errors in certain cases such as case 3 illustrated in figure 2 (last case).

Inter-subject alignment of mammillary bodies in neonatal brains
The results of this experiment are summarized in the boxplots of the HD-95, shown in figure 3. We observe that the 75th percentile is well below 1 mm for the proposed algorithm indicating a highly consistent performance.SOLID also scored the lowest in terms of the average and median HD-95.In this experiment there were no notable outliers as all algorithms were similar in performance as can be inferred from the comparable ranges of the boxplots.

Pre-operative to post-operative brain registration
Cumulative distribution functions (CDF's) of the L1 landmark distance are shown in figure 4 and the evaluation metrics used in the original BraTS-Reg challenge in table 4. The proposed algorithm scored the best in terms of Mean MAE with MIND being second best by a small margin.The robustness metric also highlights a distinction between the MI-based Elastix and the other tested algorithms, that can possibly be attributed to the sensitivity of MI to absent correspondences.To further highlight the nuances in the performance of the different algorithms tested, figure 5 shows an example of a case from the BratS-Reg dataset.

Dependence of the results on the γ parameter
The results of this last experiment are shown in figure 6.The scale on the horizontal axis is logarithmic and the range of g ˜values encompasses ten orders of magnitude.We observe that despite the large width of this range, the maximum TRE fluctuation is approximately 0.12 mm or 4.4%.This is well below the voxel size.

Discussion
In this study, we have investigated the use of second order information to locally quantify the similarity of medical images.We have attempted to include a significant amount of heterogeneous data but this was not always possible due to the limited availability of annotated data.The results obtained from our experiments, provide useful insights into the applicability of SOLID in a number of challenging registration tasks.In a brain anatomy for example, the presence of dense geometrical shapes and anatomical landmarks makes the registration problem particularly difficult.In order to expand on this point, figure 7 demonstrates the sensitivity of SOLID to changes in the shape of anatomical features inside the brain.As explained in this figure, the algorithm focuses on corners where the effect of curvature is prevalent.In the case of pre-operative to postoperative registration that we examined, the large local deformations require a similarity metric that is highly sensitive to those shape disparities and we believe that the shape operator based method proposed here can be advantageous over other methods that utilize first order information (gradients) or statistical means.This is also highlighted in figure 5.For the same experiment, the MAE of SOLID is also lower than the one reported in Mok and Chung (2022).Notably, the latter was the top performing algorithm of the BratS-Reg challenge (based on the validation cases).We believe that those are strong indications towards the favorable applicability of SOLID in   this task.The downside of utilizing higher order information of the images is that the resulting method is more sensitive to noise and often computationally expensive.
The experiments conducted in the present study indicate that the SOLID metric has ample potential to be used as a similarity metric both for mono-modal and multi-modal tasks.While various modern methods in image processing are highly specialized for certain tasks and specific types of data, the proposed approach is suitable for a broad spectrum of applications.An additional merit of the proposed method is its easy adaptation to different tasks using a very small set of tuning parameters.Indeed, as we have previously noted, the only truly unspecified parameter is the regularization weight β in (6).The integration window ω in (4) can be set to ω = 1 in most applications.The parameter g ˜was shown in section 4.5 to only affect the registration to a negligible degree.This conclusion is corroborated by all examples that we examined.On this basis, no fine tuning was required throughout the experimental process.The value that we used was 10 8 g = - ˜. Finally, concerning the computation time, the results reported in this work were obtained using our initial CPU implementation which  is computationally suboptimal, with typical registration times of a few minutes.However, the algorithm is highly parallelizable and a prototype GPU implementation that we have developed can achieve typical computation times of less than 20 seconds for registering 256 × 256 × 256 images.
Finally, it should be pointed out that although the efficacy of the proposed SOLID similarity metric has been demonstrated in the context of the presented registration algorithm, its potential application domain is much broader.For instance, we already alluded to the interplay between traditional registration algorithms and learningbased approaches.In this sense, the proposed metric could be utilized as a similarity metric for unsupervised network training.More generally, the SOLID metric provides a method for locally quantifying the similarity of mono-modal and multi-modal images and can potentially be of interest in other image processing tasks.

Conclusion
We have proposed SOLID, a novel image registration similarity metric that identifies common shapes in images and performs very favorably for challenging cases such as variable FOV, different contrast, different acquisition modality and tissue appearance changes.Using second order information, the proposed registration algorithm is also highly sensitive to local shape differences, delivering improved accuracy over the evaluated alternative methods tested.The robustness and accuracy of the proposed method has been demonstrated through a series of diverse experiments encompassing a variety of registration tasks.Considering the increasing need for greater accuracy and the constantly improving imaging techniques, we are confident that the introduction of higher order data-similarity metrics can open additional possibilities in the development of clinically relevant registration methods.et al 1996, Cocosco et al 1997, Collins et al 1998, Kwan et al 1999).The images are simulated and by construction pre-aligned.We applied a smooth local deformation in the vicinity of the lateral ventricle of the T2w image.The deformation is zero outside the region enclosed in the rectangle.In the third image, the SOLID metric is depicted with red corresponding to the maximum value of zero and blue corresponding to the minimum value of −1 as indicated in the colorbar.The magnified images demonstrate the sensitivity of the SOLID metric to the changes in the shape of anatomical features.The last image on the top row shows the area of interest in the T1w anatomy with the red points denoting the locations where the SOLID metric attains its highest values (top percentile).We observe that those points appear at the corners of geometric features, in agreement with the theoretical analysis of the shape operator as an object which encodes curvature information.
Project, KCL-Imperial-Oxford Consortium funded by the European Research Council under the European Union Seventh Framework Programme (FP/2007-2013)/ERC Grant Agreement no.[319456].We are grateful to the families who generously supported this trial.

Figure 1 .
Figure 1.Three examples of MR-CT registration.From left to right: Reference (MR) image, Moving (CT) image and registered images using the different tested algorithms.The reference (blue) and moving/registered (orange) contours of the liver, spleen and kidneys are overlaid on the anatomy.

Figure 2 .
Figure 2. The first three cases (from top to bottom) from the cross-contrast abdominal MRI dataset.For each case, coronal slices of the reference, moving and registered volumes are shown together with their overlaid counterparts (with the reference image) to demonstrate the degree of alignment before and after registration.

Figure 3 .
Figure 3. Left: Results from the inter-subject alignment of mammillary bodies.Each data point of the boxplots represents the Hausdorff 95 (HD-95) distance of a single mammillary body.The median (range) before registration was 5.64 (2.03-8.22).Those datapoints are omitted because their large values would skew the y-axis limits.We observe that SOLID achieves the lowest median of all the tested algorithms.Right: Example of a neonatal brain from the dataset.The two mammillary bodies are indicated inside the box.

Figure 4 .
Figure 4. Registration of pre-operative to post-operative T1-CE scans from the BraTS-Reg challenge.Cumulative distributions of all the landmarks L1 distance (1240 data points in total).

Figure 5 .
Figure 5. Visual comparison of registration results for a single case from the BratS-Reg dataset.The case shown (case 101) was the 'training' dataset that we used to calibrate the hyperparameters of the registration algorithms.Middle axial slices of the reference and moving volumes are shown along with the corresponding slices of the registered volumes using the four algorithms tested.The last row shows overlaid images of the reference (green) and moving/registered (red) images.An anatomical region has been been highlighted to illustrate the different local accuracy of the various algorithms.

Figure 6 .
Figure 6.The target registration error (TRE) after an MR/synthetic CT registration as a function of the g ˜parameter used.The scale of the x-axis is logarithmic and encompasses ten orders of magnitude.

Figure 7 .
Figure 7. Axial slices of a T1w and a T2w brain scan.The original 3D volumes are obtained from the Brain Web database (Kwanet al 1996, Cocosco et al 1997, Collins et al 1998, Kwan et al 1999).The images are simulated and by construction pre-aligned.We applied a smooth local deformation in the vicinity of the lateral ventricle of the T2w image.The deformation is zero outside the region enclosed in the rectangle.In the third image, the SOLID metric is depicted with red corresponding to the maximum value of zero and blue corresponding to the minimum value of −1 as indicated in the colorbar.The magnified images demonstrate the sensitivity of the SOLID metric to the changes in the shape of anatomical features.The last image on the top row shows the area of interest in the T1w anatomy with the red points denoting the locations where the SOLID metric attains its highest values (top percentile).We observe that those points appear at the corners of geometric features, in agreement with the theoretical analysis of the shape operator as an object which encodes curvature information.

Table 1 .
Notation table.Numerical gradient of the data similarity term with respect to the ith component of the vector field and evaluated at the kth iteration g ˜Numerical stability factor that we set to 10 −8 γ Numerical stability factor that is determined from g ˜and image information β Smoothness regularization weight τ Time step used in our DCT optimizer (automatically computed) 〈A, B〉Frobenius product of the tensor fields A, B on a curved space with metric g ij T(u i ) Spatial transformation associated to the vector field with components u i A Matrix of eigenvalues of the Laplace operator have performed a number of experiments to evaluate the performance of the proposed algorithm.Our experiments are tailored towards highly challenging and clinically relevant applications.For comparison, other widely used algorithms have been subjected to the same tests.These are the publicly available Elastix (Klein et al 2009, Shamonin et al 2014) using B-splines and a mutual information based similarity metric, the gradientbased EVolution (de Senneville et al 2016) and MIND

Table 2 .
DSC scores on liver, kidneys and spleen for the abdominal MR/CT registrations.Each entry shows the mean (sd) of the DSC scores across the seven different cases.

Table 3 .
DSC results on liver and kidneys for the abdominal T1/T2 registration test.

Table 4 .
Summary of pre-operative to post-operative registration task.The table shows the median absolute error (in mm), robustness metric R and MAE 30 (in mm) for the cases of the BraTS-Reg challenge training dataset.SOLID achieved the lowest mean MAE and the lowest standard deviation.