## **Mathematics and Visualization**

Evren Özarslan · Thomas Schultz · Eugene Zhang · Andrea Fuster Editors

# Anisotropy Across Fields and Scales

## Mathematics and Visualization

#### Series Editors

Hans-Christian Hege, Konrad-Zuse-Zentrum für Informationstechnik Berlin (ZIB), Berlin, Germany

David Hoffman, Department of Mathematics, Stanford University, Stanford, CA, USA

Christopher R. Johnson, Scientific Computing and Imaging Institute, Salt Lake City, UT, USA

Konrad Polthier, AG Mathematical Geometry Processing, Freie Universität Berlin, Berlin, Germany

The series Mathematics and Visualization is intended to further the fruitful relationship between mathematics and visualization. It covers applications of visualization techniques in mathematics, as well as mathematical theory and methods that are used for visualization. In particular, it emphasizes visualization in geometry, topology, and dynamical systems; geometric algorithms; visualization algorithms; visualization environments; computer aided geometric design; computational geometry; image processing; information visualization; and scientific visualization. Three types of books will appear in the series: research monographs, graduate textbooks, and conference proceedings.

More information about this series at http://www.springer.com/series/4562

Evren Özarslan • Thomas Schultz • Eugene Zhang • Andrea Fuster Editors

## Anisotropy Across Fields and Scales

Editors Evren Özarslan Linköping University Linköping, Sweden

Eugene Zhang Oregon State University Corvallis, OR, USA

Thomas Schultz University of Bonn Bonn, Germany

Andrea Fuster Eindhoven University of Technology Eindhoven, The Netherlands

ISSN 1612-3786 ISSN 2197-666X (electronic) Mathematics and Visualization ISBN 978-3-030-56214-4 ISBN 978-3-030-56215-1 (eBook) https://doi.org/10.1007/978-3-030-56215-1

Mathematics Subject Classification: 68-06, 15A69, 68U10, 92C55, 74E10

© The Editor(s) (if applicable) and The Author(s) 2021. This book is an open access publication. Open Access This book is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this book are included in the book's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the book's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Cover figure by Jochen Jankowai, Talha bin Masood, and Ingrid Hotz.

This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

## Preface

The creation of this book, the seventh in the series, started like the earlier six books. After organizing the weeklong Dagstuhl workshop "Visualization and Processing of Anisotropy in Imaging, Geometry, and Astronomy" (October 28–November 2, 2018, Dagstuhl, Germany), we decided to develop a book that would share with our readers many of the wonderful research ideas and results reported at the workshop or inspired by intriguing discussions at Dagstuhl. After a call for participation, we received contributions from the attendees of the workshop as well as from other researchers.

The central topic of this book is anisotropy. Objects, processes, and phenomena that exhibit variations along different directions are omnipresent in science, engineering, and medicine. The ability to analyze, model, and physically measure such anisotropy in each of its application areas is a common theme that resonates with mathematicians, engineers, scientists, and medical researchers. Accordingly, we divide the thirteen chapters of our book into four coherent parts.

Our book starts with three chapters that constitute Part I, whose theme is Foundations. The first chapter "Variance Measures for Symmetric Positive (Semi-) Definite Tensors in Two Dimensions" considers fourth-order tensors that represent the covariance of distributions of second-order tensors in two dimensions and have the same symmetries as the elasticity tensor. A set of invariants is introduced, which guarantees the equivalence of two such fourth-order tensors under coordinate transformations. Chapter "Degenerate Curve Bifurcations in 3D Linear Symmetric Tensor Fields" investigates fundamental bifurcations in three-dimensional linear symmetric tensor fields, with potential applications in the study of time-varying tensor fields with multi-scale topological analysis. The last chapter of Part I "Continuous Histograms for Anisotropy of 2D Symmetric Piece-Wise Linear Tensor Fields" proposes a method to compute iso-contours and continuous histograms of the anisotropy of 2D tensor fields, using component-wise tensor interpolation. The authors show that the presented technique leads to accurate anisotropy histograms. This chapter kindly provides the image on the book cover.

The second part of the book contains four chapters on image processing and visualization of different types of data. Chapter "Tensor Approximation for Multidimensional and Multivariate Data" surveys the topic of data approximation in computer graphics and visualization, focusing on tensor approximation (TA) methods for multidimensional datasets. In addition, it studies how applying TA to vector fields affects important properties such as magnitudes, angles, vorticity, and divergence.

In image processing, anisotropic models can improve results by accounting for the orientation of image structures, such as object boundaries. Chapter "Fourth-Order Anisotropic Diffusion for Inpainting and Image Compression" introduces such a model, a fourth-order partial differential equation that involves a fourth-order diffusion tensor, generalizing anisotropic edge-enhancing diffusion. This model is applied to repair damaged images and to reconstruct full images from a sparse subset of pixels, which can serve as a foundation of image compression.

Chapters "Uncertainty in the DTI Visualization Pipeline" and "Challenges for Tractogram Filtering" treat challenges in the processing and visualization of diffusion magnetic resonance imaging (dMRI) data and are the first chapters on MRI techniques to which the rest of the book is devoted. Chapter "Uncertainty in the DTI Visualization Pipeline" places emphasis on diffusion tensor imaging (DTI) and reviews the origins of uncertainty as well as the techniques developed for modeling and visualizing uncertainty in DTI.

Upon adequate processing, anisotropy information can shed light on important open problems. One such example, which has enjoyed a great deal of interest in recent years, involves exploiting the anisotropy revealed by diffusion MRI to map neural tracts, i.e., the white matter wiring of the brain. Recent studies have shown that existing tractography methods suffer from artifactual connections. Chapter "Challenges for Tractogram Filtering" reviews methods developed to filter out such connections and discusses the associated challenges in this endeavor.

Part III of this book is devoted to the mathematical modeling of anisotropy, and the fitting of such models to measured data. It starts with chapter "Single Encoding Diffusion MRI: A Probe to Brain Anisotropy", which surveys the state of the art in modeling diffusion anisotropy within the human brain, as measured by traditionally-encoded diffusion MRI featuring one pair of diffusion gradient pulses. It provides a broad overview, discussing aspects of neural tissue structure, mathematical representations of the measured signal, and biophysical models and challenges in the reliable estimation of their parameters.

It is well known that MR images are sensitive to ensemble-averaged molecular displacements, and a concrete interpretation of diffusion MRI data in terms of physical or structural parameters is challenging. Chapter "Conceptual Parallels Between Stochastic Geometry and Diffusion-Weighted MRI" sheds light on this problem by drawing a parallel of stochastic geometry, a concept that has found much success in geology, astronomy, and communications. The authors review important results from stochastic geometry and hypothesize how these could be useful for a more robust modeling of MRI data.

Many specimens of interest comprise a distribution of microscopic, individually anisotropic subdomains. An earlier work has shown that diffusion taking place within each of such subdomains can be equivalently modeled by envisioning diffusion to be taking place under a Hookean restoring force. The averaging of anisotropic signal, either numerically, or naturally due to the presence of randomly aligned pores, results in interesting residual features of the diffusion MRI signal that are informative of the underlying microstructure. Chapter "Magnetic Resonance Assessment of Effective Confinement Anisotropy with Orientationally-Averaged Single and Double Diffusion Encoding" investigates these questions for diffusion-encoding schemes featuring one as well as two pairs of pulses.

The last chapter of Part III, chapter "Riemann-DTI Geodesic Tractography Revisited", addresses again the open problem of mapping neural tracts from diffusion MRI, now from a data modeling point of view. The authors propose a new geodesic tractography paradigm by coupling the diffusion tensor to a family of Riemannian metrics, governed by control parameters. The optimal controls, and corresponding tentative tracts, show a good correspondence with tracts on simulated data.

Finally, Part IV comprises two chapters which are primarily concerned with the measurement of anisotropy using MRI. Chapter "Magnetic Resonance Imaging of T2- and Diffusion Anisotropy Using a Tiltable Receive Coil" combines the now well-established measurement of diffusion anisotropy with a quantification of directionally-dependent transverse relaxation rates, which provide complementary information on tissue microstructure. Protocols for reliable measurement of the latter are a topic of ongoing research, and this chapter presents results obtained by using a tiltable receive coil.

Finally, chapter "Anisotropy in the Human Placenta in Pregnancies Complicated by Fetal Growth Restriction" reports experimental results from measuring diffusion anisotropy in the human placenta, comparing pregnancies complicated by fetal growth restriction with normal controls. Results suggest that diffusion MRI, otherwise primarily used as a neuroimaging technique, can also provide valuable information about placental microstructure and could thus help assess placental function during pregnancy.

As you read this book, we hope that you not only enjoy it for its scientific merit but also see it perhaps as a source of inspiration. During the review process, the COVID-19 pandemic started and is still ongoing at this moment. The people involved in the production of this book (authors, reviewers, and editors), many of whom are university professors or students, had to work from home due to the need for social distancing. In-person meetings have been replaced with online discussions. For people who are parents of young children, it has been even more difficult due to the added tasks of babysitting and/or homeschooling their children. Despite all these challenges as well as the constant worry of contracting the virus and the stress associated with social distancing, our reviewers strived to honor their commitment to finish the reviews on time and provided high-quality, constructive reviews that have made each one of the chapters stronger. Similarly, our contributing authors were diligent in the revision of their work, ensuring a timely delivery of this book to the publisher. We wish to express our gratitude to them, for not only making this book possible, but also making it possible during this difficult period.

Last but not least, we would like to thank the editors of the Springer book series Mathematics and Visualization, as well as Martin Peters and Leonie Kunz (Springer, Heidelberg) for their support to publish this book, as well as the board and staff of Schloss Dagstuhl, for their excellent support in organizing the workshop. Dagstuhl once again created an enjoyable atmosphere for open interdisciplinary exchange between researchers from different fields. Without this unique setting, many participants most likely never would have had the opportunity to engage with each other's work. Finally, we would like to thank the Department of Mathematics and Computer Science of Eindhoven University of Technology for making it possible to publish this book open access.

Linköping, Sweden Evren Özarslan Bonn, Germany Thomas Schultz Corvallis, Oregon, USA Eugene Zhang Eindhoven, The Netherlands July 2020

Andrea Fuster

## Contents

#### Foundations




Mary Rutherford, and Jana Hutter


## **Foundations**

## **Variance Measures for Symmetric Positive (Semi-) Definite Tensors in Two Dimensions**

**Magnus Herberthson, Evren Özarslan, and Carl-Fredrik Westin**

**Abstract** Calculating the variance of a family of tensors, each represented by a symmetric positive semi-definite second order tensor/matrix, involves the formation of a fourth order tensor *Rabcd* . To form this tensor, the tensor product of each second order tensor with itself is formed, and these products are then summed, giving the tensor *Rabcd* the same symmetry properties as the elasticity tensor in continuum mechanics. This tensor has been studied with respect to many properties: representations, invariants, decomposition, the equivalence problem et cetera. In this paper we focus on the two-dimensional case where we give a set of invariants which ensures equivalence of two such fourth order tensors *Rabcd* and *R abcd* . In terms of components, such an equivalence means that components *Rijkl* of the first tensor will transform into the components *R ijkl* of the second tensor for some change of the coordinate system.

#### **1 Introduction**

Positive semi-definite second order tensors arise in several applications. For instance, in image processing, a structure tensor is computed from greyscale images that captures the local orientation of the image intensity variations [10, 17] and is employed to address a broad range of challenges. Diffusion tensor magnetic resonance imaging (DT-MRI) [1, 5] characterizes anisotropic water diffusion by enabling the measurement of the apparent diffusion tensor, which makes it possible to delineate the fibrous structure of the tissue. Recent work has shown that diffusion MR measurements of

M. Herberthson (B)

E. Özarslan

#### C.-F. Westin Department of Radiology Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA e-mail: westin@bwh.harvard.edu

Department of Mathematics, Linköping University, Linköping, Sweden e-mail: magnus.herberthson@liu.se

Department of Biomedical Engineering, Linköping University, Linköping, Sweden e-mail: evren.ozarslan@liu.se

restricted diffusion obscures the fine details of the pore shape under certain experimental conditions [11], and all remaining features can be encoded accurately by a confinement tensor [19].

All such second order tensors share the same mathematical properties, namely, they are real-valued, symmetric, and positive semi-definite. Moreover, in these disciplines, one encounters a collection of such tensors, e.g., at different locations of the image. Populations of such tensors have also been key to some studies aiming to model the underlying structure of the medium under investigation [8, 12, 18].

Irrespective of the particular application, let *Rab* denote such tensors,1 and we shall refer to the set of *<sup>n</sup>* tensors as {*R*(*i*) *ab* }*<sup>i</sup>* . Our desire is to find relevant descriptors or models of such a family. One relevant statistical measure of this family is the (population) variance *cd* 

$$\frac{1}{n}\sum\_{i=1}^{n}(\boldsymbol{R}\_{ab}^{(i)} - \widehat{\boldsymbol{R}}\_{ab})(\boldsymbol{R}\_{cd}^{(i)} - \widehat{\boldsymbol{R}}\_{cd}) = \left(\frac{1}{n}\sum\_{i=1}^{n}\boldsymbol{R}\_{ab}^{(i)}\boldsymbol{R}\_{cd}^{(i)}\right) - \widehat{\boldsymbol{R}}\_{ab}\widehat{\boldsymbol{R}}\_{cd}\ ,\ \boldsymbol{R}$$

where *R ab* <sup>=</sup> <sup>1</sup> *n n <sup>i</sup>*=<sup>1</sup> *<sup>R</sup>*(*i*) *ab* is the mean. (For another approach, see e.g., [8]). In this paper, we are interested in the first term, i.e., we study the fourth order tensor (skipping the normalization) *Rabcd* = *n*

$$\mathcal{R}\_{abcd} = \sum\_{i=1}^{n} \mathcal{R}\_{ab}^{(i)} \mathcal{R}\_{cd}^{(i)}, \quad \mathcal{R}\_{ab}^{(i)} \ge 0,\tag{1}$$

where *R*(*i*) *ab* <sup>≥</sup> 0 stands for *<sup>R</sup>*(*i*) *ab* being positive semi-definite. It is obvious that *Rabcd* has the symmetries *Rabcd* = *Rbacd* = *Rabdc* and *Rabcd* = *Rcdab*, i.e., *Rabcd* has the same symmetries as the elasticity tensor [14] from continuum mechanics. The elasticity tensor is well studied [13], e.g. with respect to classification, decompositions, and invariants. In most cases this is done in three dimensions. The same (w.r.t. symmetries) tensor has also been studied in the context of diffusion MR [2].

In this paper we will focus on the corresponding tensor *Rabcd* in two dimensions. First, there are direct applications in image processing, and secondly, the problems posed will be more accessible in two dimensions than in three. In particular we study the equivalence problem, namely, we ask the question: given the components *Rijkl* and *R ijkl* of two such tensors do they represent the same tensor in different coordinate systems (see Sects. 2.1.2 and 4)?

#### *1.1 Outline*

Section 2 contains tensorial matters. We will assume some basic knowledge of tensors, although some definitions are given for completeness. The notation(s) used is

<sup>1</sup>For the notation of tensors used here, see Sect. 2.1.

commented on and in particular the three-dimensional Euclidean vector space *V*(*ab*) is introduced.

In Sect. 2.1.2, we make some general remarks concerning the tensor *Rabcd* and specify the problem we focus on. Section 2.1 is concluded with some remarks on the Voigt/Kelvin notation and the corresponding visualisation in R3.

Section 2.2 gives examples of invariants, especially invariants which are easily accessible from *Rabcd* . Also, more general invariant/canonical decompositions of *Rabcd* are given.

In Sect. 3, we discuss how the tensor *Rabcd* can (given a careful choice of basis) be expressed in terms of a 3 × 3 matrix, and how this matrix is affected by a rotation of the coordinate system in the underlying two-dimensional space on which *Rabcd* is defined.

In Sect. 4 we return to the equivalence problem and give the main result of this work. In Sect. 4.1.1 we provide a geometric condition for equivalence, while in Sect. 4.1.2, we present the equivalence in terms of a 3 × 3 matrix. Both these characterisations rely on the choice of particular basis elements for the vector spaces employed. In Sect. 4.1.3 the same equivalence conditions are given in a form which does not assume a particular basis.

#### **2 Preliminaries**

In this section we clarify the notation and some concepts which we need. Section 2.1 deals with the (alternatives of) tensor notation and some representations. The equivalence (and related) problems are also briefly addressed. Section 2.2 accounts for some natural invariants, traces and decompositions of *Rabcd* . 

We will assume some familiarity with tensors, but to clarify the view on tensors we recall some facts. We start with a (finite dimensional) vector space *V* with dual *V*∗. A tensor of order (p,q) is then a multi-linear mapping *<sup>V</sup>* <sup>×</sup> *<sup>V</sup>* ···× *<sup>V</sup>* × *V*<sup>∗</sup> ×···× *V*<sup>∗</sup> 

*q p* <sup>→</sup> <sup>R</sup>. Moreover, a (non-degenerate) metric/scalar product *<sup>g</sup>* : *<sup>V</sup>* <sup>×</sup> *<sup>V</sup>* <sup>→</sup> <sup>R</sup> gives an isomorphism from *V* to *V*<sup>∗</sup> through *v* → *g*(*v*, ·), and it is this isomorphism which is used to 'raise and lower indices', see below. Indeed, for a fixed *v* ∈ *V*, *g*(*v*, ·) is a linear mapping *<sup>V</sup>* <sup>→</sup> <sup>R</sup>, i.e., an element of *<sup>V</sup>*∗.

#### *2.1 Tensor Notation and Representations*

There is a plethora of notations for tensors. Here, we follow the well-adopted convention [16] that early lower case Latin letters (*T <sup>a</sup> bc*) refer to the tensor as a geometric object, its type being inferred from the indices and their positions (the abstract index notation). *gab* denotes the metric tensor. When the indices are lower case Latin letters from the middle of the alphabet, *T <sup>i</sup> jk* , they refer to components of *T <sup>a</sup> bc* in a certain frame. The super-index *i* denotes a contravariant index while the sub-indices *j*, *k* are covariant. For instance, a typical vector (tensor of type (1, 0)) will be written *v<sup>a</sup>* with components *v<sup>i</sup>* , while the metric *gab* (tensor of type (0, 2)) has components *gi j* . At a number of occasions, it will also be useful to express quantities in terms of components with respect to orthonormal frames, i.e., Cartesian coordinates. This is sometimes referred to as 'Cartesian tensors', and the distinction between contra- and covariant indices is obscured. In these situations, it is possible (but not necessary) to write all indices as sub-indices, and sometimes the symbol **·** = is used to indicate that an equation is only valid in Cartesian coordinates. For example *Ti* **·** = *Tijk* δ *jk* instead of *T <sup>i</sup>* = *T <sup>i</sup> jkg jk* = *T ik <sup>k</sup>* . Often this is clear form the context, but we will sometimes use **·** = to remind the reader that a Cartesian assumption is made. Here, the Einstein summation convention is implied, i.e., repeated indices are to be summed over, so that for instance *T <sup>i</sup>* = *T <sup>i</sup> jkg jk* = *T ik k* = *n j*=1 *n k*=1 *T i jkg jk* = *n k*=1 *T ik <sup>k</sup>* if each index

ranges from 1 to *n*. We have also used the metric *gi j* and its inverse *gi j* to raise and lower indices. For instance, since *gi j v<sup>i</sup>* is an element of *V*∗, we write *gi j v<sup>i</sup>* = *v <sup>j</sup>* . 

We also remind of the notation for symmetrisation. For a two-tensor *<sup>T</sup>*(*ab*) <sup>=</sup> <sup>1</sup> <sup>2</sup> (*Tab* + *Tba*), while more generally for a tensor *Ta*1*a*2···*an* of order (0, *n*) we have

$$T\_{(a\_1 a\_2 \cdots a\_n)} = \frac{1}{n!} \sum\_{\pi} T\_{a\_{\pi(1)} a\_{\pi(2)} \cdots a\_{\pi(n)}}$$

where the sum is taken over all permutations π of 1, 2,..., *n*. Naturally, this convention can also be applied to subsets of indices. For instance, *Ha*(*bc*) <sup>=</sup> <sup>1</sup> <sup>2</sup> (*Habc* + *Hacb*).

#### **2.1.1 The Vector Space of Symmetric Two-Tensors**

In any coordinate frame a symmetric tensor *Rab* (i.e., *Rab* = *Rba*) is represented by a symmetric matrix *Ri j* (2 × 2 or 3 × 3 depending on the dimension of the underlying space). In the two-dimensional case, with the underlying vector space *<sup>V</sup><sup>a</sup>* <sup>∼</sup> <sup>R</sup><sup>2</sup> , this means that *Rab* lives in a three-dimensional vector space, which we denote by *V*(*ab*). *V*(*ab*) is equipped with a natural scalar product: < *Aab*, *Bab* >= *AabBab*, making it into a three-dimensional Euclidean space. Here *AabBab* = *AabBcd gacgbd* , i.e, the contraction of *AabBcd* over the indices *a*, *c* and *b*, *d*, and the tensor product *AabBcd* itself is the tensor of order (0, 4) given by (*AabBcd* )*vaub*w*cmd* = (*Aabvaub*)(*Bcd*w*cmd* ) together with multi-linearity.

#### **2.1.2 The Tensor** *Rabcd* **and the Equivalence Problem**

As noted above, *Rabcd* given by (1) has the symmetries *Rabcd* = *R*(*ab*)*cd* = *Rab*(*cd*) and *Rabcd* = *Rcdab*, and it is not hard to see that this gives *Rabcd* six degrees of freedom in two dimensions. (See also Sect. 2.1.3.) It is also interesting to note that *Rabcd* provides a mapping *V*(*ab*) → *V*(*ab*) through

$$
\mathcal{R}\_{ab} \mapsto \mathcal{R}\_{abcd} \mathcal{R}^{cd},
$$

and that this mapping is symmetric (due to the symmetry *Rabcd* = *Rcdab*). Given *Rabcd* there are a number of questions one can ask, e.g.,


We will now focus on the *equivalence problem* in two dimensions. This problem can be formulated as above: given, in terms of components, two tensors (with the symmetries we consider) *Rijkl* and *R ijkl* , do they represent the same tensor in the sense that there is a coordinate transformation taking the components *Rijkl* into the components *R ijkl*? In other words, does there exist an (invertible) matrix *Pmi* so that --

$$R\_{ijkl} = \bar{R}\_{mnp} P^m{}\_i P^n{}\_j P^o{}\_k P^p{}\_l?$$

This problem can also be formulated when *Rijkl* and *R ijkl* are expressed in Cartesian frames. Then the coordinate transformation must be a rotation, i.e., given by a rotation matrix *Q<sup>i</sup> <sup>j</sup>* ∈ SO(2). Hence, the problem of (unitary) equivalence is: Given *Rijkl* and *R ijkl* , both expressed in Cartesian frames, is there a matrix (applying the 'Cartesian convention') *Qi j* ∈ SO(2) so that -

$$R\_{ijkl} = \ddot{R}\_{mnop} \mathcal{Q}\_{mi} \mathcal{Q}\_{nj} \mathcal{Q}\_{ok} \mathcal{Q}\_{pl}?$$

#### **2.1.3 The Voigt/Kelvin Notation**

Since (in two dimensions) the space *V*(*ab*) is three-dimensional, one can introduce coordinates, for example *Ri j* <sup>=</sup> *x y y z* ∼ *x y z* and use vector algebra on R3. This is used in the Voigt notation [15] and the related Kelvin notation [6]. As always, one must be careful to specify with respect to which basis in *<sup>V</sup>*(*ab*) the coordinates *<sup>x</sup> y z* are taken. For instance, in the correspondence *Ri j* <sup>=</sup> *x y y z* ∼ *x y z* , the understood basis for *V*(*ab*) (in the understood/induced coordinate system) is { 1 0 0 0 , 0 1 1 0 , 0 0 0 1 }.

**Fig. 1** Left: the symmetric matrices *e* (1) *ab* , *e* (2) *ab* , *e* (3) *ab* (red) and *e* (1) *ab* + *e* (3) *ab* , *e* (2) *ab* + *e* (3) *ab* (blue) as vectors in R3. The positive semi-definite matrices correspond to vectors which are inside/above the indicated cone (including the boundary). Right: the fourth order tensors (*e* (1) *ab* + *e* (3) *ab* )(*e* (1) *cd* + *e* (3) *cd* ) and (*e* (2) *ab* + *e* (3) *ab* )(*e* (2) *cd* + *e* (3) *cd* ) depicted in blue, and *e* (3) *ab e* (3) *cd* shown in red are viewed as quadratic forms and illustrated as ellipsoids (made a bit 'fatter' than they should be for the sake of clarity)

These elements are orthogonal (viewed as vectors in *V*(*ab*)) to each other, but not (all of them) of unit length.

Since the unit matrix plays a special role, we make the following choice. Starting with an orthonormal basis {ˆ *<sup>ξ</sup>* , <sup>η</sup>ˆ} for *<sup>V</sup>*, (i.e., { <sup>ˆ</sup>*<sup>ξ</sup> <sup>a</sup>* , ηˆ*a*} for *V<sup>a</sup>*) a suitable orthonormal basis for *V*(*ab*) is {*e* (1) *ab* , *e* (2) *ab* , *e* (3) *ab* } where *e* (1) *ab* = √ 1 <sup>2</sup> (ξ*a*ξ*<sup>b</sup>* − η*a*η*b*), *e* (2) *ab* = √ 1 <sup>2</sup> (ξ*a*η*<sup>b</sup>* + η*a*ξ*b*), *e* (3) *ab* = √ 1 <sup>2</sup> (ξ*a*ξ*<sup>b</sup>* + η*a*η*b*), i.e., in the induced basis we have 1 0 0 1

$$e\_{ij}^{(1)} = \frac{1}{\sqrt{2}} \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix} \sim \hat{\mathbf{x}}, \quad e\_{ij}^{(2)} = \frac{1}{\sqrt{2}} \begin{pmatrix} 0 \ 1 \\ 1 \ 0 \end{pmatrix} \sim \hat{\mathbf{y}}, \quad e\_{ij}^{(3)} = \frac{1}{\sqrt{2}} \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \sim \hat{\mathbf{z}}. \tag{2}$$
 
$$\text{In this basis, we write an arbitrary element } M\_{ab} \in V\_{(ab)} \text{ as } M\_{ij} = \begin{pmatrix} z^{z\_x} & \stackrel{y}{z} \\ \vdots & \vdots \end{pmatrix} \text{, which is }$$

*y z*−*x* , which means that *Mab* gets the coordinates *Mi* <sup>=</sup> <sup>√</sup><sup>2</sup> *x y z* . Note that *Mi j* is positive definite if *z*<sup>2</sup> − *x* <sup>2</sup> − *y*<sup>2</sup> ≥ 0 and *z* ≥ 0. In terms of the coordinates of the Voigt notation, the tensor *Rabcd* corresponds to a symmetric mapping <sup>R</sup><sup>3</sup> <sup>→</sup> <sup>R</sup>3, given by a symmetric 3 × 3 matrix, which also shows that the degrees of freedom for *Rabcd* is six.

#### **2.1.4 Visualization in** R**<sup>3</sup>**

Through the Voigt notation, any symmetric two-tensor (in two dimensions) can be visualised as a vector in R3. Using the basis vector given by (2), we note that *e* (1) *i j* and *e* (2) *i j* correspond to indefinite quadratic forms, while *e* (3) *i j* is positive definite. We also see that *e* (1) *i j* + *e* (3) *i j* and *e* (2) *i j* + *e* (3) *i j* are positive semi-definite.

In Fig. 1 (left) these matrices are illustrated as vectors in R3. The set of positive semi-definite matrices corresponds to a cone, cf. [4], indicated in blue. When the symmetric 2 <sup>×</sup> 2 matrices are viewed as vectors in <sup>R</sup>3, the outer product of such a vector with itself gives a symmetric 3 × 3 matrix. Hence we get a positive semidefinite quadratic form on R3, which can be illustrated by an (degenerate) ellipsoid in R3. In Fig. 1 (right) (*e* (1) *ab* + *e* (3) *ab* )(*e* (1) *cd* + *e* (3) *cd* ), (*e* (2) *ab* + *e* (3) *ab* )(*e* (2) *cd* + *e* (3) *cd* ) and *e* (3) *ab e* (3) *cd* are visualised in this manner. Note that all these quadratic forms correspond to matrices which are rank one. (Cf. the ellipsoids in Fig. 2.)

#### *2.2 Invariants, Traces and Decompositions*

By an invariant, we mean a quantity that can be calculated from measurements, and which is independent of the frame/coordinate system with respect to which the measurements are performed, despite the fact that components, e.g., *T <sup>i</sup> jk* themselves depend on the coordinate system. It is this property that makes invariants important, and typically they are formed via tensor products and contractions, e.g., *T <sup>i</sup> jkT <sup>k</sup> ilg jl* . Sometimes, the invariants have a direct geometrical meaning. For instance, for a vector *v<sup>i</sup>* , the most natural invariant is its squared length *v<sup>i</sup> vi* . For a tensor *T <sup>i</sup> <sup>j</sup>* of order (1,1) in three dimensions, viewed as a linear mapping <sup>R</sup><sup>3</sup> <sup>→</sup> <sup>R</sup>3, the most well known invariants are perhaps the trace *T <sup>i</sup> <sup>i</sup>* and the determinant det(*T <sup>i</sup> <sup>j</sup>*). The modulus of the determinant gives the volume scaling under the mapping given by *T i <sup>j</sup>* , while the trace equals the sum of the eigenvalues. If *T <sup>i</sup> <sup>j</sup>* represents a rotation matrix, then its trace is 1 + 2 cos φ, where φ is the rotation angle. In general, however, the interpretation of a given invariant may be obscure. (For an account relevant to image processing, see e.g., [9]. A different, but relevant, approach in the field of diffusion MRI is found in [20].)

#### **2.2.1 Natural Traces and Invariants**

From (1), and considering the symmetries of *Rabcd* , two (and only two) natural traces arise. For a tensor of order (1, 1), e.g., *Ri j* , it is natural to consider this as an ordinary matrix, and consequently use stem letters without any indices at all. To indicate this slight deviation from the standard tensor notation, we denote e.g., *Ri <sup>j</sup>* by ¯ *R*¯. Using [·] for the trace, so that [ ¯ *R*¯] = Tr( ¯ *R*¯) = *Ra <sup>a</sup>*, we then have *Tab* <sup>=</sup> *Rabc<sup>c</sup>* <sup>=</sup> *<sup>n</sup> c* = *n*

$$
\Gamma\_{ab} = \mathcal{R}\_{abc}{}^c = \sum\_{i=1}^n \mathcal{R}\_{ab}^{(i)} \mathcal{R}\_c^{(i)} = \sum\_{i=1}^n \mathcal{R}\_{ab}^{(i)} [\bar{\tilde{\mathcal{R}}}^{(i)}], \tag{3}
$$

$$
S\_{ab} = \mathcal{R}\_{acb}{}^c = \sum\_{i=1}^n \mathcal{R}\_{ac}^{(i)} \mathcal{R}\_b^{(i)}. \tag{4}
$$

and

$$S\_{ab} = R\_{acb}{}^c = \sum\_{i=1}^n R\_{ac}^{(i)} R\_b^{(i)} \,. \tag{4}$$

Hence, in a Cartesian frame, where the index position is unimportant, we have for the matrices ¯ *T*¯ = *Ti j*, ¯ *S*¯ = *Si j T*¯ = *n S*¯ = *n*

$$\bar{\bar{T}} = \sum\_{i=1}^{n} \bar{\bar{R}}^{(i)} [\bar{\bar{R}}^{(i)}], \quad \bar{\bar{\bar{S}}} = \sum\_{i=1}^{n} \bar{\bar{R}}^{(i)} \bar{\bar{R}}^{(i)}.$$

To proceed there are two double traces (i.e., contracting *Rabcd* twice):

$$l = 1 \qquad \qquad \qquad \qquad \qquad l = 1$$
 
$$\text{are two double traces (i.e., contracting } \mathcal{R}\_{abcd} \text{ twice):}$$
 
$$T = T\_a^{\
u} = \mathcal{R}\_a^{\
u}{}\_c^c = \sum\_{i=1}^n \mathcal{R}\_a^{(i)}{}^a \mathcal{R}\_c^{(i)}{}^c = \sum\_{i=1}^n [\bar{\mathcal{R}}^{(i)}]^2 \tag{5}$$

and

$$
\Gamma = T\_a^{\
u} = R\_a^{\
u}{}^c = \sum\_{i=1} R\_a^{\prime \prime} R\_c^{\prime \prime} = \sum\_{i=1} [R^{\prime \prime}]^- \tag{5}
$$

$$
S = S\_a^{\
u} = R\_{ac}^{\
u} = \sum\_{i=1}^n R\_{ac}^{(i)} R^{(i)}^{ac} = \sum\_{i=1}^n [(\bar{\tilde{R}}^{(i)})^2]. \tag{6}
$$

In two dimensions, the difference *Tab*−*Sab* is proportional to the metric *gab*. Namely,

**Lemma 1** *With Tab and Sab given by* (3) *and* (4)*, it holds that (in two dimensions)*

$$\begin{aligned} \text{\_{ab}\text{ given by (3) and (4), it hold} \\\\ T\_{ab} - S\_{ab} = \sum\_{i=1}^{n} \det(\bar{\bar{\tilde{R}}}^{(i)}) \text{g}\_{ab} \dots \end{aligned}$$

*Proof* By linearity, it is enough to prove the statement when *n* = 1, i.e., when the sum has just one term. Raising the second index, and using components, the statement then is *Ti <sup>j</sup>* <sup>−</sup> *Si <sup>j</sup>* <sup>=</sup> det( ¯ *R*¯(1) )δ*i j* . Putting ¯ *R*¯(1) = *A*, we see that *Ti <sup>j</sup>* <sup>−</sup> *Si <sup>j</sup>* <sup>=</sup> *<sup>A</sup>*[*A*] − *<sup>A</sup>*<sup>2</sup> while det( ¯ *R*¯(1) )δ*i <sup>j</sup>* <sup>=</sup> det(*A*)*I*, and by the Cayley-Hamilton theorem in two dimensions, *A*[*A*] − *A*<sup>2</sup> is indeed det(*A*)*I*. - 

From lemma 1, it follows that *T* − *S* = 2 *n <sup>i</sup>*=<sup>1</sup> det( ¯ *R*¯(*i*) ) ≥ 0. In fact the following inequalities hold.

**Lemma 2** *With T and S defined as above, it holds that S* ≤ *T* ≤ 2*S. If T* = *S, all tensors R*(*i*) *ab have rank 1. If T* <sup>=</sup> <sup>2</sup>*S, all tensors R*(*i*) *ab are isotropic, i.e., proportional to the metric gab.* 

*Proof* Again, by linearity it is enough to consider one tensor ¯ *R*¯(1) = *A*. In an orthonormal frame which diagonalises *<sup>A</sup>*, we have *<sup>A</sup>* <sup>=</sup> *<sup>a</sup>* <sup>0</sup> 0 *c* (with *a* ≥ 0, *c* ≥ 0, *a* + *c* > 0). Hence

$$S = a^2 + c^2 \le a^2 + c^2 + 2ac = (a+c)^2 = T = 2(a^2 + c^2) - (a-c)^2 \le 2S.$$

The first inequality becomes equality when *ac* = 0, i.e., when *A* has rank one. The second inequality becomes equality when *a* = *c*, i.e., when *A* is isotropic. - **Definition 1** We define the mean rank, *rm*, by *rm* = *T*/*S*, with *T* and *S* as above. Hence, in two dimensions, 1 ≤ *rm* ≤ 2.

#### **2.2.2 A Canonical Decomposition**

It is customary [3, 7] to decompose a tensor with the symmetries of *Rabcd* into a sum where one term is the completely symmetric part:

$$R\_{abcd} = H\_{abcd} + W\_{abcd}, \text{ where } H\_{abcd} = R\_{(abcd)}, \ W\_{abcd} = R\_{abcd} - H\_{abcd}.$$

It is also customary to split *Habcd* into a trace-free part and 'trace part'. We start by defining *Hab* <sup>=</sup> *Habc<sup>c</sup>* , *H* = *Ha <sup>a</sup>* and then the trace-free part of *Hab*: *<sup>H</sup>*˚*ab* <sup>=</sup> *Hab* <sup>−</sup> <sup>1</sup> <sup>2</sup> *H gab* so that *Hab* <sup>=</sup> *<sup>H</sup>*˚*ab* <sup>+</sup> <sup>1</sup> <sup>2</sup> *H gab*. (These decompositions can be made in any dimension, but the actual coefficients, e.g., <sup>1</sup> <sup>2</sup> above and <sup>1</sup> <sup>8</sup> and <sup>3</sup> <sup>8</sup> et cetera below depend on the underlying dimension.) It is straightforward to check that

$$\check{H}\_{abcd} = H\_{abcd} - \mathsf{g}\_{(ab}H\_{cd)} + \frac{1}{8}H\mathsf{g}\_{(ab}\mathsf{g}\_{cd)} = H\_{abcd} - \mathsf{g}\_{(ab}\check{H}\_{cd)} - \frac{3}{8}H\mathsf{g}\_{(ab}\mathsf{g}\_{cd)}$$

is also trace-free. Hence we have the decomposition

$$H\_{abcd} = \mathring{H}\_{abcd} + \mathfrak{g}\_{(ab}H\_{cd)} - \frac{1}{8}H\mathfrak{g}\_{(ab}\mathfrak{g}\_{cd)} = \mathring{H}\_{abcd} + \mathfrak{g}\_{(ab}\mathring{H}\_{cd)} + \frac{3}{8}H\mathfrak{g}\_{(ab}\mathfrak{g}\_{cd)} + \cdots$$

Moreover, due to the symmetry of *Rabcd* , we find that

$$H\_{abcd} = \frac{1}{3} \left( R\_{abcd} + R\_{acbd} + R\_{adbc} \right)$$

and therefore that

$$W\_{abcd} = \frac{1}{3} \left( 2R\_{abcd} - R\_{acbd} - R\_{adbc} \right) \tag{7}$$

which implies that *Hab* <sup>=</sup> *Habc<sup>c</sup>* <sup>=</sup> <sup>1</sup> <sup>3</sup> (*Tab* <sup>+</sup> <sup>2</sup>*Sab*) and *Wab* <sup>=</sup> *Wabc<sup>c</sup>* <sup>=</sup> <sup>2</sup> <sup>3</sup> (*Tab* − *Sab*).

The degres of freedom, which for *Rabcd* is six, is distributed, where *Rabcd* ∼ {*H*˚*abcd* , *Hab*, *Wabcd* }, as

$$R\_{abcd} \sim \{\stackrel{\circ}{H}\_{abcd}, \stackrel{\circ}{H}\_{ab}, \stackrel{\circ}{W}\_{abcd}\} \sim \{\stackrel{\circ}{H}\_{abcd}, \stackrel{\circ}{H}\_{ab}, \stackrel{\circ}{H}, \stackrel{\circ}{W}\_{abcd}\}.$$

For *Hab* (or the pair *H*˚*ab*, *H*) this is clear. The total symmetry of *H*˚*abcd* leaves only five components (in a basis), *H*˚1111, *H*˚1112, *H*˚1122, *H*˚1222, *H*˚2222. However, the tracefree condition *<sup>H</sup>*˚*abcd <sup>g</sup>cd* <sup>=</sup> 0 imposes three conditions. (In an orthonormal frame, *<sup>H</sup>*˚1122 = −*H*˚1111, *<sup>H</sup>*˚2222 = −*H*˚1122 and *<sup>H</sup>*˚1112 = −*H*˚1222.) That *Wabcd* has only one degree of freedom follows from the following lemma.

**Lemma 3** *Suppose that Wabcd is given by* (7)*, and put Wab* = *Wabcd gcd , W* = *Wabgab. Then (in two dimensions)*

$$W\_{abcd} = \frac{W}{4} \left(2g\_{ab}g\_{cd} - g\_{ac}g\_{bd} - g\_{ad}g\_{bc}\right)$$

*Proof* By linearity, it is enough to consider the case when *Rabcd* = *Aab Acd* for some (symmetric) *Aab*. In terms of eigenvectors (to *A<sup>a</sup> <sup>b</sup>*) we can write *Aab* = α*xa xb* + β*ya yb*, where *xa x<sup>a</sup>* = *ya y<sup>a</sup>* = 1, *xa y<sup>a</sup>* = 0. In particular *gab* = *xa xb* + *ya yb*. From (7) we then get

$$\begin{split} W\_{abcd} &= \frac{1}{3} \left( 2R\_{abcd} - R\_{acbd} - R\_{adbc} \right) \\ &= \frac{1}{3} \left( 2A\_{ab}A\_{cd} - A\_{ac}A\_{bd} - A\_{ad}A\_{bc} \right) \\ &= \frac{1}{3} \left( 2(\alpha x\_a x\_b + \beta y\_a y\_b)(\alpha x\_c x\_d + \beta y\_c y\_d) \right. \\ &\quad - (\alpha x\_a x\_c + \beta y\_a y\_c)(\alpha x\_b x\_d + \beta y\_b y\_d) \\ &\quad - (\alpha x\_a x\_d + \beta y\_a y\_d)(\alpha x\_b x\_c + \beta y\_b y\_c)) \,. \end{split} \tag{8}$$

Expanding the parentheses, the components *xa xb xc xd* and *ya yb yc yd* vanish, leaving

$$\frac{\alpha\beta}{3}(2\mathbf{x}\_a\mathbf{x}\_b\mathbf{y}\_c\mathbf{y}\_d + 2\mathbf{y}\_a\mathbf{y}\_b\mathbf{x}\_c\mathbf{x}\_d - \mathbf{x}\_a\mathbf{x}\_c\mathbf{y}\_b\mathbf{y}\_d)$$

$$-\mathbf{y}\_a\mathbf{y}\_c\mathbf{x}\_b\mathbf{x}\_d - \mathbf{x}\_a\mathbf{x}\_d\mathbf{y}\_b\mathbf{y}\_c - \mathbf{y}\_a\mathbf{y}\_d\mathbf{x}\_b\mathbf{x}\_c) \tag{9}$$

$$=\frac{\alpha\beta}{3}\left(2\mathbf{g}\_{ab}\mathbf{g}\_{cd} - \mathbf{g}\_{ac}\mathbf{g}\_{bd} - \mathbf{g}\_{ad}\mathbf{g}\_{bc}\right),$$

where the last equality can be seen by inserting *gab* = *xa xb* + *ya yb* (for all indices) and expanding. Taking one trace, i.e., contracting with *<sup>g</sup>cd* gives *Wab* <sup>=</sup> <sup>2</sup>αβ <sup>3</sup> *gab*, and another trace gives *<sup>W</sup>* <sup>=</sup> <sup>4</sup>αβ <sup>3</sup> , which proves the lemma. -

## **3** *Rabcd* **as a Quadratic Form on** R**<sup>3</sup>**

Through the orthonormal basis for the space of symmetric two-tensors (in two dimensions) given by (2), the tensor *Rabcd* viewed as a quadratic form can be represented by a 3 × 3-matrix. Here, we will restrict ourselves to an orthonormal basis for *V*(*ab*), namely the basis {*e* (1) *ab* , *e* (2) *ab* , *e* (3) *ab* } from Sect. 2.1.3, defined in terms of the orthonormal basis{ξ *<sup>a</sup>*, η*<sup>a</sup>*} for *V<sup>a</sup>*. Thus, given *Rabcd* , we associate the symmetric matrix *Mi j* , where (the choice of an orthonormal basis justifies the mismatch of the indices *i*, *j*)

$$M\_{ij} \stackrel{\cdot}{=} R^{ab}{}\_{cd} e^{(i)}\_{ab} (e^{(j)})^{cd}, \quad 1 \le i, j \le 3.$$

It is instructive to see how the various derived tensors show up in *Mi j* . In terms of the basis (2) it is natural to look at the various parts of *Mi j* as follows

$$M\_{ij} \doteq \left(\frac{\stackrel{\times}{\times} \stackrel{\times}{\times} \stackrel{\times}{\times}}{\times \times \times} \right) \dot{=} \left(\frac{A \left| \overline{v} \right. \right) \text{.} \tag{10}$$

This splitting is natural for reasons which will become apparent in the next sections. Note, however, that with this representation it is tempting to consider coordinate changes in R3, which is not natural in this case. Rather, of interest is the change of basis in *V<sup>a</sup>* and the related *induced* change of coordinates in the representation (10). See Sect. 3.2.

#### *3.1 Representation of the Canonically Derived Parts of Rabcd*

It is helpful to see how the components of the various tensors *Tab*, *Sab*, *T* , *S*, *H*˚*abcd* , *H*˚*ab*, *H* and *W* show up as components of *Mi j* . As for *H*˚*ab*, e.g., *T*˚ *ab* denotes the trace-free part of *Tab*. Immediate is *M*33:

$$M\_{33} \stackrel{\cdot}{=} R^{ab}{}\_{cd} e^{(3)}\_{ab} (e^{(3)})^{cd} \stackrel{\cdot}{=} \frac{1}{2} R^{ab}{}\_{cd} g\_{ab} g^{cd} = \frac{1}{2} T\_{cd} g^{cd} = \frac{1}{2} T. \tag{11}$$

Similarly, for *i* = 1, 2 we have

$$M\_{i3} \doteq \frac{1}{\sqrt{2}} R^{ab}{}\_{cd} e^{(i)}\_{ab} \mathbf{g}^{cd} \doteq \frac{1}{\sqrt{2}} T^{ab}{}\_{ab} e^{(i)}\_{ab} \doteq \frac{1}{\sqrt{2}} \dot{T}^{ab} e^{(i)}\_{ab},\tag{12}$$

where the last equality follows form the trace-freeness of *e* (1) *ab* and *e* (2) *ab* . This means that the components of *T*˚ *ab* (properly rescaled) goes into *Mi j* as the components of *v* (and *v<sup>t</sup>* ) in (10). The same holds for *S*˚ *ab* and *H*˚*ab*, as *S*˚ *ab* <sup>=</sup> *<sup>T</sup>*˚ *ab* by Lemma 1, which then implies that also *<sup>H</sup>*˚*ab* <sup>=</sup> *<sup>T</sup>*˚ *ab* <sup>=</sup> *<sup>S</sup>*˚ *ab*. This latter relation follows from the trace-free part of the relation *Hab* <sup>=</sup> <sup>1</sup> <sup>3</sup> (*Tab* + 2*Sab*). Hence ⎛⎝⎞⎠⎛⎝⎞⎠

$$M\_{ij} \stackrel{\cdot}{=} \begin{pmatrix} \frac{A}{\cdot} \begin{vmatrix} \overrightarrow{T} \\ \overrightarrow{T} \end{vmatrix} \\ \frac{1}{2}T \end{pmatrix} \stackrel{\cdot}{=} \begin{pmatrix} \frac{\sigma}{2}I + \mathring{A} \begin{vmatrix} \overrightarrow{T} \\ \overrightarrow{T} \end{vmatrix} \\ \overrightarrow{T} \end{pmatrix},\tag{13}$$

where −→ *<sup>T</sup>*˚ <sup>=</sup> −→ *<sup>S</sup>*˚ <sup>=</sup> −→ *H*˚ encodes the two degrees of freedom in *T*˚ *ab* <sup>=</sup> *<sup>S</sup>*˚ *ab* <sup>=</sup> *<sup>H</sup>*˚*ab*. The matrix *<sup>A</sup>* is decomposed as *<sup>A</sup>* <sup>=</sup> <sup>σ</sup> <sup>2</sup> *<sup>I</sup>* <sup>+</sup> *<sup>A</sup>*˚ where *<sup>I</sup>* is the (2 <sup>×</sup> 2) identity matrix and *<sup>A</sup>*˚ is trace-free part of *<sup>A</sup>*. In particular, [*A*] = <sup>σ</sup>. =  *a b b c* we have *Ri j <sup>e</sup>*

To investigate [*Mi j*] = *M*<sup>11</sup> + *M*<sup>22</sup> + *M*33, i.e., the trace of *Mi j* we note that for a general symmetric matrix *Ri j* **·** (1) *i j* **·** <sup>=</sup> *<sup>a</sup>* √−*c* <sup>2</sup> , *Ri j e* (2) *i j* **·** = √ 2*b* <sup>2</sup> , *Ri j e* (3) *i j* **·** <sup>=</sup> *<sup>a</sup>* √ +*c* <sup>2</sup> . When *Mi j* is constructed from *Rabcd* which is an outer product *RabRcd* the trace is given by *<sup>M</sup>*<sup>11</sup> <sup>+</sup> *<sup>M</sup>*<sup>22</sup> <sup>+</sup> *<sup>M</sup>*<sup>33</sup> <sup>=</sup> ( *<sup>a</sup>* √−*c* <sup>2</sup> )<sup>2</sup> <sup>+</sup> ( <sup>√</sup> 2*b* <sup>2</sup> )<sup>2</sup> <sup>+</sup> ( *<sup>a</sup>* √ +*c* <sup>2</sup> )<sup>2</sup> <sup>=</sup> *a*<sup>2</sup> + 2*b*<sup>2</sup> + *c*<sup>2</sup> and from (6) this is *S*. Together with linearity, this shows that [*M*] = *M*<sup>11</sup> + *M*<sup>22</sup> + *M*<sup>33</sup> = *S* also when *Rabcd* is formed as in (1). Taking trace in (13), this gives

$$S = \sigma + \frac{1}{2}T, \quad \text{i.e.,} \quad \sigma = S - \frac{1}{2}T.$$

In addition, the relations below Eq. (7) show that

$$\begin{aligned} \text{M. Herbertson} \\ \text{Solution, the relations below Eq. (7) show that} \\ \begin{cases} H = \frac{1}{3}(T + 2S) \\ W = \frac{2}{3}(T - S) \end{cases} \quad \text{i.e.,} \quad \begin{cases} T = H + W \\ S = H - \frac{1}{2}W \end{cases} \quad \text{so that} \quad \sigma = \frac{1}{2}H - W. \end{aligned}$$

The two degres of freedom in *A*˚ corresponds to the two degrees of freedom in *H*˚*abcd* .

#### *3.2 The Behaviour of Mi j Under a Rotation of the Coordinate System in V<sup>a</sup>*

The components of *Mi j* are expressed in terms of the orthonormal basis tensors given by (2), and these in turn are based on the ON basis {ˆ *ξ* , ηˆ} for *V*. Putting the basis vectors in a row matrix  ξˆ ηˆ and the coordinates in a column matrix  <sup>ξ</sup> η so that a vector **<sup>u</sup>** <sup>=</sup> <sup>ξ</sup> <sup>ξ</sup><sup>ˆ</sup> <sup>+</sup> ηη<sup>ˆ</sup> <sup>=</sup> ξˆ ηˆ ξ η , and considering only orthonormal frames, the relevant change of basis is given by a rotation matrix *Q*(*v*) = *Qv* = cos *v* − sin *v* sin *v* cos *v* , i.e., we consider the change of basis =  =  

$$
\begin{pmatrix} \hat{\xi} \ \hat{\eta} \end{pmatrix} \rightarrow \begin{pmatrix} \hat{\xi} \ \hat{\eta} \end{pmatrix} = \begin{pmatrix} \hat{\xi} \ \hat{\eta} \end{pmatrix} \begin{pmatrix} \cos \upsilon - \sin \upsilon \\ \sin \upsilon \end{pmatrix} = \begin{pmatrix} \hat{\xi} \ \hat{\eta} \end{pmatrix} \mathcal{Q}(\upsilon).
$$

$$
\text{at for a vector} \\
\mathbf{u} = \begin{pmatrix} \hat{\xi} \ \hat{\eta} \end{pmatrix} \begin{pmatrix} \tilde{\xi} \\ \tilde{\eta} \end{pmatrix} = \begin{pmatrix} \hat{\xi} \ \hat{\eta} \end{pmatrix} \begin{pmatrix} \xi \\ \eta \end{pmatrix}, \text{the coordinates}
$$

This means that for a vector **u** = ˆ ξ˜ ˆ η˜ ξ˜ η˜ ξˆ ηˆ ξ η , the coordinates transform as

$$
\begin{pmatrix} \xi \\ \eta \end{pmatrix} \rightarrow \begin{pmatrix} \tilde{\xi} \\ \tilde{\eta} \end{pmatrix} = \mathcal{Q}^{-1}(\nu) \begin{pmatrix} \xi \\ \eta \end{pmatrix} = \mathcal{Q}^{\ell}(\nu) \begin{pmatrix} \xi \\ \eta \end{pmatrix} = \mathcal{Q}(-\nu) \begin{pmatrix} \xi \\ \eta \end{pmatrix}.
$$
 
$$\begin{array}{cccc} \dots & \dots & \dots \end{array}$$

For the components of the basis vectors *e* (1) *ab* , *e* (2) *ab* , *e* (3) *ab* we find (omitting the factor 1/ <sup>√</sup>2) 

$$\begin{aligned} \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix} &\rightarrow \begin{pmatrix} \cos\nu & \sin\nu \\ -\sin\nu & \cos\nu \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix} \begin{pmatrix} \cos\nu & -\sin\nu \\ \sin\nu & \cos\nu \end{pmatrix} = \begin{pmatrix} \cos 2\nu & -\sin 2\nu \\ -\sin 2\nu & -\cos 2\nu \end{pmatrix} \\ \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} &\rightarrow \begin{pmatrix} \cos\nu & \sin\nu \\ -\sin\nu \cos\nu \end{pmatrix} \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} \cos\nu - \sin\nu \\ \sin\nu \end{pmatrix} = \begin{pmatrix} \sin 2\nu & \cos 2\nu \\ \cos 2\nu - \sin 2\nu \end{pmatrix} \\ \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} &\rightarrow \begin{pmatrix} \cos\nu & \sin\nu \\ -\sin\nu \cos\nu \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} \cos\nu - \sin\nu \\ \sin\nu \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}, \end{aligned} \tag{14}$$

and this means that the components *Mi j* transform as

$$\begin{aligned} \text{at the components } &M\_{ij} \text{ transform as} \\\\ M\_{ij} \doteq \left(\frac{A\left|\overline{v}\right|}{\overline{v}^{\prime}\left|a\right.\right.} \right) \rightarrow \widetilde{M}\_{ij} \doteq \left(\frac{\mathcal{Q}\_{2\upsilon}^{\prime} A \mathcal{Q}\_{2\upsilon} \left| \mathcal{Q}\_{2\upsilon}^{\prime} \overline{v} \right.} \right) . \end{aligned} \tag{15}$$

But this latter expression is just

$$
\left(\frac{\mathcal{Q}\_{2\boldsymbol{\nu}}^{\boldsymbol{\epsilon}} \left| \overline{\boldsymbol{0}} \right|}{\overline{\boldsymbol{0}}^{\boldsymbol{\epsilon}} \left| 1 \right)} \right) \left(\frac{\boldsymbol{A} \left| \overline{\boldsymbol{\nu}}}{\overline{\boldsymbol{\nu}}^{\boldsymbol{\epsilon}} \left| \boldsymbol{a} \right)} \right) \left(\frac{\mathcal{Q}\_{2\boldsymbol{\nu}} \left| \overline{\boldsymbol{0}} \right|}{\overline{\boldsymbol{0}}^{\boldsymbol{\epsilon}} \left| \boldsymbol{1} \right)} \right) .
$$

hence we have the following important remark/observation:

**Remark 1** Viewing the matrix *Mi j* as an ellipsoid in R3, the effect of a rotation by an angle *v* in *V<sup>a</sup>* corresponds to a rotation of the ellipsoid by an angle 2*v* around the *z*-axis in R<sup>3</sup> (where the *z*-axis corresponds to the 'isotropic direction' given by *gab*).

#### **4 The Equivalence Problem for** *Rabcd*

The equivalence problem for *Rabcd* can be formulated in different ways (for an account in three dimensions, we refer to [3]). Given two tensors *Rabcd* and *R abcd* , both with the symmetries implied by (1), the question whether they are the same or not is straightforward as one can compare the components in any basis. However, *Rabcd* and *R abcd* could live in different (but isomorphic) vector spaces, e.g. two tangent spaces at different points, and the concept of equality becomes less clear. On the other hand, in terms of components *Rijkl* and *R ijkl* , one could ask whether there is a change of coordinates which takes one set of components into the other. If so, one can find a (invertible) matrix *P<sup>i</sup> <sup>j</sup>* so that -

$$\mathcal{R}\_{ijkl} = \widetilde{\mathcal{R}}\_{mnp} P^m{}\_i P^n{}\_j P^o{}\_k P^p{}\_l,$$

and the tensors are then said to be equivalent. As already mentioned, it is convenient to restrict the coordinate systems to orthonormal coordinates. This means that two different coordinate systems differ only by their orientation, i.e., the change of coordinates are given by a rotation matrix *Q* ∈ SO(2). Under the 'Cartesian convention' that all indices are written as subscripts, *Rabcd* and *R abcd* are equivalent if there is a matrix *Q* ∈ SO(2) so that (their Cartesian components satisfy) -

$$R\_{ijkl} = \ddot{R}\_{mnop} \mathcal{Q}\_{mi} \mathcal{Q}\_{nj} \mathcal{Q}\_{ok} \mathcal{Q}\_{pl}.$$

#### *4.1 Different Ways to Characterize the Equivalence of Rabcd and R**abcd* -

In this section, we will discuss three ways to determine whether two tensors *Rabcd* and *R abcd* are equivalent or not. In Sects. 4.1.1 and 4.1.2 we present two such methods briefly, while Sect. 4.1.3, which is more complete, contains the main result of this work.

**Fig. 2** Three identical (truncated) ellipsoids in R<sup>3</sup> with different orientations. The two leftmost ellipsoids can be carried over to each other through a rotation around the (vertical in the figure) *z*-axis, which implies that they represent the same tensor *Rabcd* (up to the meaning discussed). The right ellipsoid, despite identical eigenvalues with the two others, represent a different tensor since the rotation which carries this ellipsoid to any of the others is not around the *z*-axis

As mentioned in Sect. 1.1, the results of Sects. 4.1.1 and 4.1.2, which may be used in their own rights, rely on particular choices of basis matrices for *V*(*ab*). The formulation in Sect. 4.1.3 on the other hand, is expressed in the components of *Rabcd* (in any coordinate system) directly.


#### **4.1.1 Orientation of the Ellipsoid in** R**<sup>3</sup>**

A necessary condition for *Rabcd* and *R abcd* to be equivalent is that their corresponding <sup>3</sup> <sup>×</sup> 3-matrices *Mi j* and *<sup>M</sup>i j* have the same eigenvalues. On the other hand, this is not sufficient since the representation in R<sup>3</sup> should reflect the freedom in rotating the coordinate system in *<sup>V</sup><sup>a</sup>* <sup>∼</sup> <sup>R</sup>2. With the coordinates adopted, this corresponds to a rotation of the associated ellipsoid around the *z*-axis in R<sup>3</sup> (see Remark 1 in Sect. 3.2). This is illustrated in Fig. 2 where three ellipsoids, all representing positive definite symmetric mappings having identical eigenvalues, are shown. The two first ellipsoids can be rotated into each other by a rotation around the *z*-axis. This implies that the corresponding tensors *Rabcd* and *R abcd* are equivalent. The third ellipsoid can also be rotated into the two others, but these rotations are around directions other than the *z*-axis, which means that this ellipsoid represents a different tensor.

In the generic case, with all eigenvalues different, it is easy to test whether two different ellipsoids can be transfered into each other through a rotation around the *<sup>z</sup>*-axis. This will be the case if the corresponding eigenvectors (of *Mi j* and *<sup>M</sup>i j*) have the same angle with the *z*-axis. Hence it is just a matter of checking the *z*-components of the three normalized eigenvectors and see if they are equal up to sign.

#### **4.1.2 Components in a Canonical Coordinate System**

In a sense, this is the most straightforward method. In a coordinate system which respects *e* (3) *ab* as the *<sup>z</sup>*-axis in *<sup>V</sup>*(*ab*) <sup>∼</sup> <sup>R</sup>3, two tensors *Rabcd* and *<sup>R</sup> abcd* are equivalent if there is a rotation matrix (in two dimensions) *Q* such that

⎛

⎞

⎛


$$\text{metric Positive (Semi-) Define Tensor in Two dimensions}\tag{17}$$

$$\left(\frac{A}{\frac{\vec{T}}{\vec{T}}\,'\middle|\frac{1}{2}T}\right) = \left(\frac{\underline{Q}'\tilde{A}\underline{Q}\,\left|\underline{Q}'\stackrel{\vec{\mathcal{L}}}{\tilde{T}}\right.\right)\tag{16}$$

$$\left(\frac{\vec{A}}{\tilde{T}}\,'\middle|\frac{1}{2}T\right) = \left(\frac{\underline{Q}'\tilde{A}\underline{Q}\,\left|\underline{Q}'\stackrel{\vec{\mathcal{L}}}{\tilde{T}}\right.\right)\tag{16}$$


⎞

Hence, equivalence can be easily tested by first checking that *T* = *T* and that || −→ *<sup>T</sup>*˚ || = || −→ ˚ *T* ||. If this is the case, (and if || −→ *<sup>T</sup>*˚ || <sup>&</sup>gt; 0) one determines the rotation matrix *<sup>Q</sup>* which gives −→ *<sup>T</sup>*˚ <sup>=</sup> *<sup>Q</sup><sup>t</sup>* −→ -˚ *<sup>T</sup>* , and equivalence is then determined by if *<sup>A</sup>* <sup>=</sup> *<sup>Q</sup><sup>t</sup> AQ* or not. If || −→ *<sup>T</sup>*˚ || = || −→ -˚ *T* || = 0, the equivalence of *A* and *A* can be determined directly, i.e., by checking whether [*A*]=[*A* -] and [*A*2]=[*A* -<sup>2</sup>] or not.

#### **4.1.3 Equivalence Through (algebraic) Invariants of** *Rabcd*

If a solution is found, this is perhaps the most satisfactory way to establish equivalence, in particular if the invariants are constructed by simple algebraic operations only. (For instance, to a symmetric 3 × 3-matrix *A* one can take the three eigenvalues as invariants or else for instance the traces of *A*, *A*<sup>2</sup> and *A*3. The former set requires some calculations, but the latter is immediate.)

Examples of invariants are *T* = *Rabcd gabgcd* , *S* = *Rabcd gacgbd* and the invariants *H* = *Habgab*, *W* = *Wabgab*. To produce the invariants, we use the tensor *Rabcd* and the metric *gab*. However, if we regard *<sup>V</sup><sup>a</sup>* <sup>∼</sup> <sup>R</sup><sup>2</sup> as oriented, so that the orthonormal basis{ˆ *ξ* , ηˆ}for *V<sup>a</sup>* also is oriented, then invariants can also be formed in another way. Namely, since the space of symmetric 2 × 2 matrices is 3-dimensional, and since the metric *gab* singles out a 1-dimensional subspace, it also determines a 2-dimensional subspace *L*; all elements orthogonal to *gab*. This subspace is the set of all symmetric 2 × 2 matrices which are also trace-free. *L* can be given an orientation through an area form, which in turn inherits the orientation from *V<sup>a</sup>*.

In general, with right-handed Cartesian coordinates *x* <sup>1</sup>, *x* 2, the area form is given by = *dx* <sup>1</sup> ∧ *dx* <sup>2</sup> where (ω ∧ μ)*ab* = ω*a*μ*<sup>b</sup>* − ω*b*μ*a*. With the orthonormal basis {ˆ *ξ* , ηˆ} ( for *V<sup>a</sup>* ) also right handed, we define, cf. (2),

$$e\_{ab}^{(1)} = \frac{1}{\sqrt{2}} (\hat{\xi}\_a \hat{\xi}\_b - \hat{\eta}\_a \hat{\eta}\_b), \quad e\_{ab}^{(2)} = \frac{1}{\sqrt{2}} (\hat{\xi}\_a \hat{\eta}\_b + \hat{\eta}\_a \hat{\xi}\_b). \tag{17}$$

The area form on *L* is then ∼ *e*(1) ∧ *e*(2) , or

$$
\epsilon \sim E\_{abcd} = e\_{ab}^{(1)} e\_{cd}^{(2)} - e\_{ab}^{(2)} e\_{cd}^{(1)}.\tag{18}
$$

It is not hard to see that this definition is independent of the orientation of {ˆ *ξ* , *η*ˆ}. We observe that 2*Eabcd* = (ˆ *ξ <sup>a</sup>* ˆ *ξ <sup>b</sup>* − *η*ˆ *<sup>a</sup>η*ˆ *<sup>b</sup>*)(ˆ *ξ <sup>c</sup>η*ˆ *<sup>d</sup>* + *η*ˆ *<sup>c</sup>* ˆ *ξ <sup>d</sup>* ) − (ˆ *ξ <sup>a</sup>η*ˆ *<sup>b</sup>* + *η*ˆ *<sup>a</sup>* ˆ *ξ <sup>b</sup>*)(ˆ *ξ c* ˆ *ξ <sup>d</sup>* − *η*ˆ *<sup>c</sup>η*ˆ *<sup>d</sup>* ). By replacing ˆ *ξ* by *ω*ˆ = cos *v* ˆ *ξ* + sin *v η*ˆ and *η*ˆ by *μ*ˆ = − sin *v* ˆ *ξ* + cos *v η*ˆ, i.e., a rotated orthonormal basis, it is straightforward to check that

18 M. Herberthson et al.

$$\begin{aligned} & (\hat{\mathfrak{w}}\_a \hat{\mathfrak{w}}\_b - \hat{\mathfrak{w}}\_a \hat{\mathfrak{x}}\_b)(\hat{\mathfrak{w}}\_c \hat{\mathfrak{x}}\_d + \hat{\mathfrak{x}}\_c \hat{\mathfrak{w}}\_d) - (\hat{\mathfrak{w}}\_a \hat{\mathfrak{x}}\_b + \hat{\mathfrak{x}}\_a \hat{\mathfrak{w}}\_b)(\hat{\mathfrak{w}}\_c \hat{\mathfrak{w}}\_d - \hat{\mathfrak{x}}\_c \hat{\mathfrak{x}}\_d) \\ & = (\hat{\mathfrak{F}}\_a \hat{\mathfrak{F}}\_b - \hat{\mathfrak{y}}\_a \hat{\mathfrak{y}}\_b)(\hat{\mathfrak{F}}\_c \hat{\mathfrak{y}}\_d + \hat{\mathfrak{y}}\_c \hat{\mathfrak{xi}}\_d) - (\hat{\mathfrak{F}}\_a \hat{\mathfrak{y}}\_b + \hat{\mathfrak{y}}\_a \hat{\mathfrak{x}}\_b)(\hat{\mathfrak{F}}\_c \hat{\mathfrak{x}}\_d - \hat{\mathfrak{y}}\_c \hat{\mathfrak{y}}\_d) \end{aligned} \tag{19}$$

so that *Eabcd* is well defined. We recollect that area form *Eabcd* is defined, through the induced metric, on the plane *L* (which in turn is also defined through the metric *gab*) and the orientation on *V<sup>a</sup>*. Hence *Eabcd* can be used when forming invariants. -

We will now state the result of this work, namely the existence of six invariants which can be used to investigate equivalence of two tensors *Rabcd* and *R abcd* . We start by defining

$$\begin{aligned} S &= \mathsf{R}\_{abcd} \mathcal{g}^{ac} \mathcal{g}^{bd} \\ T &= \mathsf{R}\_{abcd} \mathcal{g}^{ab} \mathcal{g}^{cd} \\ J\_0 &= \mathsf{R}\_{abcd} \mathcal{R}^{abcd} \\ J\_1 &= T^{ab} \mathcal{T}\_{ab} \\ J\_2 &= \mathsf{R}\_{abcd} T^{ab} T^{cd} \\ J\_3 &= T^{ab} \mathcal{R}\_{abcd} E^{cdef} \mathcal{T}\_{ef} . \end{aligned} \tag{20}$$
 
$$\text{where } \mathcal{E}\_{abcd} \text{ is defined by (17) and (18). Similarly, we define } \widetilde{S}, \widetilde{T}, \widetilde{J}\_0, \widetilde{J}\_1, \widetilde{J}\_2 \text{ and }$$

*S*, *T* , *J* <sup>0</sup>, *J* <sup>1</sup>, *J* <sup>2</sup> and *J* <sup>3</sup> as the corresponding invariants formed from *R abcd* . We make the remark that for most of these invariants, their immediate interpretations still remain to be found. Rather, their value lie in the fact that they form a set which can be used to establish the equivalence in Theorem 1 below. On the other hand, some interpretations are possible. In particular, the quotient *T*/*S* (see Definition 1) lies in the interval [1, 2] and has the meaning given by Lemma 2. **Theorem 1** *Suppose that Rabcd* <sup>=</sup> *<sup>n</sup>* ------

*<sup>i</sup>*=<sup>1</sup> *<sup>R</sup>*(*i*) *ab <sup>R</sup>*(*i*) *cd , with R*(*i*) *ab* ≥ 0 *and that Rijkl are the components of Rabcd in some basis. Suppose also that R abcd* = *n <sup>i</sup>*=<sup>1</sup> *R* (*i*) *ab R* (*i*) *cd , with R* (*i*) *ab* ≥ 0 *and that R ijkl are the components of R abcd in some, possibly unrelated, basis. If (and only if) S* <sup>=</sup> -*S*, *T* = *T* -, *J*<sup>0</sup> = *J* -<sup>0</sup>, *J*<sup>1</sup> = *J* -<sup>1</sup>, *J*<sup>2</sup> = *J* -<sup>2</sup>, *J*<sup>3</sup> = *J* -<sup>3</sup>*, then there is a transformation matrix P<sup>i</sup> <sup>j</sup> such that* -

$$R\_{ijkl} = \widetilde{R}\_{mnp} P^m{}\_i P^n{}\_j P^o{}\_k P^p{}\_l.$$

*Proof* Since the invariants are defined without reference to any basis, it is sufficient to consider the components expressed in an orthonormal frame, and in that case we must prove the existence of a rotation matrix *Q* ∈ SO(2) so that -

$$R\_{ijkl} = R\_{mnop} \mathcal{Q}\_{mi} \mathcal{Q}\_{nj} \mathcal{Q}\_{ok} \mathcal{Q}\_{pl}.$$

Since

$$R\_{abcd} = M\_{ij} e^{(i)}\_{ab} e^{(j)}\_{cd} \, , \tag{21}$$

we can consider the invariants formed from the components of

Variance Measures for Symmetric Positive (Semi-) Definite Tensors in Two Dimensions 19 --

$$\text{ametric Positive (Semi-) Define Tensor in Two dimensions}\tag{19}$$

$$M\_{ij} = \left(\frac{A}{\overline{u}\overline{\ell}} \bigg| \overline{c}\right) \text{ and } \widetilde{M}\_{ij} = \begin{pmatrix} \widetilde{A} \\ \overline{\widetilde{u}\ell} \end{pmatrix} \tag{22}$$


and we must demonstrate the existence of a rotation matrix *Q* = *Q*2*<sup>v</sup>* such that

⎠

$$
\langle \omega^2 \rangle \qquad \qquad \langle \overline{u}^2 \rangle^2
$$

$$
\text{rate the existence of a rotation matrix } \mathcal{Q} = \mathcal{Q}\_{2\upsilon} \text{ such that}
$$

$$
\widetilde{A} = \mathcal{Q}\_{2\upsilon}^t A \mathcal{Q}\_{2\upsilon}, \quad \widetilde{\overline{u}} = \mathcal{Q}\_{2\upsilon}^t \overline{u}, \quad \widetilde{c} = c. \tag{23}
$$




⎠

⎝

We make the ansatz ⎝


$$
\Lambda = \mathcal{Q}\_{2\upsilon} \Lambda \mathcal{Q}\_{2\upsilon}, \quad u = \mathcal{Q}\_{2\upsilon} u, \quad c = \upsilon. \tag{23}
$$

The ansatz

$$
\mathcal{M}\_{ij} = \begin{pmatrix}
\frac{\sigma}{2} + a & b & \left|\mathbf{x}\right| \\
b & \frac{\sigma}{2} - a \left|\mathbf{y}\right| \\
\hline \mathbf{x} \ \mathbf{y} & \left|\mathbf{c}\right|
\end{pmatrix}, \quad \widetilde{M}\_{ij} = \begin{pmatrix}
\frac{\widetilde{\sigma}}{2} + \widetilde{a} & \widetilde{b} & \left|\widetilde{\mathbf{x}}\right| \\
\hline \widetilde{b} & \frac{\widetilde{\sigma}}{2} - \widetilde{a} & \widetilde{\mathbf{y}} \\
\hline \widetilde{\mathbf{x}} & \widetilde{\mathbf{y}} & \left|\widetilde{\mathbf{c}}\right|
\end{pmatrix}. \tag{24}
$$

Through (21) it is straightforward to see that

⎛

$$\begin{array}{ll} S = \sigma + c, & T = 2c, & J\_0 = 2(a^2 + b^2) + c^2 + \sigma^2/2 + 2(\mathbf{x}^2 + \mathbf{y}^2), \\ J\_1 = 2(c^2 + \mathbf{x}^2 + \mathbf{y}^2) \\ \end{array}$$

$$\begin{array}{ll} = \widetilde{S}, T = \widetilde{T}, J\_0 = \widetilde{J}\_0, J\_1 = \widetilde{J}\_1, \text{it follows that } \sigma = \widetilde{\sigma}, c = \widetilde{c}, a^2 + b^2 = \widetilde{a}^2 \\ \end{array}$$

so if *S* = -*S*, *T* = *T* , *J*<sup>0</sup> = *J* <sup>0</sup>, *J*<sup>1</sup> = *J <sup>a</sup>*<sup>2</sup> <sup>+</sup> *<sup>b</sup>*<sup>2</sup> and *<sup>x</sup>* <sup>2</sup> <sup>+</sup> *<sup>y</sup>*<sup>2</sup> <sup>=</sup> *x* 2 + *y*2. Since the isotropic part of *A*, i.e., <sup>σ</sup> <sup>2</sup> *I* is unaffected by a rotation of the coordinate system, we consider the traceless parts *<sup>A</sup>*˚ <sup>=</sup> *a b b* −*a* , -˚ *A* = *a b b* −*a* , and the task is to assert a rotation matrix *Q* such that *a* ----

$$\begin{aligned} \text{the task is to assert a rotation matrix } \mathcal{Q} \text{ such that} \\\\ \begin{pmatrix} a & b \\ b & -a \end{pmatrix} = \mathcal{Q}^{\ell} \begin{pmatrix} \widetilde{a} & \widetilde{b} \\ \widetilde{b} & -\widetilde{a} \end{pmatrix} \mathcal{Q}, \quad \begin{pmatrix} \chi \\ \chi \end{pmatrix} = \mathcal{Q}^{\ell} \begin{pmatrix} \widetilde{\chi} \\ \widetilde{\chi} \end{pmatrix}, \end{aligned}$$

if also *J*<sup>2</sup> = *J* <sup>2</sup>, *J*<sup>3</sup> = *J* 3. Again it is straightforward to calculate the remaining invariants, and we find

$$\begin{array}{l} J\_2 = 4b\mathbf{x}\mathbf{y} + 2a(\mathbf{x}^2 - \mathbf{y}^2) + 2c^3 + (4c + \sigma)(\mathbf{x}^2 + \mathbf{y}^2) \\ J\_3 = 4a\mathbf{x}\mathbf{y} - 2b(\mathbf{x}^2 - \mathbf{y}^2) \end{array}$$
 
$$\text{for } \widetilde{J}\_2, \widetilde{J}\_3. \text{ Hence, } (\text{since } \sigma = \widetilde{\sigma}, c = \widetilde{c})$$

and similarly for *J* <sup>2</sup>, *J c*)

$$\begin{aligned} \text{Find } & \widetilde{\mathbf{y}} > \widetilde{\mathbf{c}} \text{ and } \\ \text{and similarly for } \widetilde{J}\_2, \widetilde{J}\_3. \text{ Hence, } (\text{since } \sigma = \widetilde{\sigma}, c = \widetilde{c}) \\ & \begin{aligned} a^2 + b^2 &= \widetilde{a}^2 + \widetilde{b}^2 \\ \mathbf{x}^2 + \mathbf{y}^2 &= \widetilde{\mathbf{x}}^2 + \widetilde{\mathbf{y}}^2 \\ 2b\mathbf{x}\mathbf{y} + a(\mathbf{x}^2 - \mathbf{y}^2) &= 2\widetilde{b}\widetilde{\mathbf{x}}\widetilde{\mathbf{y}} + \widetilde{a}(\widetilde{\mathbf{x}}^2 - \widetilde{\mathbf{y}}^2) \\ 2a\mathbf{x}\mathbf{y} - b(\mathbf{x}^2 - \mathbf{y}^2) &= 2\widetilde{a}\widetilde{\mathbf{x}}\widetilde{\mathbf{y}} - \widetilde{b}(\widetilde{\mathbf{x}}^2 - \widetilde{\mathbf{y}}^2) \end{aligned} \tag{25}$$
  $\text{Suppose first that } \mathbf{x}^2 + \mathbf{y}^2 > 0. \text{ The equality } \mathbf{x}^2 + \mathbf{y}^2 = \widetilde{\mathbf{x}}^2 + \widetilde{\mathbf{y}}^2 \text{ then guarantees the } \widetilde{\mathbf{x}}\widetilde{\mathbf{y}} \text{ and } \widetilde{\mathbf{x}}\widetilde{\mathbf{y}} \text{ is } \widetilde{\mathbf{x}}\widetilde{\mathbf{y}} \dots \widetilde{\mathbf{x}}\widetilde{\mathbf{y}} \text{ (27).$ 

*y*<sup>2</sup> then guarantees the existence of the rotation matrix *<sup>Q</sup>* which is determined via the relation  *<sup>x</sup> y* = *Q<sup>t</sup> x y* . This can also be expressed as *Q<sup>t</sup>* 1 *x y* <sup>=</sup> *<sup>Q</sup><sup>t</sup>* 2 *x y* for some rotation matrices *Q*1, *Q*2, where *<sup>Q</sup>* <sup>=</sup> *<sup>Q</sup>*2*Q<sup>t</sup>* 1. We now choose the rotation matrix *Q*<sup>1</sup> so that in the untilded

coordinates, *y* = 0. Similarly we choose *Q*<sup>2</sup> so that for the tilded coordinates, we get a frame where *y* = 0. The equalities between the invariants in (25) then become *a*2 + *b*2 = *a*2 + -

$$\begin{array}{c} \text{qualities between th} \\ \\ a^2 + b^2 = \widetilde{a}^2 + \widetilde{b}^2 \\ x^2 = \widetilde{x}^2 \\ ax^2 = \widetilde{ax}^2 \\ -bx^2 = -\widetilde{b}\widetilde{x}^2 \end{array}$$

so that *<sup>a</sup>* <sup>=</sup> *a*, *b* = *b*. This proves the theorem when *x* <sup>2</sup> + *y*<sup>2</sup> > 0. When *x* <sup>2</sup> + *y*<sup>2</sup> = *x* 2 + *<sup>y</sup>*<sup>2</sup> <sup>=</sup> 0, i.e., *<sup>x</sup>* <sup>=</sup> *<sup>y</sup>* <sup>=</sup> *x* = *<sup>y</sup>* <sup>=</sup> 0, the remaining equality *<sup>a</sup>*<sup>2</sup> <sup>+</sup> *<sup>b</sup>*<sup>2</sup> <sup>=</sup> *a*2 + *b*<sup>2</sup> is sufficient since we can again choose frames in which *<sup>b</sup>* <sup>=</sup> *<sup>b</sup>* <sup>=</sup> 0 and *<sup>a</sup>* <sup>&</sup>gt; <sup>0</sup>,*a* > 0. It then follows that *<sup>a</sup>* <sup>=</sup> *a*. -

#### **5 Discussion**

In this work, we started with a family of symmetric positive (semi-)definite tensors in two dimensions and considered its variance. This lead us to a fourth order tensor *Rabcd* with the same symmetries as the elasticity tensor in continuum mechanics. After listing a number of possible issues to address, we focused on the equivalence problem. Namely, given the components of two such tensors *Rabcd* and *R abcd* , how can one determine if they represent the same tensor (but in different coordinate systems) or not? In Sect. 4, we saw that this could be investigated in different ways. The result of Theorem 1 is most satisfactory in the sense that it is expressible in terms of the components of the fourth order tensors directly.

There are two natural extensions and/or ways to continue this work. The first is to apply the result to realistic families of e.g., diffusion tensors in two dimensions. The objective is then, apart from establishing possible equivalences, to investigate the geometric meaning of the invariants. The other natural continuation is to investigate the corresponding problem in three dimensions. The degrees of freedom of *Rabcd* will then increase from 6 to 21, leaving us with a substantially harder, but also perhaps more interesting, problem.

**Acknowledgements** The authors acknowledge the following sources for funding: Swedish Foundation for Strategic Research AM13-0090, the Swedish Research Council 2015-05356 and 2016- 04482, Linköping University Center for Industrial Information Technology (CENIIT), VIN-NOVA/ITEA3 17021 IMPACT, Analytic Imaging Diagnostics Arena (AIDA), and National Institutes of Health P41EB015902.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Degenerate Curve Bifurcations in 3D Linear Symmetric Tensor Fields**

**Yue Zhang, Hongyu Nie, and Eugene Zhang**

**Abstract** 3D symmetric tensor fields have a wide range of applications in medicine, science, and engineering. The topology of tensor fields can provide key insight into their structures. In this paper we study the number of possible topological bifurcations in 3D linear tensor fields. Using the linearity/planarity classification and wedge/trisector classification, we explore four types of bifurcations that can change the number and connectivity in the degenerate curves as well as the number and location of transition points on these degenerate curves. This leads to four types of bifurcations among nine scenarios of 3D linear tensor fields.

#### **1 Introduction**

Tensor field visualization is an important topic in visualization, with many applications in medical imaging, solid and fluid mechanics, material science, earthquake engineering, and computer graphics.

Recent advances on tensor field visualization focus in topology-driven analysis and visualization of 3D symmetric tensor fields. Degenerate curves are one of the most fundamental topological features in a tensor field, and much research has focused on the understanding and efficient extraction of degenerate curves from piecewise linear tensor fields defined on a tetrahedral mesh [6–8].

Y. Zhang (B)

H. Nie

E. Zhang

School of Electrical Engineering and Computer Science 3117 Kelley Engineering Center, Oregon State University, Corvallis, OR 97331, USA e-mail: zhangyue@eecs.oregonstate.edu

School of Electrical Engineering and Computer Science 1148 Kelley Engineering Center, Oregon State University, Corvallis, OR 97331, USA e-mail: nieh@oregonstate.edu

School of Electrical Engineering and Computer Science 2111 Kelley Engineering Center, Oregon State University, Corvallis, OR 97331, USA e-mail: zhange@eecs.oregonstate.edu

In the book chapter, we focus on a problem that has received relatively little attention: bifurcations in tensor field topology. To make our investigation effective with potential application to real datasets, we focus on 3D linear tensor fields. We explore all the theoretically possible bifurcations. Moreover, we have conducted experiment to verify whether these bifurcations can occur.

The rest of the paper is structured as follows. Section 2 reviews past research in topology-driven analysis of symmetric tensor fields. In Sect. 3 we review relevant mathematical background and results on tensor fields. In Sect. 4 we report the findings of our exploration before concluding in Sect. 5.

#### **2 Previous Work**

Much research exists on 2D and 3D symmetric tensor fields, and we refer the readers to the survey by Kratz et al. [4] and Zhang et al. [11] for a more comprehensive review. In this book chapter we only refer to the research that is most relevant.

Delmarcelle and Hesselink [1] introduce the notion of *degenerate points* for 2D symmetric tensors, where eigenvector directions are not well-defined. Zhang et al. [12] explore the physical meanings of degenerate points in the stress tensor and strain tensor from continuum mechanics.

Hesselink et al. later extend this work to 3D symmetric tensor fields [3] and study the degeneracies in such fields. Zheng and Pang [16] point out that triple degeneracies are structurally unstable features. That is, an arbitrarily small perturbation to the field will remove such degenerate points. Zheng and Pang further show that double degeneracies, i.e., only two equal eigenvalues, form lines in the domain. In this work and subsequent research [18], they provide a number of degenerate curve extraction methods based on the analysis of the discriminant function of the tensor field. Furthermore, Zheng et al. [17] point out that near degenerate curves the tensor field exhibits 2D degenerate patterns and define separating surfaces which are extensions of separatrices from 2D symmetric tensor field topology. Tricoche et al. [9] convert the problem of extracting degenerate curves in a 3D tensor field to that of finding the ridge and valley lines of an invariant of the tensor field, thus leading to a more robust extraction algorithm. More recently, Palacios et al. [6] extract degenerate curves using an algorithm for algebraic surface extraction method called A-patches. Palacios et al. [5] introduce a number of topological editing operations with which a 3D tensor field can be edited for graphics applications.

Zhang et al. [13] describe a number of important properties of 3D linear tensor fields. They [15] show that in a 3D linear tensor field, there are at least two and at most four degenerate curves. Roy et al. [8] develop a parameterization with which all degenerate points in a 3D piecewise linear tensor field can be extracted efficiently and at any given accuracy. Zhang et al. [14] show that there are at most eight transition points in a 3D linear tensor field.

#### **3 Background on Tensors and Tensor Fields**

In this section we review the most relevant background on 2D and 3D symmetric tensors and tensor fields [14].

#### *3.1 Tensors*

A *K*-dimensional (symmetric) tensor **T** has *K* real-valued *eigenvalues*: λ<sup>1</sup> ≥ λ<sup>2</sup> ≥ ... ≥ λ*<sup>K</sup>* . The largest and smallest eigenvalues are referred to as the *major eigenvalue* and *minor eigenvalue*, respectively. When *K* = 3, the middle eigenvalue is referred to as the *medium eigenvalue*. An eigenvector belonging to the major eigenvalue is referred to as a *major eigenvector*. Medium and minor eigenvectors can be defined similarly. Eigenvectors belonging to different eigenvalues are mutually perpendicular.

The trace of a tensor **T** = (**T***i j*)is*trace*(**T**) = -*K <sup>i</sup>*=<sup>1</sup> λ*<sup>i</sup>* . **T** can be uniquely decomposed as **<sup>D</sup>** <sup>+</sup> **<sup>A</sup>** where **<sup>D</sup>** <sup>=</sup> *trace*(**T**) *<sup>K</sup>* <sup>I</sup> (<sup>I</sup> is the *<sup>K</sup>*-dimensional identity matrix) and **A** = **T** − **D**. The *deviator* **A** is a *traceless* tensor, i.e., *trace*(**A**) = 0. Note that **T** and **A** have the same set of eigenvectors. Consequently, the anisotropy in a tensor field can be defined in terms of its deviator tensor field. Another nice property of the set of traceless tensors is that it is closed under matrix addition and scalar multiplication, making it a linear subspace of the set of tensors.

The magnitude of a tensor **<sup>T</sup>** is ||**T**|| = - <sup>1</sup>≤*i*,*j*≤*<sup>K</sup> <sup>T</sup>* <sup>2</sup> *i j* = -*K <sup>i</sup>* λ<sup>2</sup> *<sup>i</sup>* , while the determinant is <sup>|</sup>**T**| = *<sup>K</sup> <sup>i</sup>*=<sup>1</sup> λ*<sup>i</sup>* .

A tensor is *degenerate* when there are repeating eigenvalues. In this case, there exists at least one eigenvalue whose corresponding eigenvectors form a higherdimensional space than a line. When *K* = 2 a degenerate tensor must be a multiple of the identity matrix.

#### *3.2 Tensor Field Topology*

We now review *tensor fields*, which are tensor-valued functions over some domain <sup>⊂</sup> <sup>R</sup>*<sup>K</sup>* . A tensor field can be thought of as *<sup>K</sup>* eigenvector fields, corresponding to the *K* eigenvalues. A *hyperstreamline* with respect to an eigenvector field *ei*(*p*) is a 3*D* curve that is tangent to *ei* everywhere along its path. Two hyperstreamlines belonging to two different eigenvalues can only intersect at the right angle, since eigenvectors belonging to different eigenvalues must be mutually perpendicular.

Hyperstreamlines are usually curves. However, they can occasionally consist of only one point, where there is more than one choice of lines that correspond to the eigenvector field. This is precisely where the tensor field is degenerate. A point

**Fig. 1** A wedge (left) and a trisector (right)

*p*<sup>0</sup> ∈ is a *degenerate point* if **T**(*p*0) is degenerate. One important topological feature of a tensor field consists of its degenerate points.

In 2*D*, the set of degenerate points of a tensor field consists of isolated points under numerically stable configurations, when the topology does not change given sufficiently small perturbation in the tensor field. An isolated degenerate point can be measured by its *tensor index* [10], defined in terms of the *winding number* of one of the eigenvector fields on a loop surrounding the degenerate point. The most fundamental types of degenerate points are *wedges* and *trisectors*, with a tensor index of <sup>1</sup> <sup>2</sup> and <sup>−</sup><sup>1</sup> <sup>2</sup> , respectively. Let *L***T***p*<sup>0</sup> (*p*) be the local linearization of **T**(*p*) at a degenerate point *p*<sup>0</sup> = *x*0 *y*0 , i.e.,

$$L\mathbf{T}\_{p0}(p) = \begin{pmatrix} a\_{11}(\mathbf{x} - \mathbf{x}\_0) + b\_{11}(\mathbf{y} - \mathbf{y}\_0) & a\_{12}(\mathbf{x} - \mathbf{x}\_0) + b\_{12}(\mathbf{y} - \mathbf{y}\_0) \\ a\_{12}(\mathbf{x} - \mathbf{x}\_0) + b\_{12}(\mathbf{y} - \mathbf{y}\_0) & a\_{22}(\mathbf{x} - \mathbf{x}\_0) + b\_{22}(\mathbf{y} - \mathbf{y}\_0) \end{pmatrix} \tag{1}$$

Then δ = *<sup>a</sup>*11−*a*<sup>22</sup> <sup>2</sup> *a*<sup>12</sup> *b*11−*b*<sup>22</sup> <sup>2</sup> *<sup>b</sup>*12 is invariant under the change of basis [2]. Moreover, *p*<sup>0</sup> is a wedge when δ > 0 and a trisector when δ < 0. When δ = 0, *p*<sup>0</sup> is a higherorder degenerate point. A *major separatrix* is a hyperstreamline emanating from a degenerate point following the major eigenvector field. A *minor separatrix* is defined similarly.

The total tensor index of a continuous tensor field over a two-dimensional manifold is equal to the *Euler characteristic* of the underlying manifold. Consequently, it is not possible to remove one degenerate point. Instead, a pair of degenerate points with opposing tensor indexes (a wedge and trisector pair) must be removed simultaneously [10]. Figure 1 shows a wedge pattern (left) and a trisector pattern (right), respectively.

In 3D, a degenerate point can have either two or three eigenvalues being the same, the latter of which is structurally unstable. Structurally stable degenerate points have two eigenvalues being the same. These eigenvalues are the *repeating eigenvalues*, while the third eigenvalue is the *dominant eigenvalue*. When the dominant eigenvalue is the major eigenvalue, the degenerate point is a *linear degenerate point* (L). On the other hand, when the dominant eigenvalue is the the minor eigenvalue, the degenerate point is a *planar degenerate point* (P). We refer to this linear/planar classification of a degenerate point as its L/P classification. Along a degenerate curve, its L/P classification does not change.

A degenerate point can also be classified in a different way, by projecting the tensor field onto the plane passing through the degenerate point and perpendicular to its dominant eigenvector. The degenerate point is also a degenerate point in the 2D projected tensor field. Consequently, the original 3D degenerate point is classified as either a wedge (W) or a trisector (T) based on theW/T type of the 2D degenerate point. Note that the projection of the 3D tensor field onto other planes not perpendicular to the dominant eigenvector does not necessarily have the same W/T classification. Along a degenerate curve, the W/T type can change, separated by degenerate points that are neither wedges nor trisectors. Such points are *transition points*.

Figure 2 illustrates these concepts with one example tensor field. The combination of the L/P and W/T classifications of a degenerate point leads to four combinations: LW (green), LT (blue), PW (yellow), and PT (red). Note that along a degenerate curve, the L/P classification is constant while the W/T classification can change. Therefore, a degenerate curve can consist of either green and blue segments, or yellow and red segments, but not other combinations of colors. Linear transition points appear between green and blue segments, while planar transition points separate yellow and red segments.

The projected tensor fields onto the plane of repeating eigenvalues are shown along the degenerate curve. Notice that at a W type point, the projected pattern shows a wedge, while at a T type point, the projected pattern shows a trisector. At a transition point, the projected pattern shows neither a wedge pattern nor a trisector pattern.

#### *3.3 3D Linear Tensor Fields*

A 3D linear tensor field is a 3D symmetric tensor field whose tensor entries are linear functions of the XYZ coordinates of the points in the domain. It has the following form:

$$T(\mathbf{x}, \mathbf{y}, z) = T\_0 + \mathbf{x}T\_x + \mathbf{y}T\_y + zT\_z \tag{2}$$

where *T*0, *Tx* , *Ty* , and *Tz* are 3D symmetric tensors. A 3D linear tensor field has all the aforementioned properties of a general tensor field. However, there are some important properties specific to 3D linear tensor fields.

**Fig. 2** Along a degenerate curve, the projection of the tensor field onto the repeating planes can exhibit 2D degenerate patterns such as a trisector (**a**) or a wedge (**c**). Between segments of wedges (yellow) and trisectors (red), transition points can appear (**b**)

First, there are either two or four degenerate curves in a 3D linear tensor field [15]. Half of these curves are L, and the other half are P.

Second, there are either zero, two, four, six, or eight transition points in a 3D linear tensor field [14].

As pointed out [8], the set of degenerate points in a 3D linear tensor field (including their asymptotic limits at infinity) is homeomorphic to a topological circle (a loop). Figure 3 illustrates this bijective map. The loop is situated on the unit sphere where the location of a degenerate point on the loop corresponds to its unit dominant eigenvector. Since eigenvectors have a sign ambiguity, there are two such loops. Every degenerate point corresponds to exactly one pair of antipodal points on the sphere. While L/P cannot change along a degenerate curve, such a switch can occur at infinity, which two degenerate curves approach in opposite directions. The number of such switch points, called ∞ points, is the same as the number degenerate curves in the field. Note that ∞ points are neither L nor P. On the other hand, they can be either W or T. This leads to nine scenarios [14]: (1) two WW curves, (2) two WT curves, (3) two TT curves, (4) four WW curves, (5) two WW curves and two WT curves, (6) one WW curve, two WT curves, and one TT curve, (7) four WT curves, (8) two WT curves and two TT curves, and (9) four TT curves.

Along a *W W* or *T T* curve, there must be an even number of transition points, while on a *W T* curve there must exist an odd number of transition points.

The color of the sphere represents the sign of the discriminant of the projected tensor field onto the plane whose normal is the displacement vector of the point on the sphere from the center of the sphere. The color is cyan if the discriminant is positive (wedge type) and magenta if the discriminant is negative (trisector type). Note the W/T classification of a degenerate point must match the sign of the discriminant on the sphere. That is, green and yellow segments must appear in the cyan region, while blue and red segments must appear in the magenta region. A transition point must appear on the boundary between cyan and magenta regions.

**Fig. 3** This figure shows the parameterization of all degenerate points in a 3D linear tensor field by a topological circle. In the left are the degenerate curves of a 3D linear tensor field, and in the right is the topological circle superimposed on the unit sphere. Between the yellow and green segments are ∞ points. In addition, transition points occur either between yellow and red segments, or between green and blue segments. Moreover, we color a point in the sphere (right: representing a unit vector) cyan if the tensor projected onto the plane perpendicular to this vector has a wedge degenerate point. On the other hand, the point is colored magenta if the projected tensor field has trisector degenerate point. Notice that the transition points on the degenerate curves (between blue and green segments and between yellow and red segments) are precisely on the boundary between the cyan and magenta regions

#### **4 Bifurcations**

We now describe our findings of bifurcations in 3D linear tensor fields.

First, there can be either two or four degenerate curves. This means that some bifurcations can change the number of degenerate curves in the field. We refer to those bifurcations which reduce the number of degenerate curves from four to two as *degenerate curve removal* and those increase the number from two to four as *degenerate curve generation*.

Second, even when the number of degenerate curves does not change, the way the ∞ points are connected can change. We refer to this as *degenerate curve reconnection*, which is consistent with the topological editing operation of the same name from [5]. Note that two degenerate curves of opposite L/P types cannot be reconnected as this would generate a degenerate curve with changing L/P type. Consequently, when there are only two degenerate curves in the field, they cannot be connected. Degenerate curve reconnection can only occur for four degenerate curves.

Third, the number of transition points in the tensor field can change. Since this number must be even, we refer to such bifurcations as either *transition point pair cancellation* if the number is decreased by two or *transition point pair generation* if the number is increased by two.

Finally, even when the number of transition point does not change in a field, it is possible that some transition point has been moved from one degenerate curve to another. We refer to such a bifurcation as a *transition point relocation*.

Below we consider these bifurcations in the context of the nine scenarios described earlier.

#### *4.1 Degenerate Curve Removal and Generation*

Note that a degenerate curve generation bifurcation must be the inverse of a degenerate curve removal bifurcation, and vice versa. Consequently, we consider them together in this section.

Figure 4 lists all theoretically possible degenerate curve generation bifurcations and their inverse bifurcations. There is a total of nine scenarios, each of which is represented as an ellipse (topological disk) with the ∞ points marked along with their W/T types. A degenerate curve removal bifurcation must take a scenario with four degenerate curves (four ∞ points on the ellipse) to one with two degenerate curves (two ∞ points). Conversely, a degenerate curve generation bifurcation must take a scenario with two degenerate curves to one with four degenerate curves. In addition, when two degenerate curves are removed, two adjacent ∞ points are removed and two ∞ points remain. Each of the ∞ point is written as either W or T (called a symbol on the ellipse). This means that not any four-symbolled ellipse can connect with any two-symbolled ellipse. For example, while the box with four *W*'s can be mapped to either one with two *W*'s or one with one *W* and one *T* , it can *not* be connected with one with two *T* 's. Similarly, the box with one *W* and three *T* 's cannot be mapped to one with two *W*'s. Furthermore, when two symbols are removed, they need to be adjacent symbols. Consequently, it is not possible to connect the ellipse with two *W*'s and two *T* 's that are interleaved to one with two *W*'s or the one with two *T* 's.

These constraints give rise to a total of ten theoretically possible bifurcations, each of which is given as an edge in Fig. 4. The following figures provide examples of degenerate curve removal bifurcations.

In Fig. 5 (left), there are initially four TT curves, two of which are linear (blue) and two planar (red). After the bifurcation (right), the two linear degenerate curves become connected by a linear wedge segment (green), which replaces the lost planar

**Fig. 4** This diagram shows all the theoretically possible degenerate curve removal and generation bifurcations in a 3D linear tensor field

**Fig. 5** This figure shows a degenerate curve removal bifurcation from the case of four TT curves (left) to two TT curves (right). The top-middle shows the non-repeating eigenvector manifold for the left, while the bottom-middle shows the non-repeating eigenvector manifold for the right. The inverse bifurcation, a degenerate curve generation exists, which swaps the before and after scenarios. Note that in these bifurcations the number of segments do not change after the bifurcation

**Fig. 6** Another degenerate curve removal bifurcation takes the case of two WW curves and two WT curves (left) to two WW curves (right). Note that the number of segments is decreased by two after the bifurcation, i.e. two transition points are lost as a result of the bifurcation

TT curve (red). Furthermore, two transition points appear on the newly connected linear degenerate curve.

The middle column compares the two fields using their parameterization: (uppermiddle) corresponding to the field in the left, and (lower-middle) corresponding to the tensor field in the right. Through the bifurcation, two infinity points are replaced by two transition points, resulting in the switch of a red segment (middle-top) to a green segment (middle-bottom), which leads to the loss of two degenerate curves and the creation of two transition points.

Figure 6 provide an additional example of such types of bifurcations. Note that in this example, the loss of two degenerate curves also results in the loss of two transition points, i.e. the loss of two colored segments. This is a different way of achieving degenerate curve removal than the example bifurcation in Fig. 5, in which the number of segments does not change.

#### *4.2 Degenerate Curve Reconnection*

Degenerate curve reconnection bifurcations do not change the number of degenerate curves. Instead, they change how the ∞ points are connected. As mentioned earlier, two degenerate curves can be connected only if they have the same L/P type. When there are only two degenerate curves, one is L and the other P, which cannot be connected after reconnection. Consequently, degenerate curve reconnection bifurcations only occur for the case of four degenerate curves.

There is a total six scenarios with four degenerate curves, as shown in Fig. 7. Since degenerate curve reconnection bifurcations do not change the number and type of ∞ points, there is at least one theoretically possible bifurcation that connects a scenario to itself. Furthermore, the two scenarios with two *W*'s and two *T* 's are connected through reconnection bifurcations.

However, it is worth noting that when reconnection occurs, the ∞ points on the two degenerate curves to be reconnected cannot be arbitrarily connected after bifurcation. For an example, consider the case of two *W*'s and two *T* 's that appear on the ellipse in an alternating fashion (the lower-middle ellipse) in Fig. 7. Suppose the top and bottom segments are to be reconnected while the left and right segment remain. Then one cannot connect the upper-left *W* ∞ point with the lower-left *T* ∞ point, as it would cause the ellipse to be broken into two topological disks, a structurally unstable case. This means that it is *not* possible to have a reconnection bifurcation that takes this scenario to itself. Instead, the only way to reconnect is to connect the upper-left *W* ∞ point with the lower-right *W* ∞ point and connect the upper-right *T* ∞ point with the lower-left *T* ∞ point, leading to the scenario of two adjacent *W* and two adjacent *T* ∞ points on the ellipse (the upper-middle ellipse).

In fact, the above argument is true in general. That is, when reconnecting, the upper-left ∞ point must be connected to the lower-right ∞ point, and the upperright ∞ point must be connected to the lower-left ∞ point. This analysis shows that there are five degenerate curve reconnection bifurcations that takes one scenario to itself, and there are two more that occur between two different scenarios (the upper-middle ellipse and the lower-middle ellipse).

Figure 8 shows the reconnection in the scenario of four WW curves, in which two linear degenerate curves (green) are reconnected, resulting in two linear degenerate curves. The two planar degenerate curves (yellows) are not reconnected. This bifurcation does not increase or decrease the number of transition points.

Figure 9 presents another degenerate curve reconnection bifurcation in which two planar WT degenerate curves are reconnected into one planar WW degenerate curve and one planar TT degenerate curve. As a result of this bifurcation, two transition points are removed, which leads to a decrease in the total number of transition points in the field. Notice the order of colored segments (middle column of Fig. 9) is changed due to the reconnection.

**Fig. 7** This diagram shows all the possible degenerate curve reconnection bifurcations in a 3D linear tensor field

**Fig. 8** This figure shows a degenerate curve reconnection bifurcation from the case of four WW curves (left) to four WW curves (right). The number of transition points does not change after the bifurcation

**Fig. 9** Another degenerate curve reconnection bifurcation takes the case of one WW curve, 2WT curves, and one TT curve (left) to four WT curves (right). In this case, the number of degenerate curve bifurcation is decreased by two

#### *4.3 Transition Point Pair Cancellation and Generation*

The total number of transition points in a 3D linear tensor field is an even number between zero and eight. That means that any fundamental bifurcation that changes the number of total transition points must either increase it by two or decrease it by two. The former is the *degenerate curve pair generation* bifurcation, and the latter *degenerate curve pair cancellation* bifurcation. They are inverse bifurcations and thus discussed together in this section.

Two examples of transition point pair cancellation bifurcations are shown in Figs. 11 and 12, for the cases of two degenerate curves and four degenerate curves, respectively. The transition point pair cancellation and generation bifurcations do not change the scenario. Consequently, for each of the nine scenarios, there is a bifurcation that takes this scenario to itself, as shown in Fig. 10. There is a total of nine transition point pair cancellation bifurcations and nine transition point pair generation bifurcations.

**Fig. 10** This diagram shows all the possible transition pair cancellation and generation bifurcations in a 3D linear tensor field for one degenerate curve

**Fig. 11** A transition point pair cancellation bifurcation that removes two transition points and one blue segment from one of the two degenerate curves (left), resulting in the same number of degenerate curves with two fewer transitions points (right)

**Fig. 12** A transition point pair cancellation operation (from to right) occurs in the case of four degenerate curves

**Fig. 13** This diagram shows all the possible transition point relocation bifurcations in a 3D linear tensor field. Note that these bifurcations have the effect of switching one W symbol to one T symbol, or vice versa. Consequently, there are only eight possible such bifurcations, all observed in our experiment

#### *4.4 Transition Point Relocation*

Transition points, along with ∞ points, divide the degenerate curves into segments of pure L/P and W/T types. While the total number of transition points in a 3D linear tensor field is eight, its distribution is not uniform among the degenerate curves.

*Transition point relocation* refers to moving one transition point from one degenerate curve to another degenerate curve of the opposite L/P type. From the viewpoint of the parameterization, this is the swap of the positions of the transition point with one adjacent infinity point on the ellipse in the counterclockwise order. This results in one fewer transition point on the original degenerate curve and one more transition point on the new degenerate curve but keeps the total number of transition points constant. Moreover, the W/T type of the segment also changes. Consequently, the segment between the transition point and infinity point will change its L/P type but maintains its W/T type, thus will change colors between red and green, or between yellow and blue. Figures 14 and 15 provide two examples of this type of bifurcations that involve respectively two degenerate curves and four degenerate curves.

**Fig. 14** This figure shows a transition point relocation bifurcation from the case of two WW curves (left) to two WT curves (right)

**Fig. 15** Another transition point relation bifurcation takes the case of four TT curves (left) to two TT and two WT curves (right)

Figure 13 shows the possible transition point relation bifurcations. Note that these bifurcations result in a switch of a W infinity point to a T type of infinity point, or vice versa. Note that each edge in this diagram corresponds to two transition point relocation bifurcations. As there are eight edges in the diagram, there is a total of 16 bifurcations, all of which have been observed during experiment.

#### **5 Conclusion**

Degenerate curves and transition points play an important role in describing the behavior of a 3D tensor field. In this book chapter, we study possible bifurcations for any 3D linear tensor field.

This is done based on the realization that there is a total of nine scenarios and four types bifurcations. Theoretically, we have identified ten degenerate curve removal bifurcations, ten degenerate curve generation bifurcations, seven degenerate curve reconnection bifurcation, nine transition point pair cancellation bifurcations, nine transition point pair generation bifurcations, and 16 transition point relation bifurcations. This leads to a total of 61 topological bifurcations for 3D linear tensor fields, all of which have been observed.

There are a number of future research directions.

First, the number of transition points can further divide each scenario into subscenarios. For example, for the scenario of two *W W* curves, there can be zero, two, four, six, or eight transition points. Furthermore, transition points are not evenly distributed on degenerate curves. This leads to multiple cases per scenario. Enumerating of all bifurcations from any sub-scenario to any other sub-scenario can lead to deeper insight about the topology of 3D symmetric tensor fields.

Second, the bifurcations among 3D tensor fields can be used to generate a multiscale framework for the topological analysis of these objects, such as degenerate curve clustering and removal. Such a framework has the potential of enabling users to inspect the topology of their tensor field data at various scales. We plan to investigate this possibility in our future research.

Third, simulation data sets usually involve *piecewise linear tensor fields*, which can have potentially more types of bifurcations than linear tensor fields that we investigate in this paper. This is a future direction that can lead to more practical applications for tensor field topology.

Fourth, neutral surfaces are the other important constituent of 3D tensor field topology. Enumerating the scenarios of neutral surfaces and bifurcations of these different scenarios can provide insight into the structure of a tensor field. We will explore this direction.

**Acknowledgments** We wish to thank our anonymous reviewers for their valuable feedback and suggestions. This research is partially supported by NSF awards (# 1566236) and (#1619383).

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Continuous Histograms for Anisotropy of 2D Symmetric Piece-Wise Linear Tensor Fields**

**Talha Bin Masood and Ingrid Hotz**

**Abstract** In this chapter we present an accurate derivation of the distribution of scalar invariants with quadratic behavior represented as continuous histograms. The anisotropy field, computed from a two-dimensional piece-wise linear tensor field, is used as an example and is discussed in all details. Histograms visualizing an approximation of the distribution of scalar values play an important role in visualization. They are used as an interface for the design of transfer-functions for volume rendering or feature selection in interactive interfaces. While there are standard algorithms to compute continuous histograms for piece-wise linear scalar fields, they are not directly applicable to tensor invariants with non-linear, often even non-convex behavior in cells when applying linear tensor interpolation. Our derivation is based on a sub-division of the mesh in triangles that exhibit a monotonic behavior. We compare the results to a naïve approach based on linear interpolation on the original mesh or the subdivision.

#### **1 Introduction**

Iso-contours or iso-surfaces of scalar fields, defined as the pre-image of an iso-value, play a central role in visualization. There is a large body of work centered around iso-contour computation and analysis with many applications. A prominent example is the *contour tree*, which keeps track of topological changes of iso-contours when changing the iso-value [5]. It provides a structural overview of the data, see Sect. 2.3 for more details. Complementary information is provided by the *continuous histogram*, which is an extension of the discrete histogram to encode the distribution of the scalar values of continuous functions, see Sect. 2.1. It can serve as a valuable quantitative signature [2] of a data set. Both structures are frequently used for inter-

T. B. Masood (B) · I. Hotz

Department of Science and Technology (ITN), Linköping University, Norrköping, Sweden e-mail: talha.bin.masood@liu.se

I. Hotz e-mail: ingrid.hotz@liu.se

<sup>©</sup> The Author(s) 2021

E. Özarslan et al. (eds.), *Anisotropy Across Fields and Scales*,

Mathematics and Visualization, https://doi.org/10.1007/978-3-030-56215-1\_3

**Fig. 1** An example demonstrating the behavior of the anisotropy field in a simple tensor field given on a triangulated domain. The ellipses at the vertices represent the tensors defining the field. The total rotation of the eigenvector when moving once around the domain is 2π. The white lines represent iso-contours. **a** The anisotropy is linearly interpolated within the original mesh. It only has one minimum and no zeros or isotropic points. **b** The consistent anisotropy assuming linear interpolation of tensor components has three minima and two isotropic points. The respective join trees for the anisotropy fields are shown next to the meshes. All zero-leaves, marked in red, in the tree represent isotropic points of the tensor field. **c** The histograms for anisotropy using the two approaches in comparison

action and filtering, for example, to determine interesting iso-values or for transfer function design [19]. Histograms have also been applied for tensor field visualization and exploration to display the distribution of scalar invariants derived from tensor fields [13]. This distribution can also be understood as statistics of iso-contours. Note that we use the term *iso-contour* to refer to the complete level set of a 2D scalar field for a given iso-value, not one of its connected component which is the case in some of the related contour tree literature.

There are many standard algorithms to compute and analyze iso-contours, which are mostly based on piece-wise linear or multi-linear behavior. This assumption is, however, often violated for tensor invariants when using component-wise tensor interpolation, for example, various anisotropy definitions or the determinant of the tensor. An illustrative example of the behavior of the contours of the anisotropy in piece-wise linear tensor field in comparison to a linear approximation of the anisotropy is shown in Fig. 1. It is important to note that, there is no continuous tensor field interpolating the tensors at the vertices exhibiting the piece-wise linearly interpolated anisotropy in Fig. 1a when assuming that the tensor field is fixed on the boundary of the domain. The number of rotations of the eigenvector field when moving once around the domain (Poincaré index) is a topological invariant that determines the minimum number of isotropic points in the field. In this example, there is a total rotation of 2π of the eigenvectors, which results in two zeros in the anisotropy field. These zeros in the anisotropy field are not present in the linear anisotropy field corrupting its histogram, Fig. 1c. Therefore, it is important to consider an anisotropy field which is consistent with the chosen tensor field interpolation when analyzing its behavior. Here we have chosen a piece-wise linear interpolation of tensor components as an example. In the following we refer to the anisotropy field resulting from this interpolation as the consistent anisotropy. In this chapter, we present the derivation of an exact closed-form expression for *continuous histograms* for quadratic tensor invariants that is consistent with a piece-wise linear interpolation of a tensor field given over a two-dimensional triangulated domain.

Here, we are especially interested in the analysis of the anisotropy, an important characteristic in many applications, defined as the difference between major and minor eigenvalue. While there are many other definitions of anisotropy, this version has a distinguished meaning when analyzing the stability of the critical points and isotropic points, when perturbing the tensor field using the Frobenius norm to measure the deviation of the tensor field [18]. Further, it is frequently used in mechanical engineering applications to analyze the failure of materials. The *join tree*, a subset of the contour tree, of this anisotropy can be used for a topology-preserving simplification of 2d tensor fields [11].

Our main contributions are:


The relevant context and related work are summarized in Sect. 2, then an overview of the problem and its solution is given in Sect. 3. Fundamental definitions and notations are introduced in Sect. 4. A general expression for the anisotropy is developed in Sect. 5 and used in Sect. 6 to subdivide the domain in monotonous triangles. Section 7 derives the accurate histogram. After some notes on the implementation in Sect. 7.1 the results are discussed in Sect. 8.

#### **2 Context and Related Work**

In this section, we first summarize the development of the concept of histograms from a signature of discrete data to continuous fields. Then we summarize some aspects of tensor interpolation and invariants that motivated our work.

#### *2.1 Continuous Histograms*

Originally, *histograms* have been developed for discrete data. They go back to Pearson and refer to a graphical representation displaying the frequency of the data items as bars over the data range [10]. As such, a histogram provides a summary of the data and they are also used to compare and characterize data sets. Formally, the histogram for a discrete set of data values *F* = { *fi*|*i* ∈ 1, ··· , *m*}is a one-dimensional bar chart displaying the frequency of each distinct element *f* ∈ *F* as a bar with height

$$h(f) = \sum\_{i} \delta(f - f\_i).$$

where δ is the delta function which is equal to one if *f* = *fi* and zero otherwise. A related concept applicable to ordered data is the *cumulative histogram* which counts the cumulative number of data values smaller than a given value *f* ∈ *F*. This results in values

$$c(f) = \sum\_{f\_i \le f} h(f\_i).$$

Today, the term is often not only used for the graphical representation but also for the concept capturing the distribution of function values by binning the function range and counting the frequency of data samples within these bins. The resulting histograms, however, are strongly dependent on the binning size and the sampling strategy of the function in the domain. The continuous nature of the function between the sample points is not represented. For these reasons several concepts to extend histograms to continuous functions have been proposed resulting in a distribution of the function values. Bajaj et al. [2] have introduced the *contour spectrum* as a datasignature to find interesting iso-values for volume rendering. The contour spectrum assigns the geometric measures of the contour length or surface area to each scalar value. In addition, they consider areas and volumes of regions below or above a given iso-value. Bajaj et al. also propose a method to compute these values exactly under the assumption of a piece-wise linear interpolation and a piece-wise constant gradient. Later Carr et al. [4] investigated the relation between histograms and the contour spectrum based on a nearest-neighbor interpolation. Scheidegger et al. [16] completed this work and introduced a natural generalization of histograms to the continuous setting as the distribution given by the contour spectrum weighted by the *local isosurface density* expressed as the inverse gradient magnitude. Given a scalar field *f* and scalar value *t* the distribution is given as

$$\pi\_f(t) = \int\_{f^{-1}(t)} |\nabla f(\mathbf{x})|^{-1} dS$$

For the derivation of the distributions, they refer to Coarea formula that provides a relationship between the sum of area integrals and a global volume integral as used by Mullen et al. [14]. For piece-wise linear functions, this yields a more or less good approximation of the real distribution.

Similar concepts have also been considered for multivariate fields, for example, generalizations of scatterplotts [1] and parallel coordinates [9] to continuous data. In both cases, the mathematical model is based on mass conservation of a mapping from the function range to the scatterplot space and the parallel coordinates space respectively. The continuous histogram is a special case considering a onedimensional range. While the mathematical model is generic, the proposed computational algorithms are limited to specific interpolation models in the spatial domain. The algorithm by Bachthaler et al. [1] assumes a linear interpolation in tetrahedral cells.

In our work, we focus on invariants with a quadratic behavior and propose an exact solution pursuing a slightly different approach than most of the previous methods. Instead of directly generalizing the concept of histograms, we start with the generalization of the cumulative histogram. This approach requires neither surface area/length of iso-contours nor gradient approximations and thus makes it possible to compute an accurate distribution for more complex interpolation schemes, demonstrated here for quadratic interpolations. The continuous histogram or scalar value distribution then follows as a derivative of the cumulative histogram.

#### *2.2 Notes on Tensor Field Interpolation*

In this section, we briefly summarize some notes related to tensor field interpolation without going much into detail. This has been a frequently discussed topic in many applications and it is agreed that this is a challenging topic.

While many theories and visualization models assume data given as fields on continuous domains, the data-reality are data sets sampled at discrete locations. Depending on the origin of the data this can be voxel-based data from imaging or meshes for simulation data. In any case, it is necessary to approximate and reconstruct the field from these samples. Thereby the most commonly used methods for tensor fields are based on a component-wise interpolation. Similar to other applications and other data types these are linear, bi-, or tri-linear interpolation depending on the grid type. Concerning such interpolations, a frequently discussed topic is the nonlinear dependency of most tensor invariants on the tensor components [12]. There have been many methods proposed which explicitly try to preserve certain tensor characteristics [3, 17]. Recently there appeared a survey on interpolation methods for positive definite tensors [8].

We will, however, stay with the most simple interpolation methods, linear component-wise interpolation within tetrahedra which is also often the basis for finite element simulations. The non-linear dependence of the tensor invariants can lead to a non-convex behavior of the invariants inside the cells. An example we are especially interested in is the behavior of the anisotropy, defined as the difference between the maximum and minimum eigenvalue, which is typically decreased inside the cells. Sometimes this is referred to as "swelling-artifact", but we rather argue that this is a fundamental property of tensor behavior. It reflects the fact that the anisotropy is linked to the directional behavior of the tensor. This is expressed by the fact that the anisotropy is always zero at degenerate points. These are points where the eigenvalues are the same and there is no uniquely determined eigenvector direction. The number of degenerate points is a topological invariant of the field when fixing the boundary and thus determines the minimum number of zeros in the anisotropy field, see Fig. 1 for an example. Therefore the analysis of the anisotropy field must be handled consistently with the chosen interpolation method for the tensor field. This concerns the computation of iso-contours, the contour tree and also histograms.

For our consideration in this work, we assume that we have a continuous tensor field given on a two-dimensional triangulated domain. We further assume a piecewise linear, component-wise linear interpolation of the tensor field within these triangles. The anisotropy then has a quadratic behavior in the triangle. One can observe similar behavior for other invariants, for example, the determinant too. All the above-proposed methods for histogram computations fail when the interpolation of the data is not convex. Similarly, typical iso-contour or contour tree computation is also based on convex interpolation.

#### *2.3 Contour Trees, a Topological Summary of Scalar Functions*

The contour tree keeps track of the changes of sub- and super-level sets of a function, bounded by iso-contours, and thus provides a valuable summary of the function, compare Fig. 2. It can be assembled from the *join tree* tracing the changes in isocontours when the function value is increased from −∞ to ∞ and the *split tree* one where the function value is decreased from ∞ to −∞. In more detail, the join tree tracks the creation and merging of sub-level sets, which are recorded in a tree. At the local minima of the function, new branches are created. As the function value increases, branches are extended and merged at saddle points where two sub-level

**Fig. 2** For the scalar field on the left, the join tree, split tree and the contour tree are shown

sets merge. The global maximum of the function becomes the root of the tree. The split tree is constructed similarly while traversing the field from ∞ to −∞. The most commonly used algorithm for contour tree computation assumes a piecewise linear interpolation of the scalar field where all critical points are located at the cell vertices [5]. In the case of the anisotropy, we are especially interested in the join tree, a sub-tree of the contour tree, since many of the minima correspond to the degenerate points in the tensor field. The join tree can be used to quantify the stability of these points [18]. For consistent results, assuming a specific interpolation, a join tree computation based on the quadratic behavior or the anisotropy is essential.

#### **3 Problem Statement and Solution Overview**

Given a tensor field sampled over a 2D triangular mesh, our goal is to compute the accurate contour-tree and histogram of the tensor anisotropy consistent with linear interpolation of tensor components. Anisotropy is a quadratic function under this interpolation.

Figure 3 gives an overview of our solution approach. For each triangle in the input mesh, we first determine the coefficients of the quadratic function for the anisotropy based on tensor values at the triangle vertices. We show that anisotropy is a special quadratic function which either has a single minimum equal to zero and elliptical iso-contours, or in a degenerate case, minima along a line and the iso-contours are

**Fig. 3** Solution overview: First, the coefficients of the quadratic function are determined. Then a transformation is applied such that the minimum is at the origin and the iso-contours are circular, followed by a sub-division into monotonous triangles. For each monotonous triangle, the cumulative anisotropy histogram is computed using sub-level set areas, which are added to get the cumulative histogram for the original triangle. Finally, the derivative of the cumulative histogram yields the anisotropy histogram

pairs of parallel lines, refer to Sect. 5. The first step is the subdivision of the triangle into monotonous sub-triangles, which is the basis for the iso-contour extraction, the contour tree computation and the derivation of the histogram.We obtain the histogram as the derivative of the cumulative histogram, which is computed first from the area of the sub-level set for the isovalues. To compute the explicit area of the sub-level sets we apply a linear transformation to the coordinate system such that elliptical iso-contours become circular centered at the origin, refer to Sect. 5.1.

#### **4 Background and Notations**

In this section, we provide the notations and a brief mathematical background required for the proposed method discussed in sections later. First, in Sect. 4.1 we set up the notations for tensors and the invariant of interest to us that is tensor anisotropy. Then in Sect. 4.2, we describe the barycentric interpolation within a triangle, since this approach is used for linear interpolation of tensors within a triangle. Lastly, in Sect. 4.3, we discuss the general bi-variate quadratic function, its critical point, and the shape of its iso-contours.

#### *4.1 Second Order Symmetric Tensors and Anisotropy*

For a second order symmetric tensor, *T* = *e f f g* , the two eigenvalues are:

$$
\lambda, \mu = \frac{1}{2}(e+\mathfrak{g}) \pm \sqrt{\frac{(e+\mathfrak{g})^2}{4} - (\mathfrak{eg} - f^2)}.
$$

Depending on the application different measures for anisotropy are used. For positive definite tensors, relative measures like the fractional anisotropy are common. In mechanical engineering a typical measure is the difference between the maximum and the minimum eigenvalues α(*T* ) = λ − μ = (*e* + *g*)<sup>2</sup> − 4(*eg* − *f* <sup>2</sup>). It has been shown that this value is also related to the stability of degenerate points in tensor field topology [11] and is the measure we are mostly interested in. In the following, we will however consider the squared value of α(*T* ), which has the same topological characteristics but simplifies the computations a lot.

$$\begin{split} \nu(T) &= \alpha^2(T) = (\lambda - \mu)^2 \\ &= \left(e - g\right)^2 + 4f^2 \end{split} \tag{1}$$

#### *4.2 Barycentric Coordinates and Piece-Wise Linear Interpolation*

For all our considerations in this paper we use a component-wise linear interpolation of the tensor field. We use barycentric coordinates for interpolation in the triangle. Since we require an explicit representation of the the tensor field for the derivation of the exact histogram we briefly review the main definitions and formulas in this section.

Refer to Fig. 4, for a triangle with vertices *p*<sup>1</sup> = (*x*1, *y*1), *p*<sup>2</sup> = (*x*2, *y*2) and *p*<sup>3</sup> = (*x*3, *y*3), the barycentric coordinates (β1, β2, β3) of an arbitrary point *p* = (*x*, *y*) within a triangle are given by

$$\begin{aligned} \beta\_1 &= \frac{(\mathbf{y}\_2 - \mathbf{y}\_3)(\mathbf{x} - \mathbf{x}\_3) + (\mathbf{x}\_3 - \mathbf{x}\_2)(\mathbf{y} - \mathbf{y}\_3)}{(\mathbf{y}\_2 - \mathbf{y}\_3)(\mathbf{x}\_1 - \mathbf{x}\_3) + (\mathbf{x}\_3 - \mathbf{x}\_2)(\mathbf{y}\_1 - \mathbf{y}\_3)}, \\ \beta\_2 &= \frac{(\mathbf{y}\_3 - \mathbf{y}\_1)(\mathbf{x} - \mathbf{x}\_3) + (\mathbf{x}\_1 - \mathbf{x}\_3)(\mathbf{y} - \mathbf{y}\_3)}{(\mathbf{y}\_2 - \mathbf{y}\_3)(\mathbf{x}\_1 - \mathbf{x}\_3) + (\mathbf{x}\_3 - \mathbf{x}\_2)(\mathbf{y}\_1 - \mathbf{y}\_3)}, \\ \beta\_3 &= \frac{(\mathbf{y}\_1 - \mathbf{y}\_2)(\mathbf{x} - \mathbf{x}\_2) + (\mathbf{x}\_2 - \mathbf{x}\_1)(\mathbf{y} - \mathbf{y}\_2)}{(\mathbf{y}\_2 - \mathbf{y}\_3)(\mathbf{x}\_1 - \mathbf{x}\_3) + (\mathbf{x}\_3 - \mathbf{x}\_2)(\mathbf{y}\_1 - \mathbf{y}\_3)}. \end{aligned}$$

We assume that some scalar function *s* is sampled at the vertices of the triangle with scalar values *s*1, *s*<sup>2</sup> and *s*<sup>3</sup> at vertices *p*1, *p*<sup>2</sup> and *p*3. Then, the scalar value *s* at an arbitrary point *p* = (*x*, *y*) within the triangle can be determined as

$$s(\mathbf{x}, \mathbf{y}) = \beta\_1 \mathbf{s}\_1 + \beta\_2 \mathbf{s}\_2 + \beta\_3 \mathbf{s}\_3 \dots$$

This function *s*(*x*, *y*) is linear in *x* and *y*, and can be alternatively written as

$$s(x, \mathbf{y}) = s\_x \mathbf{x} + s\_y \mathbf{y} + s\_c \mathbf{y}$$

**Fig. 4** Barycentric coordinates and interpolation. The point *p* within the triangle *p*<sup>1</sup> *p*<sup>2</sup> *p*<sup>3</sup> can be represented using barycentric coordinates (β1, β2, β3), where β<sup>1</sup> = Area(*pp*<sup>2</sup> *p*3)/ Area(*p*<sup>1</sup> *p*<sup>2</sup> *p*3), β<sup>2</sup> = Area(*pp*<sup>1</sup> *p*3)/Area(*p*<sup>1</sup> *p*<sup>2</sup> *p*3) and β<sup>3</sup> = Area(*pp*<sup>1</sup> *p*2)/Area (*p*<sup>1</sup> *p*<sup>2</sup> *p*3). Let *s* be some scalar quantity sampled on the vertices *p*1, *p*<sup>2</sup> and *p*<sup>3</sup> as *s*1,*s*<sup>2</sup> and *s*<sup>3</sup> respectively. Then the value of *s* at any point *p* inside the triangle is given by *s*(*p*) = β1*s*<sup>1</sup> + β2*s*<sup>2</sup> + β3*s*<sup>3</sup>

with

$$\begin{aligned} s\_x &= \frac{(y\_2 - y\_3)s\_1 + (y\_3 - y\_1)s\_2 + (y\_1 - y\_2)s\_3}{(y\_2 - y\_3)(x\_1 - x\_3) + (x\_3 - x\_2)(y\_1 - y\_3)} \\ s\_y &= \frac{(x\_3 - x\_2)s\_1 + (x\_1 - x\_3)s\_2 + (x\_2 - x\_1)s\_3}{(y\_2 - y\_3)(x\_1 - x\_3) + (x\_3 - x\_2)(y\_1 - y\_3)} \\ s\_c &= \frac{(x\_2y\_3 - x\_3y\_2)s\_1 + (x\_3y\_1 - x\_1y\_3)s\_2 + (x\_1y\_2 - x\_2y\_1)s\_3}{(y\_2 - y\_3)(x\_1 - x\_3) + (x\_3 - x\_2)(y\_1 - y\_3)} \end{aligned}$$

#### *4.3 Bivariate Quadratic Functions and Their Critical Points*

As the anisotropy, as defined in Eq. (1), is a quadratic function we summarize some general facts about quadratic functions and define the notation that we will use later in the paper. In the following we assume that the quadratic function is not identical to zero.

A general quadratic function in two variables can be written as *s*(*x*, *y*) = *Ax* <sup>2</sup> + *Bxy* + *Cy*<sup>2</sup> + *Dx* + *E y* + *F* or in matrix form as

$$\mathbf{s}(\mathbf{x}, \mathbf{y}) = (\mathbf{x}, \mathbf{y}) \cdot \mathbf{M} \cdot \begin{pmatrix} \mathbf{x} \\ \mathbf{y} \end{pmatrix} + T \cdot \begin{pmatrix} \mathbf{x} \\ \mathbf{y} \end{pmatrix} + F \tag{2}$$

with *M* = *A B*/2 *B*/2 *C* and *T* = *D E* . The critical point of the scalar function *s* is a point *p* = (*x*, *y*) where the gradient ∇*s*(*x*, *y*) = 0 is zero. The partial derivatives of *s*(*x*, *y*) with respect to the two variables is:

$$
\partial \mathbf{s} / \partial \mathbf{x} = 2A\mathbf{x} + B\mathbf{y} + D \tag{3}
$$

$$
\partial \mathbf{s} / \partial \mathbf{y} = \mathcal{D} \mathbf{C} \mathbf{y} + B \mathbf{x} + E \tag{4}
$$

The location of the critical point *pc* = (*xc*, *yc*) of *s* can be obtained after solving the linear equations ∂*s*/∂*x* = 0 and ∂*s*/∂*y* = 0 using Eqs. (3) and (4). The resulting coordinates are

$$\chi\_c = \frac{-2CD + BE}{4AC - B^2}, \qquad \chi\_c = \frac{-2AE + BD}{4AC - B^2}.\tag{5}$$

The function *s* and its critical point can be classified based on the sign of the determinant of the Hessian, *H*

$$H = 4AC - B^2.\tag{6}$$

If *H* > 0, then the critical point is either a maximum or minimum and the isocontours are ellipses, compare Fig. 5a, b. The type of the critical point depends on

**Fig. 5** The possible cases for a general quadratic function. **a** The critical point is a minimum and the iso-contours are elliptical. **b** The critical point is a maximum and the iso-contours are ellipses. **c** The critical point is a saddle and the iso-contours are hyperbolas. **d** The critical point does not exist and the function has a minimum value along a line, while the iso-contours are a pair of parallel lines. **e** The other case when the iso-contours are a pair of parallel lines, again the critical point does not exist but the function has a maximum value along a line. **f** The case when the critical point does not exist and the iso-contours are parabolic. We show that only the cases (**a**) and (**d**) are possible for the quadratic function for tensor anisotropy, the other four cases shown here are not possible

the sign of *A* and *C*. If *A*,*C* > 0, then the critical point is a minimum, otherwise it is a maximum. In the other case when *H* < 0, then the critical point is a saddle and the iso-contours are hyperbolas.

For the case when *H* = 0, there are no isolated critical points and the iso-contours are either parabolas, parallel or coincident lines. Note that it is possible that there are degenerate lines. The type of the iso-contour when *H* = 0 depends on the invariant *I* defined as

$$I = BDE - AE^2 - CD^2.\tag{7}$$

If *I* = 0, the iso-contours are parabolic, compare Fig. 5f, otherwise they are pair of parallel lines, compare Fig. 5d, e.

#### **5 Anisotropy for 2D Piece-Wise Linear Tensor Fields**

In this section, we have a closer look at the anisotropy function for a linear tensor field. The main observation is in the non-degenerate case, that the anisotropy always has exactly one minimum with a value of zero. This corresponds to the fact that the tensor field always has exactly one critical point. All iso-contours are ellipses.

Given a triangle with vertices *p*<sup>1</sup> = (*x*1, *y*1), *p*<sup>2</sup> = (*x*2, *y*2) and *p*<sup>3</sup> = (*x*3, *y*3). Let the tensors at these vertices be *T*1, *T*<sup>2</sup> and *T*3, respectively. Let the components of the tensors be

$$T\_1 = \begin{pmatrix} e\_1 \ f\_1 \\ f\_1 \ g\_1 \end{pmatrix}, \quad T\_2 = \begin{pmatrix} e\_2 \ f\_2 \\ f\_2 \ g\_2 \end{pmatrix}, \quad T\_3 = \begin{pmatrix} e\_3 \ f\_3 \\ f\_3 \ g\_3 \end{pmatrix}. \tag{8}$$

The barycentric coordinates can be used for linear interpolation of the tensor field within the triangle. Tensor *T* at an arbitrary point *p* = (*x*, *y*) within the triangle can be found as:

$$T(p) = T(\mathbf{x}, \mathbf{y}) = \begin{pmatrix} e(\mathbf{x}, \mathbf{y}) \ f(\mathbf{x}, \mathbf{y}) \\ f(\mathbf{x}, \mathbf{y}) \ g(\mathbf{x}, \mathbf{y}) \end{pmatrix} \tag{9}$$

As described earlier in Sect. 4.2, the tensor components *e*, *f* and *g* linearly interpolated within the triangle as can be written as

$$e(\mathbf{x}, \mathbf{y}) = e\_{\mathbf{x}}\mathbf{x} + e\_{\mathbf{y}}\mathbf{y} + e\_{\mathbf{c}},\tag{10}$$

$$f(\mathbf{x}, \mathbf{y}) = f\_{\mathbf{x}}\mathbf{x} + f\_{\mathbf{y}}\mathbf{y} + f\_{\mathbf{c}},\tag{11}$$

$$\mathbf{g}(\mathbf{x}, \mathbf{y}) = \mathbf{g}\_x \mathbf{x} + \mathbf{g}\_y \mathbf{y} + \mathbf{g}\_c. \tag{12}$$

The function, we are interested in is the anisotropy ν, the explicit expression for which can be obtained by substituting the linear expressions for tensor components in Eq. (1) yielding

$$\begin{split} \nu(T) = & \left( (e\_x - g\_x)^2 + 4f\_x^2 \right) \mathbf{x}^2 + 2 \left( (e\_x - g\_x)(e\_\mathbf{y} - g\_\mathbf{y}) + 4f\_x f\_\mathbf{y} \right) \mathbf{x} \mathbf{y} \\ &+ \left( (e\_\mathbf{y} - g\_\mathbf{y})^2 + 4f\_\mathbf{y}^2 \right) \mathbf{y}^2 + 2 \left( (e\_x - g\_x)(e\_c - g\_c) + 4f\_x f\_c \right) \mathbf{x} \\ &+ 2 \left( (e\_\mathbf{y} - g\_\mathbf{y})(e\_c - g\_c) + 4f\_\mathbf{y} f\_c \right) \mathbf{y} + \left( (e\_c - g\_c)^2 + 4f\_c^2 \right) . \end{split} \tag{13}$$

Comparing Eq. (13) with the general quadratic Eq. (2), we obtain the coefficients of the quadratic function in dependence of the tensor components:

$$A = \left( (e\_x - g\_x)^2 + 4f\_x^2 \right) \ge 0,\tag{14}$$

$$B = 2\left( (e\_x - g\_x)(e\_\mathbf{y} - g\_\mathbf{y}) + 4f\_x f\_\mathbf{y} \right),\tag{15}$$

$$C = \left( (e\_\mathbf{y} - g\_\mathbf{y})^2 + 4f\_\mathbf{y}^2 \right) \ge 0,\tag{16}$$

$$D = 2\left( (e\_x - g\_x)(e\_c - g\_c) + 4f\_x f\_c \right),\tag{17}$$

$$E = 2\left( (e\_\mathbf{y} - \mathbf{g}\_\mathbf{y})(e\_c - \mathbf{g}\_c) + 4f\_\mathbf{y}f\_c \right),\tag{18}$$

$$F = \left( (e\_c - g\_c)^2 + 4f\_c^2 \right). \tag{19}$$

Substituting the above equations in Eq. (6), we derive the determinant of Hessian for the anisotropy to be (for derivation refer to Appendix A):

$$H = 16\left(f\_x(e\_\mathbf{y} - \mathbf{g}\_\mathbf{y}) - f\_\mathbf{y}(e\_\mathbf{x} - \mathbf{g}\_\mathbf{x})\right)^2 \ge 0. \tag{20}$$

From Eq. (20), we can conclude that the iso-contours of anisotropy function are never hyperbolic. In most cases, when *H* is strictly greater than 0, the contours are elliptical. Moreover, in that scenario, from Eqs. (14) and (16), we can deduce that the type of the critical point is always a minimum.

Appendix A contains the detailed analysis of ν. We show that the bi-variate quadratic function ν is a special function that has elliptical iso-contours or in some cases a pair of parallel lines, ruling out the possibility of hyperbolic or parabolic iso-contours.

#### *5.1 Field Normalization Using Coordinate Transformations*

In the following, we derive a coordinate transformation such that the bivariate quadratic scalar function determined for a triangle has a standard format. This allows for the application of a unified strategy for further analysis.

Equation (13) gives the expression of anisotropy in general bivariate quadratic form. We have also established that iso-contours of this function will are elliptical. Therefore we first transform the coordinates such that elliptical iso-contours are centered at the origin and aligned to the axes. This can be achieved by applying a translation and rotation, both rigid-body transformations preserving areas. Then, a transformation is applied such that iso-contours of the bivariate quadratic function become circles rather than ellipses. This can be achieved by applying a non-uniform scaling, a linear transformation which distorts the area by a constant factor given by the determinant of the transformation matrix. The approach described above is illustrated in Fig. 6.

Refer to Eqs. (13)–(19) for the detailed expression of anisotropy and refer to Eq. (2) for the general quadratic equation in matrix form. After translation such that the minimum *pc* = (*xc*, *yc*) falls into the origin, the expression for anisotropy can be written as

$$\nu(\mathbf{x}\_t, \mathbf{y}\_t) = (\mathbf{x}\_t, \mathbf{y}\_t) \cdot \mathcal{M} \cdot \begin{pmatrix} \mathbf{x}\_t \\ \mathbf{y}\_t \end{pmatrix} + F\_t \tag{21}$$

with the translated coordinates *xt* = *x* − *xc*, *yt* = *y* − *yc* and *Ft* = *F* + (*BDE*−*AE*2−*C D*2) <sup>4</sup>*AC*−*B*<sup>2</sup> . Using Eqs. (14)–(19), it can be shown that *Ft* <sup>=</sup> 0 for <sup>ν</sup>. This implies that minimum value of ν is zero at the critical point and this point is a degenerate point in the tensor field. Equation (21) becomes

$$\nu(\mathbf{x}\_t, \mathbf{y}\_t) = (\mathbf{x}\_t, \mathbf{y}\_t) \cdot M \cdot \begin{pmatrix} \mathbf{x}\_t \\ \mathbf{y}\_t \end{pmatrix} \tag{22}$$

**Fig. 6** Field normalization using coordinate transformations. **a** The original quadratic function, *<sup>f</sup>* (*x*, *<sup>y</sup>*) <sup>=</sup> *<sup>x</sup>*<sup>2</sup> <sup>+</sup> *x y* <sup>+</sup> *<sup>y</sup>*<sup>2</sup> <sup>+</sup> <sup>0</sup>.4*<sup>x</sup>* <sup>+</sup> <sup>0</sup>.5*<sup>y</sup>* <sup>+</sup> <sup>0</sup>.07. **<sup>b</sup>** The function after applying translation such that the minimum is at the origin, *<sup>f</sup>* (*xt*, *yt*) <sup>=</sup> *<sup>x</sup>*<sup>2</sup> *<sup>t</sup>* <sup>+</sup> *xt yt* <sup>+</sup> *<sup>y</sup>*<sup>2</sup> *<sup>t</sup>* . **c** After applying rotation such that elliptical iso-contours are axes aligned, *<sup>f</sup>* (*xr*, *yr*) <sup>=</sup> <sup>0</sup>.5*x*<sup>2</sup> *<sup>r</sup>* <sup>+</sup> <sup>1</sup>.5*y*<sup>2</sup> *<sup>r</sup>* . **d** After scaling such that elliptical iso-contours are transformed into circular iso-contours, *<sup>f</sup>* (*xs*, *ys*) <sup>=</sup> *<sup>x</sup>*<sup>2</sup> *<sup>s</sup>* <sup>+</sup> *<sup>y</sup>*<sup>2</sup> *s*

Now, we apply a rotation *O* = (*EV*1|*EV*2) to align the elliptic iso-contours with the axis using the eigenvectors *EV*<sup>1</sup> and *EV*<sup>2</sup> of the matrix *M*

$$
\begin{pmatrix} \mathbf{x}\_r \\ \mathbf{y}\_r \end{pmatrix} = O \cdot \begin{pmatrix} \mathbf{x}\_t \\ \mathbf{y}\_t \end{pmatrix}, \tag{23}
$$

which results in the diagonal representation of the anisotropy

$$\nu(\mathbf{x}\_r, \mathbf{y}\_r) = \lambda\_1 \mathbf{x}\_r^2 + \lambda\_2 \mathbf{y}\_r^2 \tag{24}$$

where λ<sup>1</sup> and λ<sup>2</sup> are the eigenvalues of *M*.

Lastly, we apply non-uniform scaling to obtain circular iso-contours:

$$\nu(\mathbf{x}\_s, \mathbf{y}\_s) = \mathbf{x}\_s^2 + \mathbf{y}\_s^2 \quad \text{where} \quad \mathbf{x}\_s = \sqrt{\lambda\_1}\mathbf{x}\_r, \quad \mathbf{y}\_s = \sqrt{\lambda\_2}\mathbf{y}\_r. \tag{25}$$

The area distortion because of this scaling is given by the factor <sup>√</sup>λ1λ2.

#### **6 Subdivision in Monotonous Sub-triangles**

We consider a triangle with vertices *p*<sup>1</sup> = (*x*1, *y*1), *p*<sup>2</sup> = (*x*2, *y*2) and *p*<sup>3</sup> = (*x*3, *y*3) and the tensors at these vertices are *T*1, *T*<sup>2</sup> and *T*3. The tensors are linearly interpolated within the triangle and the anisotropy is a quadratic function. For the extraction of iso-contours, the computation of the contour tree as well as for the histogram we require piece-wise monotonous behavior inside the triangles, which is given in this chapter. Similar, subdivisions have also been proposed by Dillard et al. [6] and by Nucha et al. [15]. There are five different cases depending on the location of the global minima and the local minima at the edges. Especially the cases when the minimum *pc* lies within the triangle or outside the triangle need a different treatment for the computation of the anisotropy histogram.

**Fig. 7** All the possible cases and the corresponding sub-divisions into monotonous triangles. **a**, **b** The case when the minimum is inside the triangle.We can subdivide the triangle into six monotonous triangles. **c**, **d** The case when the minimum is outside the triangle, but the function restricted to the triangle edges has a minimum on all the edges. Four monotonous triangles can be generated. **e**, **f** The case when two of the edges have minima, three monotonous triangles are generated. **g**, **h** The case when only one triangle edge has a minimum, two monotonous triangles are created. **i**, **j** The case when none of the edges has a minimum, no triangle subdivision is required. **k**, **l** Lastly, the degenerate case when the function does not have a point minimum but has a degenerate minimum along a line. In this case, the iso-contours are a pair of parallel lines

	- 1. No triangle edge has an edge minimum. See Fig. 7i.
	- 2. One triangle edge has an edge minimum. See Fig. 7g.
	- 3. Two triangle edges have edge minima. See Fig. 7e.
	- 4. All three triangle edges have edge minima. See Fig. 7c.

In all these cases, the triangle can be sub-divided into an appropriate number of monotonous triangles as described in Lines 14–30 of Algorithm 1. Although these sub-divided triangles are monotonous and look similar to Case A, an important difference is that in general none of the vertices lies at the origin.

#### **7 Computation of the Histogram for** *ν*

In this section, we derive the exact continuous histogram of the anisotropy for linearly interpolated tensor fields. We use the term *histogram* here not strictly as it has been originally introduced but in the sense of the distribution of function values. It can also be considered as the *weighted contour spectrum* using the terminology of Bajaj et al. [2] as introduced by Scheidegger et al. [16]. In contrast to previous methods, we approach the problem by first computing the cumulative histogram *C H* and then derive the histogram from it by computing its derivative. This makes the exact computation of the histogram much more feasible.

$$CH(\nu\_0) = Area(\nu \le \nu\_0) = \sum\_{\triangle\_i} Area\_i(\nu \le \nu\_0)$$

where *Areai*(ν ≤ ν0) is the area of the sublevel set in triangle *<sup>i</sup>* . Specifically, we consider here the anisotropy with its quadratic behavior, however the method also directly applies for linear fields.

In our derivation of *Areai*(ν ≤ ν0) we consider the following setting. Given is a triangle *ABC* with vertices *A* = *p*<sup>1</sup> = (*x*1, *y*1), *B* = *p*<sup>2</sup> = (*x*2, *y*2) and *C* = *p*<sup>3</sup> = (*x*3, *y*3) and respective tensors *T*1, *T*<sup>2</sup> and *T*3, which are linearly interpolated within the triangle. Further we ordered the vertices such that the anisotropy values are ν<sup>1</sup> < ν<sup>2</sup> < ν3, using simulation of simplicity [7] to resolve the ties if required. To obtain the contribution of the triangle to the cumulative histogram at the value ν we compute the area of the sublevel set at ν within the triangle. In the following, we assume that anisotropy is monotonous within the triangles resulting from the subdivision introduced in Sect. 6. We further assume that the transformations as described in Sect. 5.1 have been applied. This means that the anisotropy has the form as given in Eq. (25) with a global minimum *pc* = (0, 0) at the origin and the level sets that are circles. The area within the transformed triangle is distorted by a constant factor, which can be appropriately multiplied to get the exact areas for the original triangle. We consider two cases depending on whether the vertex *A* of the triangle vertex lies at the origin or not.

#### *Case 1: The global minimum pc lies at one triangle vertex*

Let's assume that the global minimum *pc* lies in vertex *A* = *p*<sup>1</sup> of the triangle *ABC*. Then the shape of the sublevel set for ν can have two different types depending on whether ν is smaller or larger than ν<sup>2</sup> = *r* <sup>2</sup> *AB*. If ν ≤ ν<sup>2</sup> the sublevel set is a sector of a circle with radius *<sup>r</sup>* <sup>=</sup> <sup>√</sup><sup>ν</sup> and opening angle <sup>θ</sup>*A*, as shown in Fig. 8. The area of the sublevel set for ν ∈ [ν1, ν2] is then given by

$$Area(\nu) = \frac{\theta\_A \nu}{2}.\tag{26}$$

where θ*<sup>A</sup>* is angle between the edges *AB* and *AC*. The rate of change of the area is given by

$$\frac{\partial Area(\nu)}{\partial \nu} = \frac{\theta\_A}{2},\tag{27}$$

which is a constant not depending on the specific value of ν.

If the isovalue ν is greater than ν<sup>2</sup> and less than ν3, the iso-contour is composed of a circular sector and an additional more complex shape *DBFE* as illustrated in Fig. 8. This region is enclosed by two circular segments *DB* and *E F* and two line segments *DE* and *B F* and can be computed as

$$\begin{split} Area(BDEF) &= Area(BGFF) + Area(FHE) - Area(BGD) \\ &= \left(\frac{(r\_{AB}\sin\theta\_A + r\sin\theta)(r\cos\theta - r\_{AB}\cos\theta\_A)}{2}\right) \\ &+ \left(\frac{\theta r^2}{2} - \frac{r^2\sin\theta\cos\theta}{2}\right) - \left(\frac{\theta\_A r\_{AB}^2}{2} - \frac{r\_{AB}^2\sin\theta\_A\cos\theta\_A}{2}\right) \\ &= \frac{\theta r^2 - \theta\_A r\_{AB}^2}{2} + r r\_{AB} \left(\frac{\sin\theta\_A\cos\theta - \cos\theta\_A\sin\theta}{2}\right). \end{split}$$

**Fig. 8** A triangle with monotonous function behaviour where the global minimum *pc* lies on one vertex *A* = *p*1. Due to the normalization of the triangle, *pc* is at the origin and the iso-contours are circular. If ν ≤ ν<sup>2</sup> the sublevel set is a subset of the light blue area. If ν>ν<sup>2</sup> the sublevel area is composed of the complete light blue region and the area is highlighted in darker blue

With *r* <sup>2</sup> = ν and *r* <sup>2</sup> *AB* = ν<sup>2</sup> this results in

$$Area(BDEF) = \frac{\theta \nu - \theta\_A \nu\_2}{2} + \sqrt{\nu \nu\_2} \frac{\sin(\theta\_A - \theta)}{2}. \tag{28}$$

In the expression above, the given value of ν determines the values of θ.

#### *Case 2: The global minimum pc does not lie at any triangle vertex*

These triangles are the more generic case, we still assume that the anisotropy is monotonous inside the triangle and the vertices such that the anisotropy values are ν<sup>1</sup> < ν<sup>2</sup> < ν3. The triangles look similar to Case 1, but none of the vertices lie at the origin, compare Fig. 9. To compute the area of a sublevel set *ABFE* within these triangles, we considering the sublevel sets in two triangles where we can use the algorithm of Case 1. In Fig. 9 they are the triangles *DBC* and *DHC*. For the final area, we have in addition to consider the area of triangle *AHB* which has to be added or subtracted depending on the exact position of *A*.

$$Area(ABFE) = Area(DBFG) - Area(DHEG) \pm Area(\triangle AHB) \qquad (29)$$

**Fig. 9** A triangle *ABC* with monotonous function behaviour where the global minimum *pc* does not lie on any vertex

#### *7.1 Implementation*

We compute the cumulative histogram of anisotropy by computing the sublevel set area. The detailed algorithm is provided in Algorithm 1. For a given tensor mesh, we pre-process all the triangles to determine the coefficients of the quadratic function for anisotropy within the triangle, compare Sect. 5. Each triangle is appropriately subdivided into monotonous triangles, compare Sect. 6. Then for each anisotropy value in the histogram bin, we compute the sublevel set area by adding up the sublevel set area in all monotonous triangles. We also handle triangles with degenerate minimum appropriately.

**Parallelization**. The algorithm described in Algorithm 1 has lots of options for parallelization. The loops at Lines 4 and 38 are embarrassingly parallel because each triangle in the mesh can be pre-processed in parallel. Similarly, each bin in the histogram can be computed in parallel. Further, with atomic add operations, the computation of the sublevel set area for anisotropy value ν corresponding to a particular histogram bin can be parallelized over the set of monotonous triangles. These are the loops at Lines 41 and 44 in Algorithm 1.

**Numerical issues**. Since there are a few floating-point operations and checks involved in the Algorithm 1, floating-point errors can be introduced which may propagate to the output resulting in a wrong histogram. We handle these errors by introducing reasonable checks, for example making sure that the cumulative histogram thus obtained is always a monotonically increasing function and sums up to the total area of the domain for the largest value of anisotropy. However, a deeper study of the errors and more robust computation is left for future work. We are confident that using a multi-precision library for computation will remove the floating-point errors.

**Run times**. We implemented the Algorithm 1 as a prototype in Java 1.8. This proof of concept implementation does not include extensive code and memory optimizations. The experiments reported in following Sect. 8 were performed on a workstation with 12 core Intel Xeon processor and 32 GB of RAM.

The running time depends on the size of the input mesh and the number of bins in the output histogram. For all the experiments we chose 500 as the number of bins in the histogram. The synthetic data used in Sect. 8.1 is a small data set with the mesh containing only 32 triangles. It took about 0.02 s to compute the continuous histogram for the meshes in this ensemble. The histogram computation for the two point load data reported in Sect. 8.2 took 0.2 s. This triangle mesh contained 1682 triangles. The run time improved to 0.1 s when using multi-threading. Lastly, for the DTI data set containing 722 triangles (Sect. 8.3), it took 0.09 and 0.17 s to compute the histogram for brain and noisy regions respectively. The times improved by a factor of 2 for both cases when using multi-threading.


#### **8 Results**

We apply the proposed approach of computing anisotropy histograms to three different case studies. We show results for synthetic, simulation and experimental data. For all the case studies we compute the continuous histograms for three different interpolations of the anisotropy.


The resulting anisotropy fields are directly shown using a color map where dark blue refers to an anisotropy value of zero and red assigned to the maximum value. We also show a set of iso-contours in these images as white lines. In addition, we computed the join tree for all cases, which are always the same for interpolation [b] and [c]. Finally, we computed the continuous histogram where the results for one data set are plotted in one image, interpolation [a] displayed as a red curve, interpolation [b] as a green curve and interpolation [c] as a black curve.

#### *8.1 Synthetic Data*

The first example is a synthetic data set where tensors with user specified tensor components are placed at grid locations of a 5 × 5 grid. This grid is triangulated to provide a mesh with 32 triangles. This simple example is well suited to demonstrate the differences between the histogram of the consistent anisotropy field [b], and the methods utilizing a linear approximation of the anisotropy on the original mesh [a], and on the subdivided mesh with monotonous triangles [b], as shown in Fig. 10. It can immediately be seen that for interpolation [a] the topology of the iso-contours is not in accordance with the tensor field. While the iso-contours based on the tensor interpolation differ between interpolation, they have the same topology [b] and [c]. This is also reflected in the corresponding join trees, which are the same for both fields, for more details see Sect. 8.

In the next step, we randomly perturb the directions of the tensors at the vertices without changing their eigenvalues to generate an ensemble of four tensor fields. Note that since we do not change the eigenvalues, the anisotropy at the vertices of the mesh remains unchanged after perturbation. Hence, for interpolation [a] all the fields, the iso-contours, the join trees, and their histograms will be the same as evident from Fig. 11a. However, if we use quadratic function resulting form the tensor interpolation

**Fig. 10** Comparison of the behavior of the anisotropy field in a synthetic dataset. The anisotropy is linearly interpolated within the original mesh [**a**], and the subdivided monotonous triangles [**b**]. The anisotropy assuming linear interpolation of tensor components [**c**]. The upper row shows the resulting field as a color-map superimposed with contour lines. The ellipses at the vertices represent the tensors defining the field. The second row shows the respective join trees for the three anisotropy fields. The red squares correspond to the zeros inside of the original triangles. All zero-leaves in the tree represent degenerate points of the tensor field topology

[c] of anisotropy, we clearly observe the differences within the ensemble members as shown in Fig. 11c. Similarly, the histograms for the ensemble members are different as shown by black curves in the plots in Fig. 11d. While for interpolation [b], the subdivision into monotonous triangles helps in identifying the differences, Fig. 11b, the histograms are still not accurate and have a bias toward larger anisotropy values, Fig. 11d. The respective contour trees are shown in Fig. 12. The contour tree for the interpolation [a] is the same for all 4 fields and already given in Fig. 11a, so we only show the trees for the sub-division in monotonous triangles. The degenerate points of the tensor field where the anisotropy is zero appear also as minima in the join tree and are highlighted in red. The join trees provide an overview of the possible cancellations of degenerate points and thus their stability [18]. Comparing the trees it can be seen that the four data sets vary significantly for their topological structure and stability of its degenerate points. The join tree for the linear anisotropy, Fig. 11a, has no zero and is thus not consistent with the direction fields given by the tensors.

**Fig. 11** Random directions tensor dataset with constant eigenvalues. **a** The original mesh with anisotropy computed at the mesh vertices and linearly interpolated within the triangles. **b** The subdivided mesh with monotonous triangles. Anisotropy is linearly interpolated within the monotonous triangles. **c** The anisotropy resulting from linear interpolation of tensor components. **d** summarizes all histograms with red curves for interpolation [a], green curves for interpolation [b] and black curves for interpolation [c]

#### *8.2 Simulation Data*

The difference between the three different interpolations for the anisotropy field gave significantly different results for the synthetic data. In the next step, we want to investigate the impact of these differences on real-world data. At first, we consider a simulation data set from mechanical engineering. It is a well-known data set, often

**Fig. 12** The join trees for the four ensemble members shown in Fig. 11. The trees summarize the hierarchical structure of the minima in the anisotropy field and the degenerate points in the tensor field topology. Degenerate points inside the triangles are marked as red nodes in the tree

**Fig. 13** Two-point load data: simulation of the stresses inside a solid block upon applying two forces. **a** shows a schematic of the set-up of the simulation, **b** shows the eigenvector directions in one slice, **c** is a close up with color showing the signed anisotropy using bi-linear interpolation on the original mesh

referred to as two-point load, that represents a stress field in a solid block resulting from the application of two external forces, Fig. 13. The data is given on a cubic mesh. The anisotropy measure used so far corresponds to the squared von Mises stress, which plays an important role in failure analysis of mechanical components. The direction field in one slice is shown in Fig. 13b. In our analysis, we consider a section of this slice, which is shown in Fig. 13c displaying one eigenvector field. Color represents the anisotropy using a bi-linear interpolation in the original mesh. In our analysis, we use a triangulated version of the data.

Figure 14 shows the anisotropy fields using the three different interpolations in comparison. In Fig. 14a one can observe the asymmetry introduced by the triangulation but is similar to Fig. 13c. The anisotropy fields in Fig. 14b, c are both based on the monotonous subdivision and are very similar to each other but differ strongly from Fig. 14a. The asymmetry due to the triangulation is largely reduced. The minima inside the original triangles in the field capture the locations of the degenerate points of the direction field. The corresponding histogram can be seen in Fig. 14d, e. As expected they differ strongly for very small values but are rather similar for large values of the anisotropy.

**Fig. 14** Two point load simulation data with focus region around one of the loading points. **a** The original mesh with anisotropy computed at the mesh vertices and linearly interpolated within the triangles. **b** The subdivided mesh with monotonous triangles. Anisotropy is linearly interpolated within the monotonous triangles. **c** The anisotropy resulting from linear interpolation of tensor components. The isotropic points in the tensor field are highlighted by small red spheres. **d** the three cumulative histograms and (**e**) the histograms. The red curves correspond to (**a**), the green curves to (**b**) and the black curve to (**c**)

#### *8.3 Measurement Data*

As the last example, we examine two sections of a slice of 3D Diffusion Tensor Imaging data. Specifically, we compute and compare the anisotropy histograms of noisy regions outside the brain and a region inside the brain. The 2D slice from the data along with selected regions is shown in Fig. 15.

Figure 16 shows the histograms computed for the noisy region. With the interpolation approach [c] for computing histograms, we can observe a high contribution of anisotropy values near zero, which hints at the existence of a lot of degenerate points in the noisy region. This is not captured by the histogram computed with interpolation approach [a]. It should be noted that interpolation [b] (green plot) although yields better results than interpolation [a] (red plot), it is still quite different than the accurate histogram based on component-wise tensor interpolation (black plot). We

**Fig. 15** Two segments selected from Diffusion Tensor Imaging data. The 20 × 20 2D grid shown in red outline is chosen from noisy region of the data while the selection shown in black outline is chosen from the region occupied by the brain. We plot the histograms for these selections in Figs. 16 and 17 respectively

did the same experiment for a region that is within the brain and hence contains less noise. The results are shown in Fig. 17. Here we observe, that histograms are very similar whether we use the accurate quadratic function for anisotropy of linearly interpolate it within the triangles.

#### **9 Conclusions**

In this paper, we explore the behavior of the anisotropy, as an example for a nonlinear derived tensor invariant, when applying a linear component-wise interpolation of the tensor field. We demonstrate that a linear interpolation of the invariant itself, the interpolation approach [a], leads to a topology of iso-contours that in many cases is not consistent with any tensor interpolation and leads to a bias in the histogram. With this analysis, we want to emphasize the importance of being consistent with the chosen interpolation in all analysis steps. For our analysis, we have chosen a linear interpolation of tensor components, which is the most commonly used method in simulations and provides a valid continuous field. We do not want to make a statement about the quality or suitability of different interpolation methods in specific applications, however we want to point out that the independent interpolation of the direction field and the anisotropy violates in many cases the preservation of topological invariants and does not result in a valid tensor field.

**Fig. 16** Anisotropy fields in a small noisy subset of brain dataset. **a** Linear interpolation in the original mesh. **b** Linear interpolation within the subdivided mesh with monotonous triangles. **c** The anisotropy resulting from linear interpolation of tensor components. **d** The comparison of cumulative histograms under these settings. **e** The comparison of the corresponding histograms with red (**a**), green (**b**), and black (**c**)

More specifically we have presented a derivation for the computation of isocontours and histograms in this setting. Component-wise linear interpolation of tensor components results in a quadratic function for anisotropy. The method is based on a subdivision of the mesh into triangles with monotonous behavior. This subdivision with a linear interpolation of the anisotropy, interpolation approach [b], already results in iso-contours that are topologically equivalent to the iso-contours based on the derived anisotropy field. However, the histograms are not accurate and show a bias towards larger anisotropy values. This is especially prominent in regions with many degenerate points. In areas of high anisotropy, interpolation [b] provides a good approximation. The method described in this chapter, the interpolation approach [c], can be used to compute an accurate continuous histogram for anisotropy using component-wise tensor interpolation.

**Fig. 17** A small subset of DTI brain dataset is selected within the brain region. **a** Linear interpolation in the original mesh, **b** Linear interpolation in monotonous triangles. **c** The anisotropy resulting from linear interpolation of tensor components. **d** and **e** display the respective cumulative histograms and histograms as (**a**) red, (**b**) green, (**c**) black plots

All derivations in this chapter are given for the anisotropy defined as the difference between the major and minor eigenvalue, for 2D tensor fields. Although not trivial, the extension of this work to 3D tensor fields is feasible. An extension to the determinant, which is also quadratic but not elliptic, would be possible in a similar way. A general extension to other non-linear tensor invariants, however, might not be possible in a closed form and will require a good approximation schema. Therefore, we plan to explore methods for efficient approximations to the distributions with clear error bounds. Also, computing continuous scatterplots to visualize the space of multiple invariants at once is an interesting topic and will be subject of future work.

**Acknowledgments** We thank Jochen Jankowai for creating Fig. 13 included in this chapter. This work was supported through the SeRC (Swedish e-Science Research Center) and the ELLIIT environment for strategic research in Sweden. We further want to thank the participants of the Seminar 18442 on Visualization and Processing of Anisotropy in Imaging, Geometry, and Astronomy, at Schloss Dagstuhl Leibniz-Zentrum for computer sciences, for fruitful discussions.

#### **Appendix A**

#### *A. Detailed Analysis of Anisotropy and Its Iso-Contours*

#### **Proof of** *H* **≥ 0**

Substituting the Eqs. (14)–(19) in Eq. (6), we can analyse the determinant of Hessian for anisotropy ν:

$$\begin{split} H &= 4AC - B^2 \\ &= 4\left( (e\_x - g\_x)^2 + 4f\_x^2 \right) \left( (e\_y - g\_y)^2 + 4f\_y^2 \right) - 4\left( (e\_x - g\_x)(e\_y - g\_y) + 4f\_x f\_y \right)^2 \\ &= 4\left( (e\_x - g\_x)^2 (e\_y - g\_y)^2 + 4f\_x^2 (e\_y - g\_y)^2 + 4f\_y^2 (e\_x - g\_x)^2 + 16f\_x^2 f\_y^2 \right) \\ &\quad - 4\left( (e\_x - g\_x)^2 (e\_y - g\_y)^2 + 8f\_x f\_y (e\_x - g\_x)(e\_y - g\_y) + 16f\_x^2 f\_y^2 \right) \\ &= 16\left( f\_x^2 (e\_y - g\_y)^2 + f\_y^2 (e\_x - g\_x)^2 - 2f\_x f\_y (e\_x - g\_x)(e\_y - g\_y) \right) \\ &H = 16\left( f\_x (e\_y - g\_y) - f\_y (e\_x - g\_x) \right)^2 \ge 0 \end{split} \tag{30}$$

Since *H* ≥ 0, we have shown that the iso-contours of νare never hyperbolic.

#### **Proof of** *I* **= 0 when** *H* **= 0**

Let us analyze the case when *H* = 0 to complete the analysis of behaviour of anisotropy in all cases.

$$H = 16\left(f\_x(e\_\mathbf{y} - \mathbf{g}\_\mathbf{y}) - f\_\mathbf{y}(e\_x - \mathbf{g}\_x)\right)^2 = 0$$
 
$$\text{or,} \quad f\_x(e\_\mathbf{y} - \mathbf{g}\_\mathbf{y}) - f\_\mathbf{y}(e\_x - \mathbf{g}\_x) = 0$$
 
$$\text{or,} \quad \frac{e\_\mathbf{y} - \mathbf{g}\_\mathbf{y}}{e\_x - \mathbf{g}\_x} = \frac{f\_\mathbf{y}}{f\_x}$$
 
$$\text{or,} \quad e\_\mathbf{y} - \mathbf{g}\_\mathbf{y} = \frac{f\_\mathbf{y}}{f\_x}(e\_x - \mathbf{g}\_x) \tag{31}$$

Substituting (31) in (16) and using (14):

$$\begin{aligned} \mathbf{C} &= \left( \left( e\_{\mathbf{y}} - \mathbf{g}\_{\mathbf{y}} \right)^{2} + 4f\_{\mathbf{y}}^{2} \right) \\ &= \frac{f\_{\mathbf{y}}^{2} (e\_{\mathbf{x}} - \mathbf{g}\_{\mathbf{x}})^{2}}{f\_{\mathbf{x}}^{2}} + 4f\_{\mathbf{y}}^{2} \\ &= \frac{f\_{\mathbf{y}}^{2}}{f\_{\mathbf{x}}^{2}} \left( (e\_{\mathbf{x}} - \mathbf{g}\_{\mathbf{x}})^{2} + 4f\_{\mathbf{x}}^{2} \right) \\ \text{or,} \quad \mathbf{C} &= \frac{f\_{\mathbf{y}}^{2}}{f\_{\mathbf{x}}^{2}} A \end{aligned} \tag{32}$$

Substituting (31) in (18) and using (17):

$$\begin{aligned} E &= 2\left( (e\_\mathbf{y} - g\_\mathbf{y})(e\_c - g\_c) + 4f\_\mathbf{y}f\_c \right) \\ &= 2\left( \frac{f\_\mathbf{y}(e\_x - g\_x)(e\_c - g\_c)}{f\_x} + 4f\_\mathbf{y}f\_c \right) \\ &= \frac{f\_\mathbf{y}}{f\_x} \cdot 2\left( (e\_x - g\_x)(e\_c - g\_c) + 4f\_xf\_c \right) \\ \text{or,} \quad E &= \frac{f\_\mathbf{y}}{f\_x}D \end{aligned} \tag{33}$$

From Eqs. (32) and (33):

$$\frac{C}{A} = \frac{E^2}{D^2} = \frac{f\_\text{y}^2}{f\_\text{x}^2} \tag{34}$$

$$or, \quad AE^2 = CD^2\tag{35}$$

Substituting (31) in (15) and using (16):

$$\begin{aligned} B &= 2\left( (e\_x - g\_x)(e\_y - g\_y) + 4f\_x f\_y \right) \\ &= 2\left( \frac{f\_x(e\_y - g\_y)^2}{f\_y} + 4f\_x f\_y \right) \\ &= 2\frac{f\_x}{f\_y}\left( (e\_y - g\_y)^2 - 4f\_y^2 \right) \\ \text{or,} \quad B &= 2\frac{f\_x}{f\_y}C \end{aligned} \tag{36}$$

Multiplying Eqs. (33) and (36), we obtain:

$$BE = \mathcal{Z}CD\tag{37}$$

Let us evaluate the Eq. 7 now:

$$\begin{aligned} I &= BDE - AE^2 - CD^2 \\ I &= BDE - 2CD^2 \\ I &= D(BE - 2CD) \\ I &= D(0) = 0 \end{aligned} \qquad \text{using} \qquad (35)$$

$$\begin{aligned} I &= D(0) = 0 \end{aligned} \qquad \text{using} \qquad (37) \tag{38}$$

From Eq. (38), we conclude that when *H* is 0, *I* is also 0. This means that the iso-contours of ν are never parabolic.

To conclude, based on Eqs. (30) and (38), we deduce that the contours of anisotropy ν and hence α are either ellipses or a set of parallel lines.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Image Processing and Visualization**

## **Tensor Approximation for Multidimensional and Multivariate Data**

**Renato Pajarola, Susanne K. Suter, Rafael Ballester-Ripoll, and Haiyan Yang**

**Abstract** Tensor decomposition methods and multilinear algebra are powerful tools to cope with challenges around multidimensional and multivariate data in computer graphics, image processing and data visualization, in particular with respect to compact representation and processing of increasingly large-scale data sets. Initially proposed as an extension of the concept of matrix rank for 3 and more dimensions, tensor decomposition methods have found applications in a remarkably wide range of disciplines. We briefly review the main concepts of tensor decompositions and their application to multidimensional visual data. Furthermore, we will include a first outlook on porting these techniques to multivariate data such as vector and tensor fields.

#### **1 Introduction**

Data approximation is widely used in the fields of computer graphics and scientific visualizations. One way to achieve it is to decompose the data into a more compact and compressed representation. The general idea is to express a dataset by a set of bases, which are used to reconstruct the dataset to its approximation when needed (see Fig. 1). More specifically, such a representation usually consists of the actual bases and the corresponding coefficients describing the relationship between the original data and the actual bases. Typically, such compact representations consist of less data than the original dataset, capture the most significant features, and, moreover, describe the data in a format that is convenient for adaptive data loading and access.

R. Ballester-Ripoll IE University, Madrid, Spain e-mail: rafael.ballester@ie.edu

R. Pajarola (B) · H. Yang University of Zürich, Zürich, Switzerland e-mail: pajarola@ifi.uzh.ch

S. K. Suter Zurich University of Applied Sciences, Zürich, Switzerland e-mail: sues@zhaw.ch

**Fig. 1** Compact data representation for a 3rd-order tensor A (a volume) by bases and coefficients that can be used to reconstruct the data to its approximation <sup>A</sup>-

Bases for compact data representation can be classified into two types: *pre-defined* and *learned* bases. Pre-defined bases comprise a given function or filter, which is applied to the dataset without any a-priori knowledge of the correlations in the dataset. In contrast, learned bases are generated from the dataset itself. Established examples of the former are the Fourier transform (FT) and the wavelet transform (WT). Wellknown examples of the latter are the principal component analysis (PCA), the singular value decomposition (SVD) and vector quantization. Using predefined bases is often computationally cheaper, but learned bases potentially remove more redundancy from a dataset.

Generally, PCA-like methods are able to extract the main data direction of the dataset and represent the data in another coordinate system such that it makes it easier for the user to find the major contributions within the dataset. To exploit this, PCAs higher-order extension—*tensor approximation* (TA)—can be used for multidimensional datasets. The first occurrence of TA was in [21]. The idea of multiway analysis, however, is generally attributed to Catell in 1944 [11]. It took a few decades until tensor approximations received more widespread attention, namely by several authors in the field of psychometrics [10, 19, 52].

#### *1.1 Higher-Order Data Decompositions*

The matrix SVD works on 2D data arrays and exploits the fact that the dataset can be represented with a few highly significant coefficients and corresponding reconstruction vectors based on the matrix rank reduction concept. The SVD, being a multilinear algebra tool, computes (a) a rank-*R* decomposition and (b) orthonormal row and column vector matrices. Unlike for 2D, the extension to higher-orders is not unique and these two main properties are captured by two different models that are both called tensor approximations : the Tucker model [14, 15, 25, 49, 52] preserves the orthonormal factor matrices while the CP model (from CANDECOMP [10] and PARAFAC [19]) preserves the rank-*R* decomposition.

Generally speaking, a tensor is a term for a higher-order generalization of a vector or a multidimensional array. In TA approaches, a multidimensional input dataset in array form, i.e., a tensor, is factorized into a sum of rank-one tensors or into a product of a core tensor (coefficients that describe the relationship to input data) and matrices (bases), i.e., one for each dimension. This factorization process is generally known as*tensor decomposition*, while the reverse process of the decomposition is the *tensor reconstruction*.

Tensor decompositions have been widely studied in other fields and were reviewed [13, 25, 34] and summarized [27, 44]. Since TA was emerging from various disciplines, it was developed under multiple names. The CP model was independently developed under the terms CANDECOMP and PARAFAC, therefore, it is sometimes referenced with a single name. The Tucker model takes its name from Tucker, who initially worked on the *three-mode factor analysis* (3MFA), which is sometimes referred to as the Tucker3 model, also called the *three mode PCA* (3MPCA) [27, 28, 49]. Similarly the model was referenced to as *n-mode PCA* [23] since it is equivalent to applying PCA *n* times, each time along a different mode of the tensor. In [14] all these previous works were captured and written down as generalization of the SVD as *multilinear singular SVD*, which is usually termed as higher-order SVD or HOSVD. Furthermore, it was also called *n-mode SVD* in [54, 55].

Given an input tensor, different decompositions capture different types of structures and result in varying numbers of coefficients. For a given accuracy, the number of CP ranks required to decompose a tensor is usually much larger to that of Tucker ranks. On the other hand, CP's storage cost grows only linearly with respect to its ranks, whereas that relationship becomes exponential in the case of Tucker. In sum, there is no silver bullet: CP is more suitable than Tucker for certain types of data, and vice versa. As a rule of thumb:


The data sets addressed in this chapter fit in the first category, and so we restrict our experiments to the Tucker model. To further illustrate the usual advantage of Tucker over CP for spatial visual data, see Fig. 2, we compare the number of coefficients and root mean squared error (RMSE) obtained with CP vs. Tucker using different numbers of ranks for a 256 × 256 × 256 CT scan of a bonsai.

#### *1.2 TA Applications in Graphics and Visualization*

TA approaches have been applied to a wide range of application domains. Starting from psychometrics, in recent years, TA has been applied to visual data. A highly studied area is TA used for image ensembles [20, 35, 42, 43, 54, 58–60, 63] and/or TA used for pattern recognition, e.g., [17, 32, 40, 41, 43, 59]. In (real-time) rendering, tensor decompositions have been used as a method for global illumination models, e.g., for bidirectional reflectance distribution functions (BRDFs) [9, 45] or precomputed radiance transfer (PRT) [45, 50, 51]. Furthermore, TAs have successfully been used in graphics in the context of bidirectional texture functions (BTFs) [3,

**Fig. 2** RMSE achieved after CP vs. Tucker compression at different ranks *R*. For Tucker we take *<sup>R</sup>*<sup>1</sup> <sup>=</sup> *<sup>R</sup>*<sup>2</sup> <sup>=</sup> *<sup>R</sup>*<sup>3</sup> <sup>=</sup> *<sup>R</sup>*. The dataset used is a bonsai CT scan of size 256<sup>3</sup>

18, 38, 39, 51, 55, 61, 62], texture synthesis [62], time-varying visual data [61, 62], 3D face scanning [56] and compression in animation [29, 31, 33, 36, 37, 53, 57].

In scientific visualization, TA methods have first been introduced for interactive multiresolution and multiscale direct volume rendering [6–8, 46–48]. Additionally, their compact representation power has been exploited for 3D volume data compression [4, 6] with notable advantages over other approaches at extreme compression ratios [2]. In this work, we explore the multiscale feature expressiveness of TA methods for the first time on vector fields, i.e. multidimensional multivariate data . Hence we interpret the vector field as a 4D array, or as a 4th-order data tensor. However, the rank-reduction of TA is only applied to the three spatial dimension.

#### *1.3 Motivation and Contributions*

In the results section (Sect. 9) we demonstrate that the feature sensitive approximation power of TA methods carries over from scalar to vector fields. These first results are promising and encourage the extension of tensor compression techniques to multivariate data fields, e.g. for compact storage and quick visualization at variable feature scales of large vector data in the fields of computational fluid dynamics or biomedicine. Moreover, based on the compression-domain data filtering and processing capabilities demonstrated in [5], we expect that important vector-field operators such as divergence or vorticity as well as other feature extraction operations can be analyzed and performed directly in the compressed TA format.

#### **2 Singular Value Decomposition**

The SVD is a widely used matrix factorization method to solve linear least-square problems. The SVD can be applied to any square or rectangular matrix **<sup>A</sup>** <sup>∈</sup> <sup>R</sup>*<sup>M</sup>*×*<sup>N</sup>* . Hence, the decomposition is always possible. The aim of the SVD is to produce a diagonalization of the input matrix **A**. Since the input matrix **A** is not symmetric, two bases (matrices) are needed to diagonalize **A**. Therefore, the SVD produces a matrix factorization into two orthogonal bases **<sup>U</sup>** <sup>∈</sup> <sup>R</sup>*<sup>M</sup>*×*<sup>M</sup>* and **<sup>V</sup>** <sup>∈</sup> <sup>R</sup>*<sup>N</sup>*×*<sup>N</sup>* and a diagonal matrix - <sup>∈</sup> <sup>R</sup>*M*×*<sup>N</sup>* , as expressed in Eq. (1) (matrix form) or Eq. (2) (summation form).

$$\mathbf{A} = \begin{array}{c} \mathbf{U}\Sigma\mathbf{V}^{\mathrm{T}} \end{array} \tag{1}$$

$$\mathbf{A} = \begin{array}{c} \mathbf{U} \Sigma \mathbf{V}^{\mathrm{T}} \\\\ a\_{mn} = \sum\_{r=1}^{R} u\_{mr} \sigma\_r v\_{nr} \end{array} \tag{1}$$

The bases **U** and **V** contain orthogonal unit length vectors **u***<sup>j</sup>* and **v** *<sup>j</sup>* , respectively, and represent a *r*-dimensional column space (R*<sup>M</sup>* ) and a *r*-dimensional row space (R*<sup>N</sup>* ). Hence, the bases **U** and **V** are even orthonormal. The diagonal matrix contains the *singular values* σ*r*, where σ<sup>1</sup> ≥ σ<sup>2</sup> ≥···≥ σ*<sup>R</sup>* ≥ 0, where *R* = min(*M*, *N*). A *singular value* and a pair of *singular vectors* of a square or rectangular matrix **A** correspond to a non-negative scalar σ and two non-zero vectors **u***<sup>j</sup>* and **v** *<sup>j</sup>* , respectively. The vectors **u***<sup>j</sup>* are the *left singular vectors*, and the vectors **v** *<sup>j</sup>* are the *right singular vectors* (see Fig. 3). The number of non-zero singular values determines the rank *R* of the matrix **A**.

In applications truncated versions of the SVD are frequently desired. That is, only the first *K* singular values σ<sup>1</sup> ...σ*<sup>K</sup>* and the corresponding *K* singular vectors **u**<sup>1</sup> ... **u***<sup>K</sup>* and **v**<sup>1</sup> ... **v***<sup>K</sup>* are used for the reconstruction. This approach is referred to as low-rank approximation of a truncated SVD. Basically, each weighted outer vectorproduct term σ*<sup>j</sup>* · **u***<sup>j</sup>* ◦ **v** *<sup>j</sup>* corresponds to a rank-one component (see also Fig. 4), and the SVD of matrices or images consequently represents a 2D data array eventually as a sum of such rank-one components.

**Fig. 3** Visualization of the summed form of the SVD as shown in Eq. (2)—illustrating the singular values with the corresponding left and right singular vector pairs

**Fig. 4** Simple 2D functions can be represented as an outer product of two 1D functions, and more complex ones as a weighted sum of several such components

#### **3 Tensor Approximation Notation and Definitions**

The notation taken here is mostly taken from De Lathauwer et al. [13, 14], Smilde et al. [44], as well as Kolda and Bader [25], who follow the notation proposed by Kiers [24]. To illustrate higher-order extensions we mostly make examples of order three.

#### *3.1 General Notation*

A tensor is a multidimensional array (or an *N*-way data array): a 0th-order tensor (tensor0) is a scalar, a 1st-order tensor (tensor1) is a vector, a 2nd-order tensor is a matrix, and a 3rd-order (tensor3) is a volume, see Fig. 5. We consistently use the letter 'a' to represent the data, following the notation of, e.g., [14, 15, 51, 61, 62]. We use lower case letters for a scalar *a*, lower case boldface letters for a vector **a** in R*<sup>I</sup>*<sup>1</sup> , capital boldface letters for a matrix **A** in R*<sup>I</sup>*1×*I*<sup>2</sup> , and calligraphic letters for a 3rd-order tensor <sup>A</sup> in <sup>R</sup>*<sup>I</sup>*1×*I*2×*I*<sup>3</sup> .

**Fig. 5** A tensor is a multidimensional array: a 2nd-order tensor is a matrix **A** and a 3rd-order tensor is a volume A

**Fig. 6** Slices of a 3rd-order tensor A

The *order* of a tensor is the number of data directions, also referred as *ways* or *modes*. Along a mode *n*, the index *in* runs from 1 to *IN* . By using lower script indices for the modes, we can extend the index scheme to any order, i.e., *I*1, *I*2, *I*3, *I*4,... . The *i*th entry of a vector **a** is denoted by *ai* , an element (*i*1,*i*2) of a matrix **A** is denoted by *ai*1*i*<sup>2</sup> , and an element (*i*1,*i*2,*i*3) of a 3rd-order tensor A is denoted by *ai*1*i*2*i*<sup>3</sup> .

The general term *fibers*is used as a generalization for vectors taken along different modes in a tensor. A fiber is defined by fixing every index but one. A matrix column is thus a mode-1 fiber and a matrix row is a mode-2 fiber. 3rd-order tensors have column, row, and *tube* fibers, denoted by **a***i*<sup>1</sup> , **a***i*<sup>2</sup> , and **a***i*<sup>3</sup> , respectively. Sometimes, fibers are also called mode-*n* vectors.

*Slices* are two-dimensional sub-sections of a tensor (e.g., one fixed index in a 3rdorder tensor). For a 3rd-order tensor A, there are, for example, *frontal*, *horizontal*, and *lateral* slices, denoted by **A***i*<sup>3</sup> , **A***i*<sup>1</sup> , and **A***i*<sup>2</sup> , respectively as illustrated in Fig. 6.

For computations, a tensor is often reorganized into a matrix what we denote as *tensor unfolding* (sometimes called *matricization*). There are two main unfolding strategies, *backward cyclic unfolding* [14] and *forward cyclic unfolding* [24] as shown in Fig. 7. An unfolded tensor in matrix shape is denoted with a subscript in parentheses, e.g., **A**(*n*).

#### *3.2 Computing with Tensors*

While many more operations on tensors exist, here we only outline the most common products used within the scope of this work.


in Fig. 8: <sup>A</sup> <sup>=</sup> **<sup>b</sup>**(1) ◦ **<sup>b</sup>**(2) ◦ **<sup>b</sup>**(3) , where an element (*i*1,*i*2,*i*3) of A is given by *ai*1*i*2*i*<sup>3</sup> <sup>=</sup> *<sup>b</sup>*(1) *<sup>i</sup>*<sup>1</sup> *<sup>b</sup>*(2) *<sup>i</sup>*<sup>1</sup> *<sup>b</sup>*(3) *i*3 .

• The *inner product* of two same-sized tensors <sup>A</sup>, <sup>B</sup> <sup>∈</sup> <sup>R</sup>*I*1×*I*2×···×*IN* is the sum of the products of their entries, i.e., Eq. (3). ...

If two same-sized tensors  $\mathcal{R}, \mathcal{B} \in \mathbb{R}^{I\_1 \times I\_2 \times \dots \times I\_N}$  is the sum of entries, i.e., Eq. (3).

 $(\mathcal{R}, \mathcal{B}) = \sum\_{i\_1=1}^{I\_1} \sum\_{i\_2=1}^{I\_2} \dots \sum\_{i\_N=1}^{I\_N} a\_{i\_1 i\_2 \dots i\_N} b\_{i\_1 i\_2 \dots i\_N} \tag{3}$ 

• The *n-mode product* [14] multiplies a tensor by a matrix (or vector) in mode *n*. The *<sup>n</sup>*-mode product of a tensor <sup>B</sup> <sup>∈</sup> <sup>R</sup>*I*1×*I*2×···×*IN* with a matrix **<sup>C</sup>** <sup>∈</sup> <sup>R</sup>*Jn*×*In* is denoted by B ×*<sup>n</sup>* **C** and is of size *I*<sup>1</sup> ×···× *In*−<sup>1</sup> × *Jn* × *In*+<sup>1</sup> ×···× *IN* . That is, element-wise we have Eq. (4). (<sup>B</sup> <sup>×</sup>*<sup>n</sup>* **<sup>C</sup>**)*<sup>i</sup>*1...*in*−<sup>1</sup> *jn in*+1...*iN* <sup>=</sup>

$$(\mathcal{B}\times\_n\mathbf{C})\_{i\_1\ldots i\_{n-1}j\_n i\_{n+1}\ldots i\_N} = \sum\_{i\_n=1}^{I\_n} b\_{i\_1 i\_2 \ldots i\_N} \cdot c\_{j\_n i\_n} \tag{4}$$

Each mode-*n* fiber is multiplied by the matrix **C**. The idea can also be expressed in terms of unfolded tensors (reorganization of a tensor into a matrix as described in Sect. 3.1).

The *n*-mode product of a tensor with a matrix is related to a change of basis in the case when a tensor defines a multilinear operator [25]. The *n*-mode product is the generalized operand to compute *tensor times matrix* (TTM) multiplications, and can best be illustrated using unfolded tensors as in Fig. 9.

$$\mathcal{A} = \mathcal{B} \times\_n \mathbf{C} \Leftrightarrow \mathbf{A}\_{(n)} = \mathbf{C} \mathbf{B}\_{(n)} \tag{5}$$

• The *norm of a tensor* <sup>A</sup> <sup>∈</sup> <sup>R</sup>*I*1×*I*2×···×*IN* is defined analogously to the matrix Frobenius norm **A***<sup>F</sup>* and is the square root of the sum squares of all its elements, i.e., Eq. (6). 

$$\begin{aligned} \text{In the square root of the sum squares of all its elements, i.e.,}\\ \|\mathcal{R}\|\_F = \sqrt{\sum\_{i\_1=1}^{I\_1} \sum\_{i\_2=1}^{I\_2} \dots \sum\_{i\_N=1}^{I\_N} a\_{i\_1 i\_2 \dots i\_N}^2} \end{aligned} \tag{6}$$

#### *3.3 Rank of a Tensor*

In order to describe the definitions of the tensor rank, the definition for the matrix rank is recaptured. The *matrix rank* of a matrix **A** is defined over its column and row ranks, i.e., the column and row matrix rank of a matrix **A** is the maximal number of linearly independent columns and rows of **A** that can be chosen, respectively. For matrices, the column rank and the row rank are always equal and, a matrix rank is therefore simply denoted as rank(**A**). A *tensor rank* is defined similarly to the matrix rank, however, there are differences. In fact, the extension of the rank concept is not uniquely defined in higher orders and we review the definitions for the tensor ranks from [14] here.

• The *n-rank* of a tensor A, denoted by *Rn* = r*ankn*(A), is the dimension of the vector space spanned by mode-*n* vectors, where the mode-*n* vectors of A are the column vectors of the unfolding **A**(*n*), and r*ankn*(A) = r*ank*(**A**(*<sup>n</sup>*)). Unlike matrices, the different *n*-ranks of a tensor are not necessarily the same.


$$\mathcal{R} = \mathbf{b}^{(1)} \circ \mathbf{b}^{(2)} \circ \cdots \circ \mathbf{b}^{(N)} \tag{7}$$

• The *tensor rank R* = r*ank*(A) is the minimal number of rank-one tensors that yield A in a linear combination (see [12, 14, 25, 30]). Except for the special case of matrices, the tensor rank is not necessarily equal to any of its *n*-ranks, but it always holds that *Rn* ≤ *R*.

#### **4 Tensor Decompositions**

In tensor decompositions an input tensor <sup>A</sup> <sup>∈</sup> <sup>R</sup>*I*1×*I*2×···×*IN* is decomposed into a set of factor matrices **<sup>U</sup>**(*n*) <sup>∈</sup> <sup>R</sup>*In*×*Rn* and coefficients <sup>B</sup> <sup>∈</sup> <sup>R</sup>*R*1×*R*2×···×*RN* that describe the relationship/interactivity between <sup>A</sup> and the set of **<sup>U</sup>**(*n*) .

Historically, as seen earlier, tensor decompositions are a higher-order extension of the matrix SVD. The nice properties of the matrix SVD, i.e., rank-*R* decomposition and orthonormal row-space vectors and column-space vectors, do not extend uniquely to higher orders. The rank-*R* decomposition can be achieved with the socalled CP model, while the orthonormal row and column vectors are preserved in the so-called Tucker model. An extensive review of the two models and further hybrid models can be found in [25]. Here, we only outline the Tucker model that we apply in our experiments.

#### *4.1 Tucker Model*

The Tucker model is a widely used approach for tensor decompositions. As given in Eq. (8), any higher-order tensor is approximated by a product of a core tensor <sup>B</sup> <sup>∈</sup> <sup>R</sup>*<sup>R</sup>*1×*R*2×···×*RN* and its factor matrices **<sup>U</sup>**(**n**) <sup>∈</sup> <sup>R</sup>*In*×*Rn* , where the products <sup>×</sup>*<sup>n</sup>* denote the *n*-mode product as outlined in Sect. 3.2. This decomposition can then again be reconstructed to its approximation <sup>A</sup>-. The missing information of the input tensor <sup>A</sup> that cannot be captured by <sup>A</sup> is denoted with the error *e*. The Tucker decomposition is visualized for a 3rd-order tensor in Fig. 10. Equivalently, a Tucker decomposition can also be represented as a sum of rank-one tensors as in Eq. (9) and illustrated in Fig. 11.

$$\begin{aligned} \underbrace{\mathbf{u}\_{1}\mathbf{u}\_{\cdot 1}\mathbf{u}\_{\cdot 2}\mathbf{u}\_{\cdot 3}\mathbf{u}\_{\cdot 4}}\_{I\_{3}} &= \underbrace{\mathbf{a}\_{1}\mathbf{u}\_{\cdot 1}\mathbf{u}\_{\cdot 3}\mathbf{u}\_{\cdot 1}}\_{R\_{2}} \underbrace{h\_{1}\mathbf{u}^{(0)}}\_{R\_{3}} \mathbf{u}^{(0)}\_{I\_{2}} \underbrace{\mathbf{u}^{(2)}}\_{R\_{2}} \mathbf{u}^{(2)}\_{I\_{3}} \mathbf{u}^{(0)}\_{I\_{2}} + \mathbf{e}^{(0)}\_{I\_{1}}\mathbf{u}^{(0)}\_{I\_{1}} \mathbf{u}^{(0)}\_{I\_{2}} \end{aligned}$$

**Fig. 10** Tucker 3rd-order tensor: <sup>A</sup> <sup>=</sup> <sup>B</sup> <sup>×</sup><sup>1</sup> **<sup>U</sup>**(1) <sup>×</sup><sup>2</sup> **<sup>U</sup>**(2) <sup>×</sup><sup>3</sup> **<sup>U</sup>**(3) + e

*r*1=1 *R*2 *r*2=1 *R*3 *<sup>r</sup>*3=<sup>1</sup> *br*1*r*2*r*<sup>3</sup> · **<sup>u</sup>**(1) *<sup>r</sup>*<sup>1</sup> ◦ **<sup>u</sup>**(2) *<sup>r</sup>*<sup>2</sup> ◦ **<sup>u</sup>**(3) *<sup>r</sup>*<sup>3</sup> <sup>+</sup> *<sup>e</sup>*

$$\mathcal{A} = \mathcal{B} \times\_1 \mathbf{U}^{(1)} \times\_2 \mathbf{U}^{(2)} \times\_3 \cdots \times\_N \mathbf{U}^{(N)} + e \tag{8}$$

$$
\mathcal{R} = \mathcal{B} \times\_1 \mathbf{U}^{(1)} \times\_2 \mathbf{U}^{(2)} \times\_3 \cdots \times\_N \mathbf{U}^{(N)} + e \tag{8}
$$

$$
\mathcal{R} = \sum\_{r\_1=1}^{R\_1} \sum\_{r\_2=1}^{R\_2} \dots \sum\_{r\_N=1}^{R\_N} b\_{r\_1 r\_2 \dots r\_N} \cdot \mathbf{u}\_{r\_1}^{(1)} \diamond \mathbf{u}\_{r\_2}^{(2)} \diamond \cdots \diamond \mathbf{u}\_{r\_N}^{(N)} + e \tag{9}
$$

The column vectors **<sup>u</sup>**(*n*) *rn* of the factor matrices **<sup>U</sup>**(*n*) <sup>∈</sup> <sup>R</sup>*In*×*Rn* are usually orthonormal and can be thought of as principal components *Rn* in each mode *n* [25]. The core tensor <sup>B</sup> <sup>∈</sup> <sup>R</sup>*R*1×*R*2×···×*RN* represents a projection of the original data <sup>A</sup> <sup>∈</sup> <sup>R</sup>*I*1×*I*2×···×*IN* onto its factor matrices and is always of the same order as the input data. The core tensor is computed in general, as shown in Eq. (10), and for orthogonal factor matrices as in Eq. (11). The element-wise core tensor computation is denoted in Eq. (12). In other words, the core tensor coefficients *br*1*r*2...*rN* represent the relationship between the Tucker model and the original data.

$$\mathcal{B} = \mathcal{H} \times\_1 \mathbf{U}^{(1)^{(-1)}} \times\_2 \mathbf{U}^{(2)^{(-1)}} \times\_3 \dots \times\_N \mathbf{U}^{(N)^{(-1)}} \tag{10}$$

$$\mathcal{B} = \mathcal{R} \times\_1 \mathbf{U}^{(1)^\top} \times\_2 \mathbf{U}^{(2)^\top} \times\_3 \dots \times\_N \mathbf{U}^{(N)^\top} \tag{11}$$

$$
\mathcal{B} = \mathcal{R} \times\_1 \mathbf{U}^{(1)^\top} \times\_2 \mathbf{U}^{(2)^\top} \times\_3 \dots \times\_N \mathbf{U}^{(N)^\top} \tag{11}
$$

$$
\mathcal{B} = \sum\_{i\_1=1}^{I\_1} \sum\_{i\_2=1}^{I\_2} \dots \sum\_{i\_N=1}^{I\_N} a\_{i\_1 i\_2 \dots i\_N} \cdot \mathbf{u}\_{i\_1}^{(1)^\top} \circ \mathbf{u}\_{i\_2}^{(2)^\top} \circ \dots \circ \mathbf{u}\_{i\_N}^{(N)^\top} \tag{12}
$$

The Tucker decomposition is not unique, which means that we can modify the core tensor B without affecting the model fit as long as we apply the same changes to the factor matrices (so-called core tensor rotations), for more details see [25].

Often, we are interested in compact models, which enable a compression of the input data. For example, after computing a (full) Tucker decomposition the core tensor B has the same size as the original input A and all the factor matrices are square. However, we are more interested in reduced-size, approximative Tucker decompositions, where <sup>B</sup> is an element of <sup>R</sup>*<sup>R</sup>*1×*R*2×*R*<sup>3</sup> with *<sup>R</sup>*<sup>1</sup> <sup>&</sup>lt; *<sup>I</sup>*1, *<sup>R</sup>*<sup>2</sup> <sup>&</sup>lt; *<sup>I</sup>*<sup>2</sup> and *R*<sup>3</sup> < *I*3. Using so-called *rank-reduced tensor decompositions* or *truncated tensor decompositions* one can directly obtain more compact decompositions.

#### **5 Tensor Rank Reduction**

As seen in Sect. 3.3, the extension of the matrix rank concept to higher orders is not unique and we will mostly follow the rank-(*R*1, *R*2,..., *RN* ) tensor decomposition and reduced-rank approximation of the Tucker model here.

#### *5.1 Rank-R and Rank-(R***1***, R***2***,..., RN) Approximations*

A simple rank-one approximation is defined as <sup>A</sup>- = λ · **u**(1) ◦ **u**(2) ···◦ **u**(*N*) from the rank-one tensor (vector) product (◦) of its *<sup>N</sup>* basis vectors **<sup>u</sup>**(*n*) <sup>∈</sup> <sup>R</sup>*In* and a weight factor λ. Hence a tensor A could be approximated by a linear combination of many rank-one approximations as in Eq. (13). This approximation, also known as a CP model, is called a *rank-R approximation*. =

is as in Eq. (13). Thus approximation, also known as a CP approximation.

 $\widetilde{\mathcal{R}} = \sum\_{r=1}^{R} \lambda\_r \cdot \mathbf{u}\_r^{(1)} \diamond \mathbf{u}\_r^{(2)} \diamond \cdots \diamond \mathbf{u}\_r^{(N)} \tag{13}$ 

Alternatively, if we allow all weighted tensor (vector) products **u**(1) *<sup>i</sup>*<sup>1</sup> ◦ **<sup>u</sup>**(2) *<sup>i</sup>*<sup>2</sup> ◦···◦ **u**(*N*) *iN* of any arbitrary index combinations*i*1*i*<sup>2</sup> ... *iN* , we end up with the Tucker model of Sect. 4.1 and Eq. 12 where the weight factors for all index combinations form the core tensor B. Choosing *R*<sup>1</sup>,...,*<sup>N</sup>* < *I*<sup>1</sup>,...,*<sup>N</sup>* we end up with a *rank-*(*R*1, *R*2,..., *RN* ) *approximation* of A, which is given by a decomposition into a lower-rank tensor A- <sup>∈</sup> <sup>R</sup>*<sup>I</sup>*1×*I*2×···×*IN* with r*ankn*(A-) <sup>=</sup> *Rn* <sup>≤</sup> <sup>r</sup>*ankn*(A). The approximated tensor <sup>A</sup>- is the *<sup>n</sup>*-mode product <sup>×</sup>*<sup>n</sup>* of factor matrices **<sup>U</sup>**(*n*) <sup>∈</sup> <sup>R</sup>*In*×*Rn* and a core tensor <sup>B</sup> <sup>∈</sup> R*<sup>R</sup>*1×*R*2×···×*RN* in a given reduced rank space (Eq. (14)). This rank-(*R*1, *R*2,..., *RN* ) approximation was previously introduced as the *Tucker* model. A-

$$\widetilde{\mathcal{R}} = \mathcal{B} \times\_1 \mathbf{U}^{(1)} \times\_2 \mathbf{U}^{(2)} \times\_3 \cdots \times\_N \mathbf{U}^{(N)} \tag{14}$$

In general, a rank-reduced approximation is sought such that the least-squares cost function of Eq. (15) is minimized.

Tensor Approximation for Multidimensional and Multivariate Data 85

$$\text{dimensional and Multivariant Data}\tag{85}$$

$$\widetilde{\mathcal{H}} = \arg\min\_{\widetilde{\mathcal{H}}} \|\mathcal{H} - \widetilde{\mathcal{H}}\|\:^2\tag{15}$$

Given that (*R*1, *R*2,..., *RN* ) are sufficiently smaller than the initial dimensions (*I*1, *<sup>I</sup>*2,..., *IN* ), the core coefficients <sup>B</sup> <sup>∈</sup> <sup>R</sup>*<sup>R</sup>*1×*R*2×···×*RN* and the factor matrices **<sup>U</sup>**(*n*) <sup>∈</sup> <sup>R</sup>*In*×*Rn* can lead to a compact approximation of <sup>A</sup> of the original tensor A. In particular, the multilinear rank-(*R*1, *R*2,..., *RN* ) is typically explicitly chosen to be smaller than the initial ranks in order to achieve a compression of the input data (see also [2, 4, 6]).

#### *5.2 Truncated Tensor Decomposition*

Similar to matrix SVD, tensor rank reduction can be used to generate lower-rank reconstructions <sup>A</sup> of the input A. The tensor rank parameters *Rn* are chiefly responsible for the number of TA bases and coefficients that are used for the reconstruction and hence are responsible for the approximation level. In higher orders, the CP decomposition is not directly rank-reducible, however, the truncation of the Tucker decomposition is possible due to the *all-orthogonality* property of the core tensor.

For a 3rd-order tensor, all-orthogonality means that the different horizontal matrix slices of the core B (the first index *i*<sup>1</sup> is kept fixed, while the two other indices, *i*<sup>2</sup> and *i*3, are free) are mutually orthogonal with respect to the scalar product of matrices (i.e., the sum of the products of the corresponding entries vanishes). The same holds for the other slices with fixed indices *i*<sup>2</sup> and *i*<sup>3</sup> (see [14]). Therefore, given an initial sufficiently accurate rank-(*R*1, *R*2, *R*3) Tucker model, we can progressively choose lower ranks *Kn* < *Rn* for reduced quality reconstructions. As indicated in Fig. 12, the ranks *Kn* indicate how many factor matrix columns and corresponding core tensor entries are used for the reconstruction.

Note that the ordering of the coefficients in the Tucker core tensor B is not strictly decreasing in contrast to the decreasing singular values in the matrix SVD case. However, in practice it can be shown that progressive tensor rank reduction in the Tucker model works very well for adaptive reconstruction of the data at different accuracy levels.

**Fig. 12** Illustration of a rank reduced Tucker tensor reconstruction: A reduced range of factor matrix columns with corresponding fewer core tensor entries reconstructs a lower quality approximation but at full resolution

#### **6 Tucker Decomposition Algorithms**

There are various strategies for how to compute and generate a tensor decomposition. The most popular and widely used group of algorithms belongs to the *alternating least squares* (ALS) algorithms, the other group of algorithms uses various Newton methods. The respective algorithms differ for the computation of the different tensor models, and we will mainly focus on the Tucker model in our review.

For the Tucker model, the first decomposition algorithms used were a simple higher-order SVD (HOSVD) (see [14]), the so-called *Tucker1* [52], a three-mode SVD. However, the truncated decompositions of higher orders are not optimal in terms of best fit, which is measured by the Frobenius norm of the difference. Starting from a HOSVD algorithm, tensor approximation ALS algorithms [26, 28] were developed, where one of the first Tucker ALS was the so-called *TUCKALS* [49]. Later various improvements accelerated [1] or optimized the basic TUCKALS method. The *higher-order orthogonal iteration* (HOOI) algorithm [15] is an iterative algorithm that performs a better fit for a truncated HOSVD version.

Newton methods are also used for the Tucker decomposition or rank-(*R*1, *R*2,..., *RN* ) approximation. They typically start with a HOOI initialization and then converge faster to the final point. [16] developed a *Newton-Grassman optimization* approach, which takes much fewer iterations than the basic HOOI - even though one single iteration is more expensive due to the computation of the Hessian. While the HOOI is not guaranteed to converge, the Newton-Grassmann Tucker decomposition is guaranteed to converge to a stationary point. Another Newton method was proposed by [22], who developed a *differential-geometric Newton* algorithm with a fast quadratic convergence of the algorithm in a neighborhood of the solution. Since this method is not guaranteed to converge to a global maximum, they support the method by starting with an initial guess of several HOOI iterations, which increases the chances of converging to a solution.

#### **7 Tensor Reconstruction**

The *tensor reconstruction* from a reduced-rank tensor decomposition can be achieved in multiple ways. One alternative is a progressive reconstruction: Each entry in the core tensor B is considered as weight for the outer product between the corresponding column vectors in the factor matrices **U**(*n*) . This gives rise to Eq. (16) for the Tucker reconstruction. = ...

$$
\widetilde{\mathbf{M}} = \sum\_{r\_1=1}^{R\_1} \dots \sum\_{r\_N=1}^{R\_N} \dots \sum\_{r\_N=1}^{R\_N} \mathbf{u}\_{r\_1 r\_2 \dots r\_N} \cdot \mathbf{u}\_{r\_1}^{(1)} \diamond \mathbf{u}\_{r\_2}^{(2)} \diamond \cdots \diamond \mathbf{u}\_{r\_N}^{(N)} \tag{16}
$$

$$
\widetilde{\mathbf{M}} = \sum\_{r\_1=1}^{R\_1} \sum\_{r\_2=1}^{R\_2} \dots \sum\_{r\_N=1}^{R\_N} b\_{r\_1 r\_2 \dots r\_N} \cdot \mathbf{u}\_{r\_1}^{(1)} \diamond \mathbf{u}\_{r\_2}^{(2)} \diamond \cdots \diamond \mathbf{u}\_{r\_N}^{(N)} \tag{16}
$$

This reconstruction strategy corresponds to forming rank-one tensors and cumulatively summing them up. The accumulated weighted *subtensors* then form the approximation <sup>A</sup> of the original data A. In particular for the Tucker model, this is an expensive reconstruction strategy since it involves multiple for-loops to run over all the summations, for a total cost of *O*(*R<sup>N</sup>* · *I <sup>N</sup>* ) operations.

#### *7.1 Element-Wise Reconstruction*

A simple approach is to reconstruct each required element of the approximated dataset individually, which we call *element-wise reconstruction*. Each element*ai*1*i*2*i*<sup>3</sup> is reconstructed, as shown in Eq. (17) for the Tucker reconstruction. That is, all core coefficients multiplied with the corresponding coefficients in the factor matrices are summed up (weighted sum). *ai*1*i*2...*iN* =

$$\widetilde{a}\_{i\_1 i\_2 \dots i\_N} = \sum\_{r\_1 r\_2 \dots r\_N} b\_{r\_1 r\_2 \dots r\_N} \cdot \mu\_{i\_1 r\_1}^{(1)} \cdot \mu\_{i\_2 r\_2}^{(2)} \cdot \dots \cdot \mu\_{i\_N r\_N}^{(N)} \tag{17}$$

Element-wise reconstruction requires *O*(*R<sup>N</sup>* ) operations on average. It can be beneficial for applications where only a sparse set of reconstructed elements are needed.

#### *7.2 Optimized Tucker Reconstruction*

A third reconstruction approach—applying only to the Tucker reconstruction—is to build the *n*-mode products along every mode, which leads to a TTM multiplication for each mode, e.g., TTM1 along mode 1, (see also Eq. (5)). This is analogous to the Tucker model given by Eq. (14). The intermediate results are then multiplied along the next modes, e.g., TTM2 and finally TTM3. In Fig. 13 we visualize the TTM reconstruction, and the intermediate results B and B as well as the final approximation <sup>A</sup>-, applied to a 3rd-order tensor using *n*-mode products.

**Fig. 13** Forward cyclic TTM multiplications after [24] along the three modes (*n*-mode products)

Since it exploits matrix multiplications, this optimized algorithm is much faster than progressive reconstruction (Eq. (16)). Its cost is dominated by *O*(*R*<sup>1</sup> · *I <sup>N</sup>* ). In 3D, in particular, it takes *O*(*R*<sup>1</sup> · *I*<sup>1</sup> · *I*<sup>2</sup> · *I*<sup>3</sup> + *R*<sup>1</sup> · *R*<sup>2</sup> · *I*<sup>2</sup> · *I*<sup>3</sup> + *R*<sup>1</sup> · *R*<sup>2</sup> · *R*<sup>3</sup> · *I*3) operations.

#### **8 Useful TA Properties for Scientific Visualization**

As stated in the introduction, TA is the higher-order generalization of the matrix SVD, which can offer either properties of (a) rank-*R* decomposition or (b) orthonormal row-space and column-space vectors. In higher orders, the orthonormal row and column vectors are preserved in the Tucker model which thus supports progressive rank-reduced approximations and reconstructions.

#### *8.1 Spatial Selectivity and Subsampling*

The Tucker model (Sect. 4.1) consists of one factor matrix per mode (data direction) **<sup>U</sup>**(**n**) <sup>∈</sup> <sup>R</sup>*In*×*Rn* and one core tensor <sup>B</sup> <sup>∈</sup> <sup>R</sup>*R*1×*R*2×···×*RN* . The core tensor <sup>B</sup> is in effect a projection of the original data <sup>A</sup> onto the basis of the factor matrices **<sup>U</sup>**(**n**) . In case of a volume, the Tucker model has three modes, as illustrated in Fig. 10, and defines an approximation <sup>A</sup>- <sup>=</sup> <sup>B</sup> <sup>×</sup><sup>1</sup> **<sup>U</sup>**(1) <sup>×</sup><sup>2</sup> **<sup>U</sup>**(2) <sup>×</sup><sup>3</sup> **<sup>U</sup>**(3) of the original volume <sup>A</sup> (using *n*-mode products ×*n*).

The row and column axes of the factor matrices represent two different spaces: (1) the rows correspond to the spatial dimension in the corresponding mode, and (2) the columns to the approximation quality. These two properties can be exploited for multiresolution modeling (spatial selection and subsampling of rows) and multiscale approximation (rank reduction on the columns) (see also Fig. 14). In [5, 47] we demonstrated how these features can be exploited for multiresolution and multiscale reconstruction as well as filtering of compressed volume data in the context of interactive visualization.

**Fig. 14** Factor matrix properties along the vertical axis supporting: (1.1) spatial selectivity, (1.2) spatial subsampling, and (2) low-rank approximation

#### *8.2 Approximation and Rank Reduction*

As described earlier, the Tucker model defines a rank-(*R*1, *R*2, *R*3) approximation, where a small *Rn* corresponds to a low-rank approximation (with details removed) and a large *Rn* corresponds to a more accurate approximation of the original. The highest rank *Rn* for the initial Tucker decomposition has to be given explicitly. However, rank reductions can be applied after the initial decomposition (similar to the rank reduction in matrix SVD). Even though the core tensor coefficients are not guaranteed to be in decreasing order, as in matrix SVD, in practice it can be shown that progressive tensor rank reduction in the Tucker model works well for adaptive visualization of the data at different feature scales [2, 4–6, 47].

As illustrated in Fig. 12 in Sect. 5.2, the ranks indicate how many factor matrix columns and corresponding core tensor entries are used for a desired reconstruction. Thus, given a rank-(*R*1, *R*2, *R*3) Tucker model, we can specifically or progressively choose lower ranks *Kn* < *Rn* for a reduced quality reconstruction, at the original spatial output resolution given by *In* (or also subsampled).

Figure 15 shows the progressive rank reduction from an initial rank-(256, 256, 256) Tucker decomposition of an original 512<sup>3</sup> example volume. Shown are the visual results and the data reduction of the approximation at variable reduced-rank reconstructions. The numbers of coefficients include all core tensor and factor matrix entries that are used, e.g. a rank-(32, 32, 32) reconstruction corresponds to 32<sup>3</sup> + 3 · 512 · 32 = 81 920 coefficients. The data reduction ratio can be derived by dividing the number of coefficients by 5123, which for *R* = 32 results in using only 0.06% of the original amount of data. In particular, Fig. 15 demonstrates the power of lowrank tensor approximations that can be used for multiscale feature visualization or progressive image refinement in volume rendering.

**Fig. 15** Multiscale volume visualization by tensor rank reduction, corresponding to 0.02, 0.06, 0.27, 1.71 and 12.79% of the original amount of data used for *R* = 16, 32, 64, 128, 256

#### **9 Application to Multivariate Data**

Encouraged by the data reduction power and the approximation quality of tensor approximations as shown in Fig. 16, we have extended the Tucker decomposition and reduced rank reconstruction to vector field data. Compared to scalar 3D volumes (e.g. from MRI or CT scans), a 3D vector field V (e.g. of velocities from a weather or fluid simulation) can be interpreted as a multivariate data field with *D*-dimensional vector-valued entries at each position in a *I*<sup>1</sup> × *I*<sup>2</sup> × *I*<sup>3</sup> grid. Thus we can interpret V as a 4th-order tensor <sup>V</sup> <sup>∈</sup> <sup>R</sup>*I*1×*I*2×*I*3×*<sup>D</sup>*. Note that the TA rank-reduction (Sect. 5) is only applied to the three spatial dimension as we are not interested in a dimensionality reduction of the vector-valued field value itself.

#### *9.1 Dataset*

For our first experiments we used a data set from the *Johns Hopkins Turbulence Database* http://turbulence.pha.jhu.edu/ representing a direct numerical simulation of incompressible magnetohydrodynamic (MHD) equations (see also Fig. 17). The data set contains, among other output variables, 3 velocity components that we used, hence *D* = 3, covers a 3D grid of 2563 cells (downsampled subset from the original 10243), hence *I*<sup>1</sup> = *I*<sup>2</sup> = *I*<sup>3</sup> = 256, and is the first time step from the output of the simulation.

The vector field therefore covers a 3D cube which we visualize using direct volume and streamline rendering techniques with color-coding of velocity, divergence, vorticity or error magnitudes in ParaView. Transparency is used to reduce clutter and opacity such as to focus the visualizations on the high-magnitude value regions. Note that this cubic vector field is dense and rendered over a black background, thus always appearing as a cube like object in the images.

**Fig. 16 a** A 512<sup>3</sup> isotropic turbulence volume of 512MB, **b** visually identical compression result using 51.2MB and **c** result after extreme compression down to only 1.71MB using the TTHRESH method [2]. All the visualizations are generated using ParaView

#### *9.2 Vector Field Magnitude and Angle*

As can be seen in Fig. 17, the velocity magnitude and structure of the flow directions is very well maintained down to fairly course reconstructions using *R* = 32. Especially the (important) regions with larger velocity magnitudes are very well preserved with respect to their flow orientation as visible from the streamline visualization. The rendering applies an opacity transfer function setting which is almost linear, slight curve below the diagonal, to the *vector field magnitude*, highlighting the important high-velocity regions.

To see how many details are lost after rank reduction of the velocity field data set, we calculated and visualized the error information for both magnitude and angle deviation in Figs. 18 and 19, respectively. For the magnitude, we normalized the error to be the percentage relative to the local vector field magnitude value. The rendering uses an opacity mapping which is almost linear, slight curve below the diagonal, to the *vector field magnitude*. This makes low magnitude and low opacity areas, and their errors, transparent or dark and highlights any errors in the more critical high-velocity regions. The completely dark images in Fig. 18e–h demonstrate that the magnitude error after rank reduction using Tucker decomposition is close to zero even with for *R* = 16, which is 1/16 of the original full rank of 256. The barely noticeable dark blue regions in Fig. 18a–d correspond to low errors in high-velocity regions, with errors in the range of less than 10% down to 10−8% (with 60% being white).

Compared to the almost negligible relative magnitude errors of the velocity field, however, the angular error of the same data set after rank reduction is more prominent as shown in Fig. 19. The maximum angular error π = 180◦ is shown in red while small errors are shown in dark blue (0◦). The rendering applies an opacity setting almost linear to the *angle offset*, hence in Fig. 19 large angular errors are highlighted.

**Fig. 17** Scalar velocity magnitude (top) and vector field streamline (bottom) visualization of the forced MHD turbulence simulation at variable reduced rank reconstructions *R* = 32, 64, 128, 256

**Fig. 18** Relative magnitude error (in %) of the velocity field with reduced rank reconstructions *R* = 1, 2, 4, 8, 16, 32, 64, 128

**Fig. 19** Absolute angular error [0◦, 180◦] of the velocity field with reduced rank reconstructions *R* = 1, 2, 4, 8, 16, 32, 64, 128

However, this does not relate to the local strength of the vector field since large angle differences in low-magnitude areas are not that important. Directly comparing Figs 17 and 19, which have the same viewing configuration, one can observe that large angular errors occur mostly in very low-velocity areas and may thus not be that relevant. In particular, since for very short vectors small errors can cause large angular changes.

#### *9.3 Vorticity and Divergence*

We also inspected the influence of low-rank tensor compression on the two most fundamental features of vector field data: divergence and vorticity. As can be seen in Fig. 20, the global structure of the magnitude from both the divergence field and the vorticity field are still very well preserved with rank *R* = 64, which is 1/4 of the original full rank of 256. The values are given in their absolute ranges of[0.36, −0.43] and [0.52, 10−5] for the divergence and vorticity respectively. The rendering applies an opacity setting which is almost linear, slight curve below the diagonal, to the *magnitude values*, highlighting the important regions.

For the vorticity vector field, we also calculated the relative magnitude error as for the velocity vector field data shown in Fig. 18, thus as percentage of the vorticity magnitude, and similar conclusions can also be drawn here for the results shown in Fig. 21. All errors are very low, and barely noticeable for rank-reductions down to *R* = 16, which corresponds to 1/16 of the original data volume.

Additionally, we also computed the absolute angular error for the vorticity vector field, in this case applying an opacity proportional to the *magnitude of the raw vector field*, and the results are shown in Fig. 22. We can observe that the angular error almost everywhere is in the color range of dark to light blue which maps to errors from 0 up to 45◦ in this plot, and thus the directional vorticity information seems to be well preserved.

**Fig. 20** Magnitude of the vector field's divergence (top) and vorticity (bottom) at variable reduced rank reconstructions *R* = 32, 64, 128, 256

**Fig. 21** Relative magnitude error (in %) of the vorticity field with reduced rank reconstructions *R* = 1, 2, 4, 8, 16, 32, 64, 128

**Fig. 22** Absolute angular error [0◦, 130◦] of the vorticity field with reduced rank reconstructions *R* = 1, 2, 4, 8, 16, 32, 64, 128

#### **10 Conclusions**

In the analysis conducted in this first study on vector fields, we have shown that tensor approximation methods are not only very useful for multidimensional scalar fields but can also be applied to multivariate data, thus extending to vector and possibly tensor fields. In general we can observe that for the important high-velocity vector field regions, the low-rank reconstructions maintain the important overall structures of the flow features, in particular also the vorticity. Furthermore, we note that the MHD simulation model in theory should be divergence-free and thus the numerical results report very low divergence numbers. Therefore, we are very satisfied with the result that the low-rank tensor reconstructions do not result in an unexpected and uncontrolled enlargement of these divergence values, keeping them within the numerical range of the simulation.

**Acknowledgements** This work was partially supported by the University of Zurich's Forschungskredit "Candoc" (grant numbers FK-16-012 and 53511401), a Swiss National Science Foundation grant (SNF) (project n◦200021\_132521), a Hasler Foundation grant (project number 12097), and the EU FP7 People Programme (Marie Curie Actions) under REA Grant Agreement n◦290227. Furthermore, we would like to acknowledge the *Computer-Assisted Paleoanthropology* group and the *Visualization and MultiMedia Lab* at University of Zürich for the acquisition of the Hazelnut dataset (https://www.ifi.uzh.ch/en/vmml/research/datasets.html in Fig. 15. Also we acknowledge the Johns Hopkins Turbulence Database http://turbulence.pha.jhu.edu/ for the data used in Fig. 16 as well as their forced MHD simulation data http://turbulence.pha.jhu.edu/Forced\_ MHD\_turbulence.aspx used in our experiments.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Fourth-Order Anisotropic Diffusion for Inpainting and Image Compression**

**Ikram Jumakulyyev and Thomas Schultz**

**Abstract** Edge-enhancing diffusion (EED) can reconstruct a close approximation of an original image from a small subset of its pixels. This makes it an attractive foundation for PDE based image compression. In this work, we generalize second-order EED to a fourth-order counterpart. It involves a fourth-order diffusion tensor that is constructed from the regularized image gradient in a similar way as in traditional second-order EED, permitting diffusion along edges, while applying a non-linear diffusivity function across them. We show that our fourth-order diffusion tensor formalism provides a unifying framework for all previous anisotropic fourth-order diffusion based methods, and that it provides additional flexibility. We achieve an efficient implementation using a fast semi-iterative scheme. Experimental results on natural and medical images suggest that our novel fourth-order method produces more accurate reconstructions compared to the existing second-order EED.

#### **1 Introduction**

The increased availability and resolution of imaging technology, including digital cameras and medical imaging devices, along with advances in storage capacity and transfer bandwidths, have led to a proliferation of large image data. This makes image compression an important area of research. Image compression techniques can be divided into two main groups: Lossy and lossless compression. Lossless compression techniques permit restoration of the full, unmodified image data, which however limits the achievable compression rates. Our work is concerned with lossy compression, which achieves much higher compression rates by replacing the original image with an approximation that can be stored more efficiently.

I. Jumakulyyev · T. Schultz (B)

B-IT and Department of Computer Science II, University of Bonn, Friedrich-Hirzebruch-Allee 5, 53115 Bonn, Germany e-mail: schultz@cs.uni-bonn.de

I. Jumakulyyev e-mail: ijumakulyyev@cs.uni-bonn.de

We continue a line of research that has explored the use of Partial Differential Equations (PDEs) for lossy image compression [12, 13, 23, 28]. This approach is based on storing only a small subset of all pixels, and interpolating between them in order to restore the remaining ones. There is a strong similarity between that interpolation process and image inpainting, whose goal it is to reconstruct missing or corrupted parts of an image. PDE-based methods for image inpainting and compression are inspired by the physical phenomenon of heat transport. It is described by the heat diffusion equation

$$
\partial\_t u = \text{div}(D \cdot \nabla u) \,, \tag{1}
$$

which relates temporal changes in a heat concentration ∂*tu* to the divergence of its spatial gradient ∇*u*. When diffusion takes place in an isotropic medium, the diffusivity *D* is a scalar that determines the rate of heat transfer. In an anisotropic medium, heat spreads out more rapidly in some directions than in others. In those cases, *D* is a diffusion tensor, i.e., a symmetric matrix that encodes this directional dependence.

When applied to image processing, the gray value at a certain location is interpreted as the heat concentration *u*. In diffusion-based image inpainting, Eq. (1) is used to propagate information from the known pixels, whose intensity is fixed, to the unknown pixels which will ultimately reach a steady state in which their intensity is determined by their surrounding known pixels. In this sense, Eq. (1) has a filling-in effect that can be exploited for image compression.

Different choices of the diffusivity function *D* lead to different kinds of diffusion. Linear diffusion [20] and nonlinear diffusion [24] were widely used for image smoothing and image enhancement. Edge structures in images can be enhanced by employing a diffusion tensor which allows diffusion in the direction perpendicular to the local gradient, while applying a nonlinear diffusivity function along the gradient direction. This idea has led to the development of anisotropic nonlinear edge-enhancing diffusion (EED) [34]. Among the six variants that were evaluated for image compression by Gali´c et al. [13], EED led to the most accurate reconstructions. Subsequently, this idea was applied to three-dimensional data compression [25], and combined with motion compensation in order to obtain a framework for video compression [2]. When combined with a suitable scheme for selecting and storing the preserved pixels, a few additional optimizations, and at sufficiently high compression rates, anisotropic diffusion has been shown to beat the quality even of JPEG2000 [27].

In this paper, we introduce a novel fourth-order PDE that generalizes second-order EED, and achieves even more accurate reconstructions. We build on prior works that proposed fourth-order analogs of the diffusion equation, and used them for image processing [10, 19, 21, 22, 26, 35]. In particular, we extend a work by Gorgi Zadeh et al. [15], who introduced the idea of steering anisotropic fourth-order diffusion with a fourth-order diffusion tensor. However, their method focuses on the curvature enhancement property of nonlinear fourth-order diffusion [10] in order to better localize ridge and valley structures. Deriving a suitable PDE for image inpainting requires a different definition of the diffusion tensor, more similar to the one in edgeenhancing diffusion [34]. Two anisotropic fourth-order PDEs for inpainting were previously introduced by Li et al. [21]. However, they only apply them to image restoration tasks in which small parts of an image are missing (such as in Fig. 8), not to the reconstruction from a small subset of pixels. Moreover, we demonstrate that the fourth-order diffusion tensor based framework is more general in the sense that it can be used to express anisotropic fourth-order diffusion as it was described by Hajiaboli [19] or by Li et al. [21], while providing additional flexibility.

#### **2 Background and Related Work**

We will now formalize the above-mentioned idea of diffusion-based inpainting (Sect. 2.1), and review two concepts that play a central role in our method: Anisotropic nonlinear diffusion (Sect. 2.2) and fourth-order diffusion (Sect. 2.3). Further details can be found in works by Gali´c et al. [13] and Weickert [34], respectively. Finally, we provide additional context with a brief discussion of alternative approaches to image compression (Sect. 2.4).

#### *2.1 Diffusion-Based Inpainting*

In order to apply Eq. (1) to image smoothing, we have to restrict it to the image domain , and specify the behavior along its boundary ∂. It is common to assume that no heat is transferred through that boundary (homogeneous Neumann boundary condition). Moreover, the positive real line (0,∞) is typically taken as the time domain. The resulting PDE can be written as

$$\begin{aligned} \partial\_t u &= \text{div}(D \cdot \nabla u), \quad \Omega \times (0, \infty) \end{aligned} \quad \text{,} \quad \begin{aligned} \Omega \times (0, \infty) \end{aligned} \quad \text{,} \quad \begin{aligned} \partial\_t \alpha &= 0, \\ \partial\_t \alpha &= 0, \end{aligned} \quad \text{,} \quad \begin{aligned} \Omega &= \int\_0^t \nabla u \cdot \left( \frac{1}{2} u \nabla \alpha \right) \end{aligned} \quad \text{,} \quad \begin{aligned} \Omega &= \int\_0^t \nabla u \cdot \left( \frac{1}{2} u \nabla \alpha \right) \end{aligned}$$

where *<sup>n</sup>* is the normal vector to the boundary ∂. The original image *<sup>f</sup>* : <sup>→</sup> <sup>R</sup> is used to specify an initial condition *u* = *f* at *t* = 0. For increasing diffusion time *t*, *u* will correspond to an increasingly smoothed version of the image.

In image inpainting, we know the pixel values on a subset *K* ⊂ of the image, and aim to reconstruct plausible values in the unknown regions. A diffusion-based model for inpainting can be derived from the one for smoothing, by modeling the set of locations at which the pixel values are known with Dirichlet boundary conditions. In this case, *<sup>f</sup>* : *<sup>K</sup>* <sup>→</sup> <sup>R</sup> will be used to model the known values. In inpainting-based image compression, *K* will consist of a small fraction of the pixels in the original image. With this, we obtain the following model for inpainting:

$$\begin{aligned} \partial\_t u &= \text{div}(D \cdot \nabla u), & \Omega \backslash K \times (0, \infty) \text{ }, \\ \partial\_n u &= 0, & \partial \Omega \times (0, \infty) \text{ }, & \Omega \backslash K \end{aligned} \tag{3}$$
 
$$u = f, \quad K \times [0, \infty)$$

In this case, the diffusion process spreads out the information from the known pixels to their spatial neighborhood. For time *t* → ∞, image smoothing and inpainting both converge to a steady state, i.e., lim*<sup>t</sup>*→∞ ∂*tu* = 0. However, the steady-state of smoothing is trivial (*u* approaches a constant image with average gray value), while the Dirichlet boundary conditions in the inpainting case ensure a non-trivial steady-state, which is taken as the final inpainting result: *u*inpainted = lim*<sup>t</sup>*→∞ *u*.

#### *2.2 From Linear to Anisotropic Nonlinear Diffusion*

So far, we assumed that the diffusion coefficient *D* is a scalar and constant, independent from the location within the image. This results in an inpainting model based on linear homogeneous diffusion [20]. With *D* = 1, it can be written as

$$
\partial\_t \mu = \Delta u, \quad \Omega \backslash K \times (0, \infty) \,. \tag{4}
$$

In this and all remaining equations in this section, the same boundary conditions are assumed as specified in Eq. (3). Despite its simplicity, it has been demonstrated that using this inpainting model for image compression can already beat the JPEG standard when applied to cartoon-like images, and selecting the retained pixels to be close to image edges [23].

When the diffusion coefficient is a scalar but depends on *u*, i.e., *D* = *g*(*u*), then we call the model inpainting based on nonlinear isotropic diffusion [24]. A common variant is to make *D* depend on the local gradient magnitude, i.e.,

$$\partial\_t \mu = \text{div}(\text{g}(||\nabla u\_{\sigma}||^2) \nabla \mu), \quad \Omega \backslash K \times (0, \infty) \,, \tag{5}$$

where *g* is a decreasing nonnegative diffusivity function, e.g., the Charbonnier diffusivity

$$\log(s^2) = \frac{1}{\sqrt{1 + \frac{s^2}{\lambda^2}}}\,,\tag{6}$$

and λ is a contrast parameter separating low from high diffusion areas [8]. In order to localize edges better and to make the problem well-posed, the image is presmoothed with a Gaussian before taking its gradient, i.e., *g*(||∇*u*<sup>σ</sup> ||<sup>2</sup>) is used instead of *g*(||∇*u*||<sup>2</sup>) [6].

In the above-discussed models, the diffusion occurs only in the gradient direction. This can be changed by replacing the scalar diffusivity with a second-order diffusion tensor, i.e., a symmetric positive definite matrix. This is the basis of anisotropic nonlinear diffusion [34],

$$
\partial\_t \mu = \text{div}(\mathbf{D} \cdot \nabla \mu), \quad \Omega \backslash K \times (0, \infty) \,. \tag{7}
$$

In edge-enhancing diffusion (EED), the diffusion tensor **D** is defined as

$$\mathbf{D} = \mathbf{g} \left( ||\nabla \boldsymbol{u}\_{\sigma}||^{2} \right) \cdot \mathbf{v}\_{1} \mathbf{v}\_{1}^{\mathrm{T}} + \mathbf{1} \cdot \mathbf{v}\_{2} \mathbf{v}\_{2}^{\mathrm{T}},\tag{8}$$

where **<sup>v</sup>**<sup>1</sup> <sup>=</sup> <sup>∇</sup>*u*<sup>σ</sup> ||∇*u*<sup>σ</sup> ||<sup>2</sup> and **<sup>v</sup>**<sup>2</sup> <sup>=</sup> <sup>∇</sup>*u*<sup>⊥</sup> σ ||∇*u*<sup>σ</sup> ||<sup>2</sup> . This means that diffusion across the edge (**v**1) is decreased depending on the gradient magnitude, while diffusion along the edge (**v**2) is allowed. Examples of EED based inpainting are included in our experimental results. In general, EED based inpainting results in better interpolated images than linear homogeneous or nonlinear isotropic PDEs. This makes it a current state-ofthe-art choice for PDE-based image compression.

#### *2.3 From Second to Fourth Order Diffusion*

All models discussed above, as well as several others that have been proposed for inpainting [36], share a common property: They rely on second order PDEs. In image denoising, higher-order PDEs have a long history, going back to work by Scherzer [26]. You and Kaveh [35] propose fourth-order PDEs as a solution to the so-called staircasing problem that arises in edge-enhancing second-order PDEs, such as the filter proposed by Perona and Malik [24]: While the second-order Perona-Malik equation creates visually unpleasant step edges from continuous variations of intensity, corresponding fourth-order methods move these discontinuities into the gradients, where they are less noticeable to the human eye [16]. Subsequently, other fourth-order PDE-based models have been introduced, and have mostly been applied for denoising [18, 19, 22].

For a specific family of higher-order diffusion filters, Didas et al. [10] have shown that, in addition to preserving average gray value, they also preserve higher moments of the initial image. Moreover, depending on the diffusivity function, they can lead to adaptive forward and backward diffusion, and therefore to the enhancement of image features such as curvature. Gorgi Zadeh et al. [15] made use of this property in order to enhance ridges and valleys, by steering fourth-order diffusion with a fourth-order diffusion tensor. Our work adapts their method in order to achieve accurate inpainting and reconstruction from a small subset of pixels.

#### *2.4 Alternative Approaches to Image Compression*

The dominant lossy image compression techniques today are JPEG and JPEG2000. They are based on the discrete cosine transform (DCT) and wavelet transform, respectively. However, they are not sensitive to the geometry of an image, i.e. those standards are not tailored to their geometrical behavior [5]. Especially, the JPEG standard involves dividing the image into small square blocks. This can cause a degradation called "blocking effect" [30], and can result in unsatisfactory reconstructions especially at high compression rates.

It is an ongoing research trend to apply machine learning methods to image compression, such as convolutional and recurrent neural networks [3, 31, 32]. Learning based approaches tend to perform very well on the specific class of images on which they were trained, but require a huge amount of data. For example, Toderici et al. [33] used for training a dataset of 6 million 1280 × 720 images taken from the web.

#### **3 Method**

We will now introduce our novel PDE (Sect. 3.1), investigate its relationship to previously proposed anisotropic fourth-order diffusion (Sect. 3.2), and comment on our chosen discretization, as well as numerical stability (Sect. 3.3).

#### *3.1 Anisotropic Edge-Enhancing Fourth Order PDE*

Our fourth-order PDE builds on a model that was proposed by Gorgi Zadeh et al. [15] for ridge and valley enhancement. It can be stated concisely using Einstein notation, where summation is implied for indices appearing twice in the same expression:

$$\partial\_t u = -\partial\_{ji} \left[ \mathcal{D}(\mathbf{H}\_\rho(u\_\sigma)) : \mathbf{H}(u) \right]\_{ij} \tag{9}$$

In this equation, **H**(*u*) denotes the Hessian matrix of image *u*. The "double dot product" **T** = D : **H** indicates that matrix **T** is obtained by applying a linear map D to **H**, and the square bracket notation [**T**]*i j* indicates taking the (*i*, *j*)th component:

$$\mathbf{f}[\mathbf{T}]\_{ij} = \left[\mathcal{D}(\mathbf{H}\_{\rho}(\mu\_{\sigma})) : \mathbf{H}(\mu)\right]\_{ij} = \left[\mathcal{D}(\mathbf{H}\_{\rho}(\mu\_{\sigma}))\right]\_{ijkl}[\mathbf{H}(\mu)]\_{kl} \tag{10}$$

Since D maps matrices to matrices, it is a fourth-order tensor. Since its role is analogous to the second-order diffusion tensor in Eq. (7), it is referred to as a fourthorder diffusion tensor.

The diffusion tensor D in Eq. (9) is a function of the local normalized Hessian **H**ρ(*u*<sup>σ</sup> ) which contains the information that is relevant to achieve curvature enhancement. For image inpainting, we propose to instead steer the fourth-order diffusion in analogy to edge-enhancing diffusion, i.e., as a function of the structure tensor **J**(*u*<sup>σ</sup> ), which is obtained from image *u* after Gaussian pre-smoothing with bandwidth σ. We construct our fourth order diffusion tensor D from its eigenvalues μ*<sup>i</sup>* and eigentensors **E***<sup>i</sup>* via the spectral decomposition:

$$\mathcal{D}\left(\mathbf{J}(\mu\_{\sigma})\right) = \mu\_1 \mathbf{E}\_1 \otimes \mathbf{E}\_1 + \mu\_2 \mathbf{E}\_2 \otimes \mathbf{E}\_2 + \mu\_3 \mathbf{E}\_3 \otimes \mathbf{E}\_3 + \mu\_4 \mathbf{E}\_4 \otimes \mathbf{E}\_4 \tag{11}$$

The eigenvalues and eigentensors are defined as

$$\begin{aligned} \mu\_1 &= g(\lambda\_1), \ \mathbf{E}\_1 = \mathbf{v}\_1 \otimes \mathbf{v}\_1 \ , \\ \mu\_2 &= 1, \ \mathbf{E}\_2 = \mathbf{v}\_2 \otimes \mathbf{v}\_2 \ , \\ \mu\_3 &= \sqrt{g(\lambda\_1)}, \ \mathbf{E}\_3 = \frac{1}{\sqrt{2}} (\mathbf{v}\_1 \otimes \mathbf{v}\_2 + \mathbf{v}\_2 \otimes \mathbf{v}\_1) \ , \\ \mu\_4 &= 0, \ \mathbf{E}\_4 = \frac{1}{\sqrt{2}} (\mathbf{v}\_1 \otimes \mathbf{v}\_2 - \mathbf{v}\_2 \otimes \mathbf{v}\_1) \ , \end{aligned} \tag{12}$$

where *g* is a nonnegative decreasing diffusivity function, λ*<sup>i</sup>* and **v***<sup>i</sup>* are eigenvalues and eigenvectors of the structure tensor **J**(*u*<sup>σ</sup> ) = ∇*u*<sup>σ</sup> ∇*u*<sup>T</sup> <sup>σ</sup> , i.e., λ<sup>1</sup> = ||∇*u*<sup>σ</sup> ||<sup>2</sup> 2, **v**<sup>1</sup> = ∇*u*<sup>σ</sup> ||∇*u*<sup>σ</sup> ||<sup>2</sup> and <sup>λ</sup><sup>2</sup> <sup>=</sup> 0, **<sup>v</sup>**<sup>2</sup> <sup>=</sup> <sup>∇</sup>*u*<sup>⊥</sup> σ ||∇*u*<sup>σ</sup> ||<sup>2</sup> . The above-defined eigentensors are orthonormal with respect to the dot product **A** : **B** = trace(**B**T**A**) [15].

Combining this new definition of the fourth-order diffusion tensor with Dirichlet boundary conditions as in Eq. (3) results in our proposed model:

$$\begin{aligned} \partial\_t u &= -\partial\_{ji} [\mathcal{D}(\mathbf{J}(u\_\sigma)) : \mathbf{H}(u)]\_{ij}, \quad \mathcal{Q} \backslash K \times (0, \infty) \,, \\\ u &= f, \quad K \times [0, \infty) \end{aligned} \tag{13}$$

As it is customary in PDE-based inpainting, we allow Eq. (13) to evolve until a steady state has been reached, i.e., the time derivative becomes negligible. In our numerical implementation, we use a Fast Semi-Iterative Scheme (FSI) [17] to greatly accelerate convergence to a large stopping time.

In the definition of our fourth-order diffusion tensor D, the choice of μ<sup>1</sup> and μ<sup>2</sup> is analogous to anisotropic edge enhancing diffusion [34]. However, two additional terms occur in the fourth-order case, μ<sup>3</sup> and μ4. As noted in [15], μ<sup>4</sup> is irrelevant, since the corresponding eigentensor **E**<sup>4</sup> is anti-symmetric, and the dot product **E**<sup>4</sup> : **H** with the Hessian of any sufficiently smooth image will be zero due to its symmetry. To better understand the role of μ3, we observe that

(f) Fourth-order EED inpainting with *µ*<sup>3</sup> = √*µ*1*µ*<sup>2</sup>

$$\begin{split} \mathbf{E}\_3 : \mathbf{H} &= \frac{1}{\sqrt{2}} \left( \mathbf{v}\_1^T \mathbf{H} \mathbf{v}\_2 + \mathbf{v}\_2^T \mathbf{H} \mathbf{v}\_1 \right) \\ &= \frac{1}{\sqrt{2}} \left( u \begin{pmatrix} \frac{\mathbf{v}\_1 + \mathbf{v}\_2}{\sqrt{2}} \end{pmatrix} \begin{pmatrix} \frac{\mathbf{v}\_1 - \mathbf{v}\_2}{\sqrt{2}} \end{pmatrix} \right), \end{split} \tag{14}$$

which amounts to a mixed second derivative of *u*, in directions along and orthogonal to the regularized image gradient <sup>∇</sup>*u*<sup>σ</sup> ||∇*u*<sup>σ</sup> ||<sup>2</sup> or, equivalently, to the difference of second derivatives in the directions that are exactly in between the two. This term vanishes if the Hessian is isotropic, or if the gradient is parallel to one of the Hessian eigenvectors. Therefore, the role of μ<sup>3</sup> can be seen as steering the amount of diffusion in cases of a Hessian anisotropy that goes along with a misalignment between gradient and Hessian eigenvectors.

Gorgi Zadeh et al. [15] simply set μ<sup>3</sup> to the arithmetic mean of μ<sup>1</sup> and μ2. In our work, we empirically evaluated several alternative options for μ<sup>3</sup> by reconstructing the test image shown in Fig. 1a, which contains one rectangle, one circle, and two stars, from a randomly selected subset of 5% of its pixels. In this experiment, we compare EED based inpainting with our novel fourth-order edge enhancing diffusion (FOEED) with different settings of μ3: Specifically, μ<sup>3</sup> = 1 corresponds to the max-

(e) Fourth-order EED inpainting with *µ*<sup>3</sup> = *<sup>µ</sup>* 1+*<sup>µ</sup>* <sup>2</sup>

2

**Fig. 1** Reconstruction of a synthetic test image (**a**) from 5% of its pixels (**b**) based on second-order diffusion (**c**) and fourth-order diffusion with different coefficients for the mixed term μ<sup>3</sup> (**d**–**f**). Visually, the reconstruction in (**f**) is most similar to the original image

(d) Fourth-order EED inpaint-

ing with *µ*<sup>3</sup> = 1


**Table 1** Numerical reconstruction errors on the test image (Fig. 1)

imum of μ<sup>1</sup> and μ<sup>2</sup> (Fig. 1d), μ<sup>3</sup> = (1 + *g*(λ1)) /2 corresponds to their arithmetic mean (Fig. 1e), and <sup>μ</sup><sup>3</sup> <sup>=</sup> <sup>√</sup>*g*(λ1) corresponds to their geometric mean (Fig. 1f). In all cases, we used the Charbonnier diffusivity (Eq. (6)), which is popular for image compression [13], the same contrast parameter (λ = 0.1) and smoothing parameter (σ = 1). The only difference is time step size, where second-order EED permitted a stable step size of 0.25, while a smaller step size of 0.05 was chosen for FOEED. A more detailed theoretical and empirical analysis of stability will be given in Sect. 3.3.

A numerical comparison of the results is given in Table 1. For evaluation, we used the well-known mean squared error (MSE) and average absolute error (AAE) between original and reconstructed images. For two-dimensional gray-valued images *u* and *v* with the same dimensions *m* × *n*, the MSE and AAE are defined as

$$\begin{aligned} \text{MSE}(\boldsymbol{\mu}, \boldsymbol{\nu}) &= \frac{1}{mn} \sum\_{i,j} (\boldsymbol{u}\_{i,j} - \boldsymbol{\nu}\_{i,j})^2 \,, \\ \text{AAE}(\boldsymbol{\mu}, \boldsymbol{\nu}) &= \frac{1}{mn} \sum\_{i,j} |\boldsymbol{u}\_{i,j} - \boldsymbol{\nu}\_{i,j}| \, . \end{aligned} \tag{15}$$

According to Table 1, the most accurate results are achieved by setting μ<sup>3</sup> to the geometric mean of μ<sup>1</sup> and μ2. Fourth-order EED with this setting produces higher accuracy than second-order EED. Visually, Fig. 1 supports this conclusion. Specifically, fourth-order EED with <sup>μ</sup><sup>3</sup> <sup>=</sup> <sup>√</sup>μ1μ<sup>2</sup> is the only variant that correctly connects the thin bar at the top of the test image, and it leads to a straighter shape of the thicker bar below, which is more similar to its original rectangular shape. In all subsequent experiments, we set <sup>μ</sup><sup>3</sup> <sup>=</sup> <sup>√</sup>μ1μ2.

#### *3.2 A Unifying Framework for Fourth-Order Diffusion*

Several fourth-order diffusion PDEs have been used for image processing previously. We can better understand how they relate to our newly proposed PDE by observing that the fourth-order diffusion tensor D introduces a unifying framework for fourthorder diffusion filters. In particular, given its coefficients *dijkl* , we can expand Eq. (9) by using Einstein notation as

$$
\partial\_t u = -\partial\_{ji} [d\_{ijkl} u\_{kl}] \tag{16}
$$

Effectively, the fourth-order diffusion tensor allows us to separately set the diffusivities for all 2<sup>4</sup> = 16 fourth-order derivatives of the two-dimensional image *u*. We will now demonstrate how several well-known fourth-order PDEs can be expressed in this framework, starting with the You-Kaveh PDE [35]

$$
\partial\_t \mu = -\Delta(\mathbf{g}(|\Delta \mu|) \Delta \mu)\,, \tag{17}
$$

which can be rewritten as

$$\begin{split} \partial\_{l} \boldsymbol{u} = & -\partial\_{\rm xx} \left[ \boldsymbol{g} \left( |\Delta \boldsymbol{u}| \right) \boldsymbol{u}\_{xx} + \boldsymbol{0} \cdot \boldsymbol{u}\_{xy} + \boldsymbol{0} \cdot \boldsymbol{u}\_{yx} + \boldsymbol{g} \left( |\Delta \boldsymbol{u}| \right) \boldsymbol{u}\_{yy} \right] \\ & - \partial\_{\rm yx} \left[ \boldsymbol{0} \cdot \boldsymbol{u}\_{xx} + \boldsymbol{0} \cdot \boldsymbol{u}\_{xy} + \boldsymbol{0} \cdot \boldsymbol{u}\_{yx} + \boldsymbol{0} \cdot \boldsymbol{u}\_{yy} \right] \\ & - \partial\_{\rm xy} \left[ \boldsymbol{0} \cdot \boldsymbol{u}\_{xx} + \boldsymbol{0} \cdot \boldsymbol{u}\_{xy} + \boldsymbol{0} \cdot \boldsymbol{u}\_{yx} + \boldsymbol{0} \cdot \boldsymbol{u}\_{yy} \right] \\ & - \partial\_{\rm yy} \left[ \boldsymbol{g} \left( |\Delta \boldsymbol{u}| \right) \boldsymbol{u}\_{xx} + \boldsymbol{0} \cdot \boldsymbol{u}\_{xy} + \boldsymbol{0} \cdot \boldsymbol{u}\_{yx} + \boldsymbol{g} \left( |\Delta \boldsymbol{u}| \right) \boldsymbol{u}\_{yy} \right]. \end{split} \tag{18}$$

Here and in all subsequent examples, many terms have zero coefficients. For brevity, we will omit them from now on.

Hajiaboli's anisotropic fourth-order PDE [19] is

$$\partial\_t \mu = -\Delta \left( \mathbf{g} (||\nabla \mu||) ^2 \boldsymbol{\mu}\_{NN} + \mathbf{g} (||\nabla \mu||) \boldsymbol{\mu}\_{TT} \right), \tag{19}$$

where *N* and *T* are unit vectors parallel and orthogonal to the gradient, respectively. It can be rewritten as

$$\begin{split} \partial\_{l}u = &-\partial\_{xx}\Big[\Big(\frac{g(||\nabla u||)^{2}u\_{x}^{2}+g(||\nabla u||)u\_{y}^{2}}{u\_{x}^{2}+u\_{y}^{2}}\Big)u\_{xx} + \bigg(\frac{g(||\nabla u||)^{2}u\_{x}u\_{y}-g(||\nabla u||)u\_{x}u\_{y}}{u\_{x}^{2}+u\_{y}^{2}}\Big)u\_{xy} \\ &+\Big(\frac{g(||\nabla u||)^{2}u\_{x}u\_{y}-g(||\nabla u||)u\_{x}u\_{y}}{u\_{x}^{2}+u\_{y}^{2}}\Big)u\_{yx} + \bigg(\frac{g(||\nabla u||)^{2}u\_{y}^{2}+g(||\nabla u||)u\_{x}^{2}}{u\_{x}^{2}+u\_{y}^{2}}\Big)u\_{yy} \\ &-\partial\_{yy}\Big[\Big(\frac{g(||\nabla u||)^{2}u\_{x}^{2}+g(||\nabla u||)u\_{y}^{2}}{u\_{x}^{2}+u\_{y}^{2}}\Big)u\_{xx} + \bigg(\frac{g(||\nabla u||)^{2}u\_{x}u\_{y}-g(||\nabla u||)u\_{x}u\_{y}}{u\_{x}^{2}+u\_{y}^{2}}\Big)u\_{xy} \\ &+\Big(\frac{g(||\nabla u||)^{2}u\_{x}u\_{y}-g(||\nabla u||)u\_{x}u\_{y}}{u\_{x}^{2}+u\_{y}^{2}}\Big)u\_{yy} + \bigg(\frac{g(||\nabla u||)^{2}u\_{y}^{2}+g(||\nabla u||)u\_{x}^{2}}{u\_{x}^{2}+u\_{y}^{2}}\Big)u\_{yy} \end{split} \tag{20}$$

From this method, Li et al. [21] derived two anisotropic fourth-order PDEs that, to our knowledge, are the only anisotropic fourth-order models that have been applied to inpainting previously. We will refer to them as Li 1

$$
\partial\_t \mu = -\Delta (\mathbf{g}(||\nabla \mu||) \mu\_{NN} + \mu\_{TT}) \tag{21}
$$

and Li 2

$$
\partial\_t \mu = -\Delta(\mu\_{TT})\,. \tag{22}
$$

#### Li 1 can be re-written as

$$\begin{split} \partial\_t u &= -\partial\_{xx} \Big[ \left( \frac{g(||\nabla u||)u\_x^2 + u\_y^2}{u\_x^2 + u\_y^2} \right) u\_{xx} + \left( \frac{g(||\nabla u||)u\_x u\_y - u\_x u\_y}{u\_x^2 + u\_y^2} \right) u\_{xy} \\ &+ \left( \frac{g(||\nabla u||)u\_x u\_y - u\_x u\_y}{u\_x^2 + u\_y^2} \right) u\_{yx} + \left( \frac{g(||\nabla u||)u\_y^2 + u\_x^2}{u\_x^2 + u\_y^2} \right) u\_{yy} \\ &- \partial\_{yy} \Big[ \left( \frac{g(||\nabla u||)u\_x^2 + u\_y^2}{u\_x^2 + u\_y^2} \right) u\_{xx} + \left( \frac{g(||\nabla u||)u\_x u\_y - u\_x u\_y}{u\_x^2 + u\_y^2} \right) u\_{xy} \\ &+ \left( \frac{g(||\nabla u||)u\_x u\_y - u\_x u\_y}{u\_x^2 + u\_y^2} \right) u\_{yx} + \left( \frac{g(||\nabla u||)u\_y^2 + u\_x^2}{u\_x^2 + u\_y^2} \right) u\_{yy} \Big], \end{split} \tag{23}$$

while Li 2 becomes

$$\begin{split} \partial\_t u &= -\partial\_{\text{xx}} \left[ \left( \frac{u\_\text{y}^2}{u\_\text{x}^2 + u\_\text{y}^2} \right) u\_{\text{xx}} + \left( \frac{-u\_\text{x} u\_\text{y}}{u\_\text{x}^2 + u\_\text{y}^2} \right) u\_{\text{xy}} + \left( \frac{-u\_\text{x} u\_\text{y}}{u\_\text{x}^2 + u\_\text{y}^2} \right) u\_{\text{yx}} + \left( \frac{u\_\text{x}^2}{u\_\text{x}^2 + u\_\text{y}^2} \right) u\_{\text{yy}} \right] \\ &- \partial\_{\text{yy}} \left[ \left( \frac{u\_\text{y}^2}{u\_\text{x}^2 + u\_\text{y}^2} \right) u\_{\text{xx}} + \left( \frac{-u\_\text{x} u\_\text{y}}{u\_\text{x}^2 + u\_\text{y}^2} \right) u\_{\text{xy}} + \left( \frac{-u\_\text{x} u\_\text{y}}{u\_\text{x}^2 + u\_\text{y}^2} \right) u\_{\text{yx}} + \left( \frac{u\_\text{x}^2}{u\_\text{x}^2 + u\_\text{y}^2} \right) u\_{\text{yy}} \right]. \end{split} \tag{24}$$

We observe that Li 1 is based on a similar idea as our proposed PDE: It permits fourth-order diffusion along the edge, while applying a nonlinear diffusivity function across the edge. However, expressing Li et al.'s models in terms of fourth-order diffusion tensors D<sup>1</sup> and D<sup>2</sup> reveals that our approach is more general. In particular, we can observe that

$$\begin{aligned} \mathcal{D}\_1: \mathbf{H} &= \operatorname{g} (||\nabla u||) \mu\_{NN} \mathbf{I} + \mu\_{TT} \mathbf{I} \ , \\ \mathcal{D}\_2: \mathbf{H} &= u\_{TT} \mathbf{I} \ , \end{aligned} \tag{25}$$

where **I** is the 2 × 2 identity matrix. In our model, D : **H** can yield arbitrary anisotropic tensors. In this sense, our model more fully accounts for anisotropy compared to the ones by Hajiaboli and Li et al.

The fourth-order Eq. (16) involves inner second derivatives of the image, which then get scaled by diffusivities, before outer second derivatives are taken. We observe that, in both cases, our model accounts for mixed derivatives that are ignored by previous approaches to anisotropic fourth-order diffusion: In the outer derivatives, this can be seen from the fact that Eq. (9) involves mixed derivatives, while Hajiaboli and Li et al. only consider the Laplacian.

Similarly, our definition of a fourth-order diffusion tensor D accounts for mixed derivatives also in the inner derivatives. Following Eq. (14), we obtain

$$\begin{split} \mathcal{D}: \mathbf{H} &= \mu\_1 (\mathbf{E}\_1 \otimes \mathbf{E}\_1): \mathbf{H} + \mu\_2 (\mathbf{E}\_2 \otimes \mathbf{E}\_2): \mathbf{H} + \mu\_3 (\mathbf{E}\_3 \otimes \mathbf{E}\_3): \mathbf{H} \\ &= \mu\_1 u\_{\mathbf{v}\_1 \mathbf{v}\_1} \mathbf{E}\_1 + \mu\_2 u\_{\mathbf{v}\_2 \mathbf{v}\_2} \mathbf{E}\_2 + \frac{\mu\_3}{\sqrt{2}} \left( u\_{\left( \frac{\mathbf{v}\_1 + \mathbf{v}\_2}{\sqrt{2}} \right) \left( \frac{\mathbf{v}\_1 + \mathbf{v}\_2}{\sqrt{2}} \right)} - u\_{\left( \frac{\mathbf{v}\_1 - \mathbf{v}\_2}{\sqrt{2}} \right) \left( \frac{\mathbf{v}\_1 - \mathbf{v}\_2}{\sqrt{2}} \right)} \right) \mathbf{E}\_3 \, . \end{split} \tag{26}$$

Comparing Eqs. (25) and (26) reveals differences in the considered directions: First, *N* and *T* are derived from the unregularized gradient, while the corresponding directions **v**<sup>1</sup> and **v**<sup>2</sup> in our model include a Gaussian pre-smoothing. A second difference is that our model involves an additional term, which is steered by μ3, and accounts for the directions in between the regularized gradient and its orthogonal vectors, i.e., **<sup>v</sup>**<sup>1</sup> √ +**v**<sup>2</sup> 2 and **<sup>v</sup>**<sup>1</sup> √ −**v**<sup>2</sup> 2 . As it was demonstrated in the previous section, this term can have a noticeable effect on the outcome. Overall, we conclude that our newly proposed model is more general than the previously published ones.

#### *3.3 Discretization and Stability*

When discretizing Eq. (13) with standard finite differences

$$\begin{split} \mu\_{xx} &\approx \frac{(u\_{i-1,j} - 2u\_{i,j} + u\_{i+1,j})}{(\Delta x)^2}, \\ \mu\_{yy} &\approx \frac{(u\_{i,j-1} - 2u\_{i,j} + u\_{i,j+1})}{(\Delta y)^2}, \\ \mu\_{xy} &\approx \frac{(u\_{i-1,j-1} + u\_{i+1,j+1} + u\_{i-1,j+1} + u\_{i+1,j-1})}{4(\Delta x)(\Delta y)}, \\ \mu\_{yx} &= u\_{xy} \end{split} \tag{27}$$

we can write it down in matrix-vector form as in [15],

$$\mathbf{u}^{k+1} = \mathbf{u}^k (\mathbf{I} - \tau \mathbf{P}\_k) \,, \tag{28}$$

where *u<sup>k</sup>* is an *mn* dimensional image vector at iteration *k*. *m*, *n* are image width and height respectively; *x* and *y* are the corresponding pixel edge lengths. **P***<sup>k</sup>* is a positive semi-definite matrix that, with step size τ , leads to the system matrix (**I** − τ **P***<sup>k</sup>* ). The notation **P***<sup>k</sup>* indicates that it is iteration dependent, i.e., **P***<sup>k</sup>* = **P**(**u***<sup>k</sup>* ).

Stability of fourth-order PDEs for image processing is typically formalized in an *L*<sup>2</sup> sense, i.e., a time step is chosen such that

$$||\mathbf{u}^{k+1}||\_2 \le ||\mathbf{u}^k||\_2 \,. \tag{29}$$

In an inpainting scenario, it depends on our initialization of the unknown pixels whether we can expect Eq. (29) to hold. Therefore, we rely on a stability analysis of the smoothing variant of our proposed PDE. This variant is obtained by removing the Dirichlet boundary conditions and instead solving a standard initial value problem. In this case, the stability analysis presented by Gorgi Zadeh et al. [15] carries over. It ensures that time step sizes

$$
\pi \le \frac{2}{16(\Delta x)^2 + 16(\Delta y)^2 + 2(\Delta x \Delta y)}\tag{30}
$$

are stable in the *L*<sup>2</sup> sense. For a spatial discretization *x* = *y* = 1, this yields τ ≤ 1/17 ≈ 0.0588. In inpainting, we empirically obtained a useful steady state with a time step size τ ≤ 0.066, independent of the initialization. The fact that this slightly exceeds the theoretical step size reflects the fact that Eq. (30) results from deriving a sufficient, not a necessary condition for stability.

Stability of fourth-order schemes generally requires a quite small time step τ . This makes it computationally expensive to reach the steady state by evaluating Eq. (28). Hafner et al. [17] propose a remedy to this problem, the so-called Fast Semi-Iterative Scheme (FSI). It extrapolates the basic solver iteration with the previous iterate and serves as an accelerated explicit scheme. The acceleration of the explicit scheme (28) is given as

$$\mathbf{u}^{m,k+1} = \alpha\_k \cdot (\mathbf{I} - \tau \mathbf{P}(\mathbf{u}^{m,k})) \mathbf{u}^{m,k} + (1 - \alpha\_k) \cdot \mathbf{u}^{m,k-1},\tag{31}$$

where **<sup>u</sup>***m*,−<sup>1</sup> := **<sup>u</sup>***m*,<sup>0</sup> and <sup>α</sup>*<sup>k</sup>* <sup>=</sup> <sup>4</sup>*k*+<sup>2</sup> <sup>2</sup>*k*+<sup>3</sup> for *<sup>k</sup>* <sup>=</sup> <sup>0</sup>,..., *<sup>n</sup>* <sup>−</sup> 1. Here *<sup>m</sup>* stands for outer cycle, i.e. *m*-th cycle with inner cycle of length *n*. And for passing to the next outer cycle, we set **u***m*+1,<sup>0</sup> := **u***m*,*<sup>n</sup>*. The stability analysis requires the matrix **P** to be symmetric. This is satisfied since the diffusion tensor D is symmetric, and symmetric central discretizations are used. In our implementation, we used *n* = 40, and stopped iterating after the first outer cycle for which **u***<sup>m</sup>* − **u***m*−<sup>1</sup><sup>2</sup> < 10−4.

#### **4 Experimental Results**

To establish the usefulness of our proposed new model, we applied it to the reconstruction of images from a sparse subset of pixels (Sect. 4.1). Moreover, we evaluate performance for a more classic inpainting task, scratch removal (Sect. 4.2). We also demonstrate how results depend on the chosen diffusivity function and contrast parameter (Sect. 4.3).

#### *4.1 Reconstruction From a Sparse Set of Pixels*

Improving image reconstruction from a sparse set of known pixels was the main motivation behind our work. Therefore, we applied it to two well-known natural images, *toucan* and *peppers,* as well as to a medical image, a slice of a *T*<sup>1</sup> weighted brain MR scan *(t1slice).* For *toucan,* we kept a random subset of only 2% of the pixels. Due to the lower resolution of the *peppers* and *t1slice* images, we kept 5% and 20%, respectively.

In all three cases, results from our approach (FOEED) were compared to results from second-order EED, as well as from the two anisotropic fourth-order PDEs proposed by Li et al. [21]. In all experiments, we used the Charbonnier diffusivity function, we set the contrast parameter to λ = 0.1, and the pre-smoothing parameter to σ = 1.

Results for*toucan* are shown in Fig. 2, for *peppers*in Fig. 3, and for*t1slice* in Fig. 4. A quantitative evaluation in terms ofMSE and AAE is presented in Table 2. In terms of the numerical results, our proposed method produced a more accurate reconstruction than any of the competing approaches. Visually, there is a clear difference between second-order (EED) and fourth-order approaches (Li1, Li2, FOEED). Especially, we found that the shapes of edges were reconstructed more accurately. For example, we noticed this around the body and face in the *toucan* image (Fig. 2). Similarly, the white and grey matter boundaries were better separated in the *t1slice* (Fig. 4).

As we expected based on the theoretical analysis in Sect. 3.2, visual differences between the fourth-order methods are more subtle. However, in the *peppers* image (Fig. 3), the tall and thin and the small and thick peppers in the foreground are much more clearly separated in the FOEED result than in any of the others.

In addition to experimenting with grayscale versions of the *toucan* and *peppers* images, we also applied EED and our FOEED filter channel-wise to the original RGB color versions. Results for *toucan* can be found in Fig. 5, for *peppers* in Fig. 6. Table 3 again provides a quantitative comparison. Similar observations can be made as in the grayscale images: Again, FOEED leads to lower reconstruction errors than EED, it visually reconstructs edges more accurately, and separates the peppers more clearly.

Finally, we reconstructed images from a larger number of pixels, to obtain visually cleaner results. Qualitative and numerical results are presented in Fig. 7 and Table 4, respectively. FOEED still yields lower numerical errors than EED. Unsurprisingly, the differences become smaller and less visually prominent as the mask density increases. The table also reveals that FOEED requires more CPU time compared to standard EED. However, due to the use of FSI in both cases, the difference in running times until convergence is much lower than the difference in time step sizes.

**Fig. 2** 1st row left: original *toucan* image of size 512 × 512; right: randomly chosen 2% of pixel values; 2nd row left: EED based inpainted image; right: Li1 based inpainted image; 3rd row left: Li2 based inpainted image; right: FOEED based inpainted image

**Fig. 3** 1st row left: original *peppers* image of size 225 × 225; Right: randomly chosen 5% of pixel values; 2nd row left: EED based inpainted image; Right: Li1 based inpainted image; 3rd row left: Li2 based inpainted image; Right: FOEED based inpainted image

**Fig. 4** 1st row left: original *t1slice* image of size 256 × 256; Right: randomly chosen 20% of pixel values; 2nd row left: EED based inpainted image; Right: Li1 based inpainted image; 3rd row left: Li2 based inpainted image; Right: FOEED based inpainted image


**Table 2** Numerical comparison of inpainting models for gray-valued images

**Fig. 5** RGB *toucan* image, reconstructed from randomly chosen 2% of pixel values using EED (left) or FOEED (right)

**Fig. 6** RGB *peppers* image, reconstructed from randomly chosen 5% of pixel values using EED (left) or FOEED (right)


**Table 3** Numerical comparison of inpainting models for RGB images

**Table 4** Numerical comparison and computation times corresponding to Fig. 7


#### *4.2 Scratch Removal*

Li et al. [21] proposed their anisotropic fourth-order PDE for more classical image inpainting tasks, such as scratch removal. We evaluated whether our more general filter can also provide a benefit in such a scenario by reconstructing a scratched version of the *peppers* image. Similar to Li et al., we first made the scratches rather thin, covering only 6% of all pixels. Results are shown in Fig. 8 and in Table 5. In this case, all methods work well: Numerical errors are small and similar between methods, and even though FOEED achieves the best numerical result, differences are difficult to discern visually.

Therefore, we created a more challenging version with thicker scratches, covering 18% of all pixels (Fig. 9). The corresponding numerical comparison is shown in Table 6. Here, FOEED achieves the most accurate reconstruction. Visually, we again observe that edges are reconstructed more accurately, and objects are more clearly separated, with fourth-order compared to second-order diffusion, and that steering it with a fourth-order diffusion tensor again provides small additional benefits over the previous methods.


**Table 5** Numerical comparison for peppers with thinner scratches (Fig. 8)

**Fig. 7** Higher quality reconstructions from a larger subset of pixels. 1st row: *toucan* image, reconstructed with EED (left) or FOEED (right) from randomly chosen 14% of pixels; 2nd row left: same for 20% of pixels from *peppers*; 3rd row left: same for 30% of pixels from *t1slice*. As expected, increasing the fraction of known pixels reduces the differences in the results of the two schemes

**Fig. 8** 1st row left: original *peppers* image of size 225×225; Right: corrupted image. 2nd row left: EED based inpainting; Right: Li1 based inpainting. 3rd row left: Li2 based inpainting; Right: FOEED based inpainting

**Fig. 9** 1st row left: original *peppers* image of size 225 × 225; Right: corrupted image. 2nd row left: EED based inpainting; Right: Li1 based inpainting. 3rd row left: Li2 based inpainting; Right: FOEED based inpainting


**Table 6** Numerical comparison for peppers with thicker scratches (Fig. 9)

#### *4.3 Effect of Diffusivity Function and Contrast Parameter*

For image inpainting with second-order PDEs, the Charbonnier diffusivity was previously found to work better than other established diffusivity functions. To assess whether this is still true in the fourth-order case, we repeated the reconstruction of the *peppers* image as shown in Fig. 3 with different diffusivities. Table 7 summarizes the results. We conclude that the Charbonnier diffusivity still appears to be optimal.

Finally, in Fig. 10, we illustrate how the reconstructed image depends on the contrast parameter λ. As expected, increasing λ leads to an increased blurring of edges. In the limit, the diffusivity function takes on values close to 1 over a substantial part of the image, and our model starts to approximate homogeneous fourth-order diffusion.


**Table 7** Numerical comparison of FOEED with different diffusivity functions

**Fig. 10** From left to right: FOEED based inpainted image with λ = 0.1, λ = 0.5, λ = 15.5

#### **5 Conclusions**

We introduced a novel fourth-order PDE for edge enhancing diffusion (FOEED), steered by a fourth-order diffusion tensor. We implemented it using a fast semiiterative scheme, and demonstrated that it achieved improved accuracy in several inpainting tasks, including reconstructing images from a small fraction of pixels, or removing scratches.

Our main motivation for using fourth-order diffusion in this context is the increased smoothness of results compared to second-order PDEs [35], which we expected to result in visually more pleasant reconstructions. The model in our current work is still based on a single edge direction at each pixel, extracted via a traditional second-order structure tensor. It is left as a separate research goal for future work to combine this with approaches for the estimation of complex structures such as crossings or bifurcations [1, 29], and with their improved reconstruction, e.g., by operating on the space of positions and orientations [4, 9, 11].

Finally, our current work only considered reconstructions from a random subset of pixels. A practical image compression codec that uses our novel PDE should investigate how it interacts with more sophisticated approaches for selecting and coding inpainting masks [27].

**Acknowledgements** This research was supported by the German Academic Exchange Service (DAAD).

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Uncertainty in the DTI Visualization Pipeline**

**Faizan Siddiqui, Thomas Höllt, and Anna Vilanova**

**Abstract** Diffusion-Weighted Magnetic Resonance Imaging (DWI) enables the invivo visualization of fibrous tissues such as white matter in the brain. Diffusion-Tensor Imaging (DTI) specifically models the DWI diffusion measurements as a second order-tensor. The processing pipeline to visualize this data, from image acquisition to the final rendering, is rather complex. It involves a considerable amount of measurements, parameters and model assumptions, all of which generate uncertainties in the final result which typically are not shown to the analyst in the visualization. In recent years, there has been a considerable amount of work on the visualization of uncertainty in DWI, and specifically DTI. In this chapter, we primarily focus on DTI given its simplicity and applicability, however, several aspects presented are valid for DWI as a whole. We explore the various sources of uncertainties involved, approaches for modeling those uncertainties, and, finally, we survey different strategies to visually represent them. We also look at several related methods of uncertainty visualization that have been applied outside DTI and discuss how these techniques can be adopted to the DTI domain. We conclude our discussion with an overview of potential research directions.

T. Höllt e-mail: T.Hollt-1@tudelft.nl

A. Vilanova e-mail: A.Vilanova@tue.nl

T. Höllt Leiden University Medical Center, Leiden, The Netherlands

A. Vilanova Eindhoven University of Technology, Eindhoven, The Netherlands

F. Siddiqui (B) · T. Höllt · A. Vilanova Delft University of Technology, Delft, The Netherlands e-mail: F.P.Siddiqui@tudelft.nl

#### **1 Introduction**

Recent advancements in magnetic resonance imaging (MRI) technology have led to the development of various remarkable techniques for the interpretation of brain anatomy. The most promising one is diffusion-weighted imaging (DWI), the only non-invasive technique for the assessment of brain white matter connectivity. This approach relies on the measurement of anisotropic diffusion of water molecules. The imaging and the interpretation of the molecular diffusion have improved with the development of techniques like diffusion tensor imaging (DTI) and high angular resolution diffusion imaging (HARDI). In this chapter, we will discuss the visualization pipeline of DTI, given its clinical applicability. However, several visualization strategies and sources of uncertainties associated are valid for more advanced models like HARDI models.

DTI allows direct in-vivo examination of the fibrous structure in the brain at a relatively low acquisition cost. By analyzing the three-dimensional shape of the diffusion tensor, it provides valuable information about the microstructure of brain tissues. Despite many advantages of this technique, some downsides limit its widespread use. The main reason is that the complexity in the data makes it notoriously difficult to infer and analyze.

The DTI visualization pipeline consists of four main stages, from data acquisition to the final visual representation of the results, as shown in Fig. 1. Each stage is based on assumptions, parameters, and estimations subject to considerable uncertainties. The uncertainties involved in each of the pipelines' stage can lead to unpredictable variations in the final output.

Several state-of-the-art reports exist on DWI visualization [62, 92, 93, 99]. However, none of them give an overview of uncertainty, or they focus on some specific aspects. Most of the visualization literature about uncertainty in DTI focuses on issues related to the visual representation rather than sources of error involved in the pipeline [36, 47, 92]. In this chapter, we discuss the DTI visualization pipeline and analyze the sources of uncertainties present at each stage. We briefly cover the approaches used for quantification of uncertainties, which are often omitted in other studies [36, 47]. We review state-of-the-art strategies for uncertainty visualization in DTI and compare their main characteristics and drawbacks. We further investigate several methodologies for uncertainty visualization in other domains that have not been explored in DWI and discuss how these techniques can be adopted in this domain. DWI models beyond DTI share a similar pipeline as the one shown in Fig. 1. However, some parameters, error sources, and visual representations would differ from the tensor model. Specifically, diffusion modeling and fiber tracking would be based on different parameters and algorithms. In this chapter, we will indicate the methods from the DTI that are valid for the more general DWI pipeline.

In Sect. 2, we discuss the background and review the visualization techniques for DTI. In Sect. 3, we discuss the sources of uncertainties involved in the visualization

**Fig. 1** The DTI visualization pipeline with sources of uncertainties at each step

pipeline and proceed with the uncertainty modeling techniques in Sect. 4. We review uncertainty visualization methods in Sect. 5 and conclude with open issues and research direction in Sect. 6.

#### **2 Background**

Diffusion refers to the constant rapid movement of microscopic particles due to the presence of thermal energy, i.e., 'Brownian motion'. DWI deals with the diffusion of water molecules present in biological tissues where the diffusion is usually restricted due to the hindrance by many obstacles such as axonal membranes, macromolecules, and myelin. This kind of restricted diffusion is known as anisotropic diffusion. Stejskal and Tanner [94] observed the anisotropic diffusion of water molecules in tissues and investigated the related modeling of the diffusion effects for MRI. The clinical application of this technique was first presented by Bihan et al. [61] with the introduction of diffusion MRI along with the concept of apparent diffusion coefficient (ADC). In some neurological conditions, the amount of diffusion is disturbed in the affected area. Through studying these changes in diffusion, the abnormalities can be detected. In the following section, we will summarize how these measurements have been used to visualize white matter tracts in the brain.

#### *2.1 Diffusion Tensor*

The pattern of diffusion anisotropy of white matter tracts in 3D space can be mathematically modeled by a second order tensor, called the diffusion tensor *D*, introduced by Basser et al. [5]. The tensor *D* is a symmetric, positive definite tensor represented by a 3 × 3 matrix with six unique elements, denoted by *Di j* as follows:

$$D = \begin{pmatrix} D\_{11} \ D\_{12} \ D\_{13} \\ D\_{21} \ D\_{22} \ D\_{23} \\ D\_{31} \ D\_{32} \ D\_{33} \end{pmatrix} \tag{1}$$

The diagonal elements in the diffusion tensor *D* represent the diffusion coefficients along the principle axes x, y and z, while the off-diagonal elements represent the correlation of the diffusion between each pair of the principal direction. The diffusion tensor *D* is symmetric about the diagonal axis (*Di j* = *Dji*). By analysis of the eigenvectors and eigenvalues λ1,λ2,λ<sup>3</sup> of the diffusion matrix, the length and the direction of the principal axes of the diffusion tensor can be determined.

The six unique values in the tensor *D* provide the intervoxel diffusion information and the microstructure of a particular voxel. However, the six-dimensional diffusion tensor is hard to infer and present to a user. For this reason, several scalar quantities have been introduced to simplify the tensor to a single value. Fractional Anisotropy (FA), the most widely used scalar measure in diffusion tensor imaging [9] , represents the extent of the diffusion anisotropicity. A lower value of FA indicates that the diffusion is free (FA = 0; isotropic), while a high value of FA implies that the diffusion is restricted to a single direction (FA = 1; anisotropic). Another popular scalar measure in DTI is mean diffusivity (MD), which represents the overall amount of diffusion. Many other scalar measures have been proposed based on the more complex behaviour of molecular diffusion and are explained in detail in surveys by Novikov et al. [75], Rajagopalan et al. [83] and Vilanova et al. [99].

**Visualization strategies**: A Glyph is a general term for geometrically plotted specifier that represent multidimensional data values. Data information is mapped to glyph characteristics such as shape and color. Glyphs provide a way to represent the full six-dimensional information of a diffusion tensor by mapping the eigenvectors and eigenvalues to the orientation and shape of a geometric primitive. The most straight forward approach to visualize the diffusion tensor are ellipsoidal glyphs [82] as shown in Fig. 2a. The orientation of the ellipsoid represents the direction of the major eigenvector, while the length represents the corresponding eigenvalue. Westin et al. [104] introduced three metrics to measure linear (λ<sup>1</sup> *>* λ2,λ3), planar (λ1=λ<sup>2</sup> *>* λ3) and spherical diffusion (λ<sup>1</sup> = λ<sup>2</sup> = λ3). Figure. 2 represents the barycentric space of diffusion tensor shapes in which the three extremes (linear, planar, and spherical) are at the corner of triangles. Among several other proposed techniques, the super quadratic glyph is considered state-of-the-art for glyph-based tensor visualization. Instead of interpolating between ellipsoidal shapes, Kindlmann [57] represents the diffusion by superquadrics with shape parameters defined by the barycentric coordinates,as shown in Fig. 2b.

(a) Ellipsoid glyphs (b) Superquadratic glyphs

**Fig. 2** Barycentric space of diffusion tensor shapes

Outside DTI, glyphs are also used to represent the orientation distribution function (ODF) of the molecular diffusion, which is commonly estimated by models that go beyond the diffusion tensor. ODF specifies the overall diffusion in a given direction, integrated over displacement magnitudes [103]. Spherical polar plots [98] parametrized surfaces [77], the HARDI glyph [81] and the HOME glyph [89] are some of the common glyph based visualization techniques for representing ODFs.

#### *2.2 Fiber Tracking*

The diffusion tensor provides per-voxel information about the orientation of the underlying neural tracts by analyzing the derived eigenvectors. By combining this information with other scalar measure, e.g. FA, one can estimate trajectories of white matter bundles in 3D space. The process of virtual reconstruction of the neural fiber tract on the basis of the diffusion tensor field is named Fiber Tracking or Tractography [8, 71]. These techniques of generating brain anatomical connectivity from the diffusion information have been summarized in review articles [42, 72, 102]. Fiber

(c) Cylindrical tubes (d) Streamtubes

(a) Polylines (b) Illuminated streamlines

**Fig. 3** Visualization techniques for deterministic tractography. Images are generated using vIST/e [1]

tracking methods have found its way in many neurological applications [18, 31, 45, 73, 74].

The fiber tracking strategies can be mainly divided into deterministic, probabilistic, and global geometric techniques. Deterministic techniques always produce the same output with the same set of inputs. Probabilistic techniques, however, add randomness in the tracking process to incorporate the inherent uncertainty. We defer an extensive discussion of probabilistic methods to Sect. 4. Global geometric methods deduce connectivity in the white matter by globally optimizing a certain cost function based on the diffusion tensor information and are outside of the scope of this chapter.

Streamline tracing is the most commonly used algorithm for tractography. It is a deterministic technique that generates trajectories by integrating the vector field defined by the main eigenvector at each voxel position. The tracing process ends when the stopping criteria are met. Several constraints can be used as stopping criteria, such as maximum turning angle or FA, to limit tracts to the region where tensors realistically represent the fiber tracts.

**Visualization strategies**: Line-based approaches are the most straightforward technique to represent deterministic fiber tracts. Numerous strategies have been introduced for the visualization of the white matter tracts, such as thin polylines [70] illuminated streamlines [114] or cylindrical tubes [8]. Zhang et al. [112] introduced streamtubes to encode the local diffusion tensor information along the cross-section of the fiber tracts at each voxel. This technique has previously been used to represent the tensor field in fluid dynamics, where they were called Hyper-streamlines [25]. Figure 3 shows the most commonly used representations for deterministic tractography.

So far, we have discussed the visualization pipeline methods used in DTI without the involvement of uncertainties. In the following sections, we will review the sources of uncertainty present in the pipeline, the modeling techniques and the strategies used to visualize them.

#### **3 Sources of Uncertainty**

Understanding the sources of uncertainties is essential to provide effective visualization. The DTI visualization pipeline involves complex stages of mathematical modeling, analysis, mapping, and rendering strategies, therefore, it is prone to uncertainty from various sources. Noise, patient movement, modeling residuals, and distortion from imaging artifacts produce uncertainty in the orientation of the diffusion tensor and are detrimental to fiber tracking algorithms. These uncertainties hamper the link between the data being measured and visualized. The sources of uncertainty involved at each stage of the DTI visualization pipeline are shown in Fig. 1. In this section, we will go through this pipeline and discuss the sources of error present at each stage. Even though we focus on the DTI modality, several of the sources of uncertainty are present in DWI pipelines that go beyond DTI.

#### *3.1 Image Acquisition*

MRI-based techniques usually suffer from various acquisition errors such as noise, motion artifacts, partial volume effects, etc. Signal to noise ratio in DWI sequences is relatively high given that signal attenuation is being measured. The effect of noise on the fiber tracking output has been widely studied in literature [3, 46, 59]. There has been a growing trend of increasing the gradient direction in DTI acquisition to improve the tractography quality. However, this further increases the acquisition time. In HARDI, the gradient directions for acquisition are much higher than that of DTI and, therefore, it needs more time. With higher acquisition time, it is more likely that the subject move during the scan, which in turn, introduces misalignment in the acquired image. These kinds of artifacts are known as motion artifacts. Providentially, these misalignments can be corrected during the registration process. Several automated techniques have been introduced to remove this artifact [113]. The finite resolution of the results also affects the output of the process. The resolution of a clinical DWI acquisition is typically in the order of millimeter (mm) in each direction, which is much lower than that of actual axons. Therefore, the signal values have to be averaged to be able to fit in a single image voxel. This loss of information is called the partial-volume effect (PVE). Several studies have been conducted in neurological literature to investigate the PVE in DTI [84, 85, 101]. Other sources of error during image acquisition involve magnetic distortion, scanner setting and others [14].

#### *3.2 Diffusion Tensor Calculation*

In DTI, the diffusion of a water molecule is mathematically represented by a secondorder tensor, known as the diffusion tensor. Numerous measurements are performed along various gradient directions to determine the molecular diffusion at each voxel. The least-squares method is the most commonly used fitting technique to calculate the diffusion tensor, but other more accurate regression procedures can also be used [5, 6]. This fitting procedure introduces a fitting error and involves a model choice. Therefore it adds variation in the outcome of the DTI procedure. DTI technique can only estimate one dominant diffusion direction per voxel, and thus, is incapable of determining the structure where the multi-fiber direction is present and, therefore, results in unreliable outcomes. HARDI models emerge to overcome this limitation and able to model complex fibrous regions of the brain. It provides a way to estimate the multi-fiber populations that can then be used for robust tractography. HARDI models are more complex and usually introduce more parameters and choices to be determined than DTI.

#### *3.3 Fiber Tracking*

Fiber tracking involves the reconstruction of the fibrous structure of the brain white matter by gradually following the local fiber orientation estimated from the diffusion tensor. There are several parameters in fiber tracking to control the tracking process, however, these parameters add variability to the fiber tracking results. There are four major sources of uncertainties in the fiber tracking algorithm:


**Region definition and filtering**: Regions are usually defined by the user to start, end, or control the fiber pathways. The seeding region refers to the starting point of the tracking process and defines the initial conditions for numerical integration. Regions are also used to extract a specific bundle of interest and filter out others to avoid visual clutter. The region definition in the fiber tracking process also adds variation in the outcome. Usually, these regions are defined manually, and therefore introduce an implicit user bias. A minor variation in the definition can result in largely different pathways. Recently, several techniques have been proposed to minimize the effect of seed region in the fiber tracking algorithms [21, 46, 102].

**Numerical approximation**: Different types of numerical approximation schemes can be implemented in the fiber tracking algorithm. Euler integration is the most straight forward technique [71] but it is a strong approximation. Higher-order methods, such as 2nd or 4th order Runge-Kutta methods [8], are typically less sensitive to noise and can be used for accurate fiber tracking. The integration step, or step size, further affect the quality of these integration schemes [97].

**Interpolation**: During the numerical approximation process, most of the time, the sample position after each integration step lies between volume grid points, hence, interpolation is needed to estimate values, based on the neighboring grid points. Several studies have been conducted to address the effect of interpolation in fiber tracking [32, 109]. Various kinds of interpolation schemes are present, each result in different pathways, and therefore, add variability in the results.

**Stopping criteria**: Different scalar measures, such as FA, MD, or curve angle, can be used as stopping criteria in the fiber tracking process. Fiber tracking algorithms are often highly sensitive to these values, meaning that a very small variation in the stopping criteria can lead to a very large change in the resulting fiber [97]. Brecheisen et al. [16] propose a visual exploration tool that allows users to investigate the behavior and sensitivity of DTI fiber tracking to stopping criteria.

In fiber tracking algorithms for HARDI models the principal directions are extracted from a multifiber representation which adds another layer of complexity to the algorithms.

#### *3.4 Visualization*

The visualization stage involves the mapping of the data into a geometric representation or visual primitives that are finally rendered on to the screen. This process can be another source the uncertainty. Various photo-realistic rendering techniques are used to simulate real world lighting as exact as possible, but this further complexity adds uncertainty in the outcome. Lighting models and shadows enhance the structural perception of the fibers and as such improve the recognition of the spatial relations between tracts; however, the controlling parameters can add further variability in the final results.

#### **4 Uncertainty Modeling**

As discussed in the previous section, many sources of uncertainties are present at each stage of the DTI visualization pipeline that affect the outcome of the process. These uncertainties propagate through the pipeline adding uncertainty in the derived quantities including diffusion tensor and fiber orientations. Estimating the error distribution of different sources is not a straight forward task. Different approaches have been used to model the uncertainty, however, each with pros and cons. We have classified the methods used for the uncertainty quantification into two categories:


#### *4.1 Analytical Methods*

Analytical methods refer to approaches that provide an explicit mathematical formulation of the error distribution. These modeling techniques are based on the Bayes theorem [56] and were first introduced by Behrens et al. [12] in DWI. They estimated the probability distribution function (PDF) of the fiber orientation by a Bayesian model. The main disadvantage of this modeling technique is that they rely on the assumption of prior and noise present in the data. These techniques are computationally inexpensive, however, their dependence on the prior assumption limit their widespread use. Most of the Bayesian model-based techniques are often combined with random sampling methods, such as Markov Chain Monte Carlo (MCMC), to determine the distribution of model parameters [11, 12, 33]. The application of Bayesian model based methods in DTI and HARDI has been reported several times [48, 54, 64].

Shortest path algorithms are another useful approach for quantifying structural brain connectivity and were first introduced by O'Donnell et al. [78]. This approach relies on computing the connections between regions of interest rather than connections from a seed. Schober et al. [88] presented the distribution of the shortest path as a Gaussian process over the solution to an ordinary differential equation (ODE). This strategy offers novel ways to quantify and visualize uncertainty arising from the numerical computation and allow marginalization over a space of feasible solutions. Hauberg et al. [40] extended this work and incorporated data uncertainty in DTI by sub-sampling the diffusion gradients and solving the noisy ODE. Several other studies using the shortest path algorithms in fiber tracking can be found in the literature [39, 63].

#### *4.2 Stochastic Methods*

Describing the probability distribution analytically and propagating it through the visualization pipeline is extremely difficult and often not feasible. The alternative and the most straight forward way to estimate the probability distribution function is to repeat the acquisition multiple times, this approach is called the bootstrap method [27]. However, for robust estimation of the PDFs, hundreds of data sets are required, which is not practical in a clinical setting. Several stochastic techniques were proposed to overcome this limitation [20]. Among them, the most widely used techniques are residual bootstrapping and wild bootstrapping [106]. These techniques rely on a single scan and estimate the probability distribution from the residuals that remain after fitting diffusion tensor to the data. In residual bootstrapping [24], the distribution is estimated by randomly assigning the residuals among gradient directions. Another possibility is to resample the data based on randomly flipping the sign of the residuals by assuming symmetry in the distribution. The latter approach is called wild bootstrapping [23]. A detailed comparison of bootstrap methods has been presented by Chung et al. [20]. Stochastic bootstrapping has been widely used for DTI [50, 60, 79, 100]. These techniques generate multiple DTI volumes through stochastic simulations for estimating the probability distribution, however, they are computationally very expensive.

Various stochastic algorithms were introduced to incorporate uncertainty in tractography by adding randomness in the tracking process. These techniques are called probabilistic tractography [11, 87, 95]. These algorithms estimate the probability density function of the fiber orientation at each voxel and determine the propagation direction by drawing random samples from the distribution. Probabilistic fiber tracking is preferable in most cases as it takes uncertainty into account and can estimate the confidence interval for each reconstructed pathways, however, they are computationally expensive [12, 26]. Koch et al. [58] propose to use Monte Carlo random walks for the estimation of the fiber connectivity. The fiber tracking algorithm proceeds through each randomly selected neighboring voxel depending on the angle between the voxel's main eigenvector and its connecting angle with the neighboring voxels. A similar approach has been used in other studies to establish a connectivity map in a probabilistic sense [10, 13, 34, 80]. Monte Carlo methods have also been used to generate fiber tracks based on random particle movement [38]. The PDF obtained from the analytical methods can be used to perform tractography with these stochastic techniques [33, 48]. These studies are based on DTI, however, the concept is extendable to HARDI as well, but they are not used much in this context [64].

#### **5 Uncertainty Visualization**

So far, we have discussed the sources of uncertainty present in the visualization pipeline and the methods used for their quantification. Visualization provides a way to communicate data effectively and efficiently, however, uncertainty is often omitted in the process. Visualizing uncertainty information in DWI can help assess the accuracy of the acquisition and modeling, which ultimately guide the users in making critical decisions. However, the visualization of complex data in itself is not straightforward, adding uncertainty representation to it further complicates the process. Issues of visual cluttering and loss of anatomical context are some of the few complications when visualizing uncertainties.

In this section, we will survey the strategies used for the visualization of the uncertainties in DTI and also discuss some related techniques used in the HARDI model. We also summarize these strategies in Table 1. The modeling column refers to the uncertainty quantification techniques, such as stochastic, bootstrapping, or analytical methods. Domain indicates the application area of the study and ensemble column categorizes the method into the local or global level. The representation specifies the measure used for the aggregations of the ensemble, and finally, the visualization column indicates the technique used to display the uncertainties. The visualization of uncertainty in DTI can roughly be divided into two categories.


#### *5.1 Local Uncertainty Visualization*

Local representations of the uncertainty depict variation per voxel inside the vector or tensor fields. Glyphs are typically used to depict the voxel-wise information of the data. Several glyph-based techniques have been proposed to visualize the inherent local uncertainty in DTI. Jones et al. [50] proposed a method to represent the confidence interval of the main fiber direction by rendering an uncertainty cone, as shown in Fig. 4a. Basser et al. [4] used a similar technique to represent the main eigenvector and their associated uncertainties. This visualization approach allows the representation of the main diffusion direction and the confidence interval concurrently, also described in Table 1. Schultz et al. [90] demonstrate a new glyph design, called HiFiVE, that provides a more detailed impression of the uncertainty. It represents the variation corresponding to the main eigenvector by rendering a double cone (blue


**Table 1** Summary of uncertainty representation and visualization strategies

(continued)


**Table 1** (continued)

color) and the density estimation of the uncertainty around it (represented as a gray surface), as shown in Fig. 4b.

Another way to represent the uncertainty in multivariate data is to estimate its covariance. It does not only express the variance in each coefficient but also indicates their linear dependencies. Since the diffusion tensor is a second-order tensor, its covariance is represented by a fourth-order tensor, however, the visualization of the fourth-order tensor is rather difficult in this context. Basser et al. [7] presented a novel technique for the spectral decomposition of the fourth-order covariance tensor

(c) Decomposed ensemble representation [110] (d) ODF glyphs [96]

**Fig. 4** Glyphs with uncertainty encoding

and introduced the concept of tensorial normal distribution. They proposed a glyph representation, called radial glyphs , which depicts the overall variance and a composite glyph for representing the eigentensor of the fourth-order covariance. They visualized the expected mean tensor and its standard deviation as three isosurfaces. Abbasloo et al. [2] highlight that the radial glyph does not convey the correlation with the mean tensor and also suffers from high visual complexity in the tensor field. They proposed a more intuitive approach for the visualization of the covariance by using multiple levels of detail. Unlike Basser et al., Abbasloo et al. visualize the confidence interval at each eigenmode separately by glyph overlays and used animation to visualize the differences in each mode. Gerrits et al. [35] pointed out the shortcoming in both of these visualization techniques and proposed a generic approach that incorporates all the coefficients of the mean tensor and covariance in a single glyph.

Various studies have been published concerning the representation of the tensor ensemble directly. Jones et al. [52] visualize the ensemble data simply by overlaying several glyphs. Although the superposition depicts the overall picture of the data, it adds visual clutter and occlusion during display. To remedy this, Zhang et al. [111] used transparency to minimize the occlusion. Abbasloo et al. [2] tried to minimize this problem by rendering the superimposed glyphs in complementary colors. Zhang et al. [110] proposed an approach to decompose the tensor data into three properties (i.e., scale, shape, and orientation), representing the structure of the underlying fibers, and measure the variation per property. A glyph based representation has been presented in this study to visualize the ensemble effectively. The variation in the ensemble is represented by Halo and texture over the surface as shown in Fig. 4c.

The orientation distribution function (ODF), associated with HARDI, specifies the overall amount of diffusion in a given direction. Unlike the diffusion tensor model, ODFs can have multiple maxima, and therefore are capable of modeling complex fibrous structure more accurately than DTI. However, this technique is computationally expensive. The representation of the ODF itself is a challenging task and adding uncertainty information only increases the complexity. Jiao et al. [49] proposed a technique to visualize uncertainty over polar ODF glyphs by using a volume rendering technique. They introduced shape inclusion probability (SIP) function to represent the orientation uncertainty of the tensor. Tournier et al. [96] presented a method to visualize uncertainties associated with ODFs by using semitransparent glyphs. They represent the mean ODF by the opaque surface and the mean + standard deviation by the transparent surface, as shown in Fig. 4d.

The visualization of uncertainty in a diffusion tensor is similar to the uncertainty representation in a vector field where orientation is considered important. Several glyph-based techniques exist in this scope. Wittenbrink et al. [108] presented a glyphs based representation of the uncertainty for atmospheric and oceanographic data. Likewise, Hlawatsch [43] and Lodha et al. [65] visualize the local uncertainty in a fluid flow field using glyphs. Zuk et al. [115] proposed a glyph design to provide uncertainty information in a bidirectional vector field. These techniques rely on the representation of the vector direction and magnitude with encoded uncertainties to depict the local uncertainty present in the field.

#### *5.2 Global Uncertainty Visualization*

In contrast to the local strategies, global uncertainty visualization in DTI aims at providing information on how accurate fiber tract information is throughout the complete tensor field, and how the inherent uncertainties accumulate during the tracking process. In DWI independently of DTI or HARDI models being used, probabilistic tractography is often used to incorporate these uncertainties. The most widely used approach to visualize fibers obtained through probabilistic tractography is to superimpose the resulting fibers in a so-called spaghetti plot [13, 22, 51], see Fig. 5a. This visualization technique, however, does not depict a clear view of the regionwise fiber connections and its uncertainty and suffers from strong cluttering. Color coding the fiber tracts according to their seed points [22] does not suffice to minimize the complexity of the visualization. Schober et al. [88] and Hauberg et al. [40] used wobbly spaghetti plot that emphasize the fact that the individual resulting paths cannot be considered as real fibers in the brain which is a common misinterpretation in spaghetti plot. Instead, they are uncertain estimates of fibers.

To overcome the complexity and clutter caused by the multiple superimposed tracts, Enders et al. [29] presented a technique to group the fibers related to a certain nerve tract and generate a surface that wraps the resulting fibers. Similarly, Mehrof et al. [68] and Chen et al. [19] presented a method to cluster the fiber with a proximity-based algorithm and generate hulls encompassing the fiber bundles, as

**Fig. 5** Global uncertainty visualization strategies

shown in Fig. 5b. The anatomical grouping helps the user to understand the underlying fibrous structure. Outside of DTI, Frest et al. [30] used a similar technique to visualize uncertainty in flow field ensembles. They performed principal component analysis to cluster the streamlines in a low dimensional space and determine the mean and confidence interval in an ensemble. These representations are visualized with a line enclosed by a transparent surface. The geometrical hulls and enclosed surfaces reduce clutter, however, they cannot resolve complex cluster shapes. To alleviate these problems, Illustrative techniques have been proposed to represent the confidence interval of the fiber bundle by creating silhouette, outline, and contours [15, 76], as shown in Fig. 5c.

To improve the understanding of ensembles of curves, it has been proposed to visualize the statistical information such as mean or confidence intervals rather than the direct ensemble visualization as spaghetti plot. Table 1 indicates the various representations used by the studies. These representations, e.g, mean and confidence interval, are the summarization of the raw samples. Unlike scalar values, the statistical measures are not well defined for curves, and therefore, several approaches have been proposed for the estimation of these terms. Brecheisen et al. [15] proposed to compute median and confidence intervals based on pre-selected distance measures between curves. In the field of fluid dynamics, a band-depth concept [66] has been introduced to analyze curve ensembles in two-dimensions [105] and three-dimensions [69]. This concept provides a way to determine centrality within the present curves and estimate the variations. Sanyal et al. [86] visualize the uncertainty in the wind trajectories by creating a ribbon along the ensemble mean. The width of a ribbon represents the variability at each point.

A widely used approach for the visualization of the global uncertainty is to represent and visualize measures derived from the probabilistic tractography. Voxel-wise fiber density computes the probability that a fiber tract traverses a voxel for a given seed region [17]. Voxel-wise fiber density [28, 38] helps to infer the anatomical connections. Another measure is the connectivity probability, which represents the probability of a fiber tract crossing a given voxel while connecting two fixed anatomical regions [91]. Von Kapri et al. [55] and McGraw et al. [67] used volume rendering for the visualization of density maps, as shown in Fig. 5d. The global visualization of the fiber tracts does not provide the local tensor information. To visualize the local uncertainty along with the probabilistic tracts, a stream tube technique has been proposed [53, 107], which maps the local uncertainty measure onto the cross-section of the tube.

A common problem with the three-dimensional approaches is that the geometrical representation often occludes the underlying information, hampering its interpretation. Various slice-based methods have been proposed for the visualization of probabilistic fibers [58, 80]. These techniques have been used in neuroscience as they provide a way to directly visualize the anatomical information, making it easy to interpret anatomical context. Goldou et al. [36, 37] presented a novel slice based approach for visualizing the probability by rendering fiber stipples. The number of stipples, present at a particular region depicts the fiber density. Hlawitschka et al. [44] proposes to use poisson-disk sampling for the generation of the fiber stipples.

Table 1 summarizes the survey indicating the domain, representation, and visualization strategies used to display the uncertainty. The table covers the approaches used for local and global uncertainty visualization in both the DWI and non-DWI domain.

#### **6 Conclusion**

Diffusion-weighted imaging relies on complex stages of signal acquisition, mathematical modeling, model assumptions, and hence, is exposed to many sources of uncertainty. Excluding this uncertainty from the visualization does not only affect the result but also cripples the user to make an effective decision. However, the efficient visualization of the uncertainty in DWI is nontrivial, as the data itself has high visual complexity and adding uncertainty to it only adds further complexity. In this chapter, we explored uncertainty in the various stages of the DTI visualization pipeline. Several of the problems and solutions discussed throughout this chapter are also valid for other models beyond Diffusion tensor, such as HARDI models. Even though we have not covered the technical background, where applicable, we have discussed the applicability of the strategies beyond DTI. Further, we have reviewed applicable uncertainty visualization techniques beyond the DWI domain.

DWI is still a growing field, considering the recent advancements and the frequent development of new techniques, this survey should not be considered complete, it rather should be enhanced in the future. Studies on uncertainty visualization so far are mostly focused on the research aspect, however, no uncertainty visualization solution exist to specifically support clinical tractography. Visual analytics, an emerging field in visualization, can be helpful in enabling detailed analysis of uncertainty present in DWI data, building on the top of the studies present in this survey. Most of the presented studies deal with uncertainty on the noise and modeling level, dealing with other sources of uncertainty and visualizing them as a whole part of the exploration is another open research direction.

We focus on the visualization pipeline and techniques mostly related to DTI, a study including other modeling techniques would be beneficial. In summary, even though the uncertainty visualization in DWI has evolved considerably in the last few years, we believe, a lot of work still needs to be done for the effective visualization and exploration of DWI data.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Challenges for Tractogram Filtering**

**Daniel Jörgens, Maxime Descoteaux, and Rodrigo Moreno**

**Abstract** Tractography aims at describing the most likely neural fiber paths in white matter. A general issue of current tractography methods is their large false-positive rate. An approach to deal with this problem is tractogram filtering in which anatomically implausible streamlines are discarded as a post-processing step after tractography. In this chapter, we review the main approaches and methods from literature that are relevant for the application of tractogram filtering. Moreover, we give a perspective on the central challenges for the development of new methods, including modern machine learning techniques, in this field in the next few years.

**Keywords** Diffusion MRI · Tractography · Tractogram filtering

#### **1 Introduction**

Diffusion magnetic resonance imaging (dMRI) is currently the method of choice for assessing the local microstructure in the white matter (WM) of the human brain in vivo. Tractography methods use this local microstructure to generate streamlines aiming at modeling the underlying anatomy of neural fibers in the brain. These streamlines are locally aligned with the estimated tissue orientation. The set of obtained streamlines is usually referred to as a *tractogram* and is the basis for different subsequent analyses.

R. Moreno e-mail: rodmore@kth.se

M. Descoteaux Université de Sherbrooke, Sherbrooke Connectivity Imaging Laboratory, 2500 Boulevard de l'Université, Sherbrooke, QC J1K 0A5, Canada e-mail: M.Descoteaux@USherbrooke.ca

© The Author(s) 2021

D. Jörgens (B) · R. Moreno

Department of Biomedical Engineering and Health Systems, KTH Royal Institute of Technology, Hälsovägen 11C, 14157 Huddinge, Stockholm, Sweden e-mail: danjorg@kth.se

E. Özarslan et al. (eds.), *Anisotropy Across Fields and Scales*, Mathematics and Visualization, https://doi.org/10.1007/978-3-030-56215-1\_7

Tractography has potential for clinically-relevant applications. For instance, visualizations of tractography data are used by neurosurgeons with the goal of preserving important neural bundles during brain tumor resection [9, 16, 64]. Moreover, it is possible to group streamlines into bundles and perform analyses based on these. This so-called bundle-based or tract-based analysis (BBA/TBA) allows to compute statistics for individual bundles, either averaged or along the streamlines. This can be used to perform group comparisons [8, 10, 60, 70] or to analyse WM changes in patients over time [5]. Focusing on the tractogram as a whole, the concept of *structural connectivity* is employed for the analysis of how pairs of cortical and subcortical regions are connected through streamlines of the tractogram [49, 59]. These connections can be summarized in graphs, which can be seen as estimations of the actual anatomical connections between GM regions through neural pathways in the WM. Such graphs serve as the basis for different group comparisons. For example, this approach has been applied to different neurological diseases, including among others epilepsy (e.g., [21]), multiple sclerosis (e.g., [31]) and Alzheimer's disease (e.g., [17, 41]), as well as for assessing differences in the brain due to normal aging (e.g., [13]).

Despite its potential in the aforementioned applications, the validation of tractography is an open challenge in the field. Several recent studies have unveiled that state-of-the-art methods to construct tractograms suffer both from *false positives* (FP), i.e., streamlines that are not related to anatomical structures [32], and *false negatives* (FN), i.e., missing streamlines to accurately describe known WM anatomy in its whole extent [2]. At the same time, findings indicate that tractography results are to some extent reproducible [34]. That means that FPs are an intrinsic problem of current methods [32]. Already in 2014, Thomas et al. hypothesised that there are inherent limitations of tractography that lead to effects of both, FPs and FNs [62]. The importance of these two problems is application dependent. For example, it has been argued that for structural connectivity analysis, a high specificity is of higher importance than sensitivity [71]. This means that, for this specific application, reducing FPs is more important than reducing FNs.

Despite the efforts in recent years, new approaches have still not overcome the limitations of tractography methods [53]. In the particular case of FP, an alternative to improving tractography methods is to remove the FPs from the tractogram in a post processing step. In the following, we refer to this approach as *tractogram filtering*. Several methods from the literature can be used for this goal. Even though some methods were not designed for tractogram filtering, they can be adapted for this purpose. In this chapter, we consider tractogram filtering as a binary classification problem in which we aim at assigning either a positive (P) or a negative (N) label to each streamline. This allows us to assess the possibility to use a particular method to define the labels P and N for, ideally, separating true positive (TP) and FP streamlines in a tractogram. In the following, we review the most relevant methods in literature, point out their issues and pose key challenges on the way to improved tractogram filtering.

The chapter is organized as follows. Section 2 describes the main approaches and methods for tractogram filtering. Section 3 describes the key challenges of tractogram filtering and gives a perspective on future developments in the field. Finally, we make some concluding remarks in Sect. 4. In order to illustrate specific properties or issues of the different strategies, we run some experiments on a very limited amount of data from the Human Connectome Project (HCP).

#### **2 Approaches for Tractogram Filtering**

In the following sections, we review methods that can be used for tractogram filtering. Based on the definition of the labels P and N for each streamline, we identified four main criteria that these methods build upon, namely: *explainability of the diffusion signal*, *inclusion and exclusion regions of interest* (ROIs), *streamline geometry or shape*, and *streamline similarity and clustering*. Figure 1 shows the problem of tractogram filtering interpreted as a binary classification problem and lists the most relevant methods in this realm together with the main criteria used to group the streamlines.

Table 1 lists the most relevant tractogram filtering methods that are reviewed in the next subsections. The columns of this table describe the most relevant characteristics of these methods, namely: *criteria*, use of *dMRI* data, required *context*, main *target* and whether or not the method is *data driven*. First, *criteria* refers to the strategies from Fig. 1 that are followed when employing the method for streamline classification. Column *dMRI* in the table indicates if the method used the dMRI data or not. As shown, only a few filtering methods make use of the acquired dMRI for performing the classification. The listed methods also differ in the required streamline *context*. While some are able to perform the classification individually per streamline, others also require the streamlines in the bundles of interest or the complete tractogram as extra inputs (column *context*). Furthermore, by design, the target of some methods is to define either the positive (i.e., P) or negative (i.e., N) label but not both. In other words, some methods aim at being more specific detecting TPs than FPs, and the other way round for others. As an example, a streamline classified as negative by Recobundles can still be a TP, since it can belong to a bundle not present in the atlas. The target of the method is important to be considered, since preferring higher specificity of P or N is application dependent. This is shown in column *target* in the table. Finally, the methods can also be grouped into *rule-based* and *data-driven* ones. While the former makes use of classical approaches, the latter use machine learning techniques for performing the classification.

In the following sections, we describe the aforementioned criteria in detail and list the most representative methods that use those criteria to perform the classification of streamlines.

**Fig. 1** Tractogram filtering can be seen as classifying each streamline in a tractogram as either positive (P) or negative (N), targeting true positives (TPs) and false positives (FPs), respectively. Relevant methods use one or more of the depicted criteria for performing this classification. Some representative methods are also depicted in the figure. For a detailed list refer to Table 1

#### *2.1 Explainability of the Diffusion Signal*

The idea behind this approach is that high-quality tractograms can be used to explain the acquired dMRI data. In other words, synthetic dMRI generated from tractograms should be very similar to the acquired dMRI data. Thus, these methods focus on finding a subset of streamlines which can generate data that approximates the measured signal as closely as possible. Streamlines not belonging to such a subset are likely implausible (or duplicates of other streamlines already contributing to the signal) and might be removed. This approach is shown in Fig. 2.

**Table 1** Representative methods for tractogram filtering. **Criteria** shows the criteria used for streamline classification (cf. Fig. 1). Column **dMRI** indicates if the method uses diffusion data. **Context** describes the contextual information required for streamline classification. **Target** shows the target labels where the method is more specific. **Data driven** specifies if the method makes use of machine learning or not


**Fig. 2** dMRI signal fitting approach. Synthetic dMRI is computed based on a microstructure model and compared to the acquired dMRI. The streamlines that are not relevant to minimizing the residual might be filtered out

As shown in Table 1, these methods require the whole tractogram as an input. In the following subsections, we will describe the most commonly used methods for dMRI signal fitting and discuss their issues.

#### **2.1.1 LiFE and COMMIT**

Linear Fascicle Evaluation (LiFE) [7, 42] and Convex Optimization Modeling for Microstructure Informed Tractography (COMMIT) [12] state the problem as follows: let **y** be a vector with the acquired diffusion signals, **A**(**T**) be a forward operator for synthesizing diffusion data from the streamlines of the tractogram **T**, **x** be a vector with weights for the contribution of every streamline to the acquired data, and η be the acquisition noise. Then **y** can be written as:

$$\mathbf{y} = \mathbf{A}(\mathbf{T})\,\mathbf{x} + \eta. \tag{1}$$

Since the weights **x** cannot be negative, it is possible to solve (1) through non-negative least squares:

$$\mathop{\arg\min}\_{\mathbf{x}\succeq 0} \min ||\mathbf{A}(\mathbf{T})\,\mathbf{x} - \mathbf{y}||\_2^2 \tag{2}$$

Filtering is performed by discarding streamlines with low weights. This formulation allows for the use of different models to couple the information from streamlines to the measured signal or derivatives thereof. First, **A** can be chosen from a large variety of forward operators proposed in the literature [39]. As an example, in the original papers, COMMIT was based on a multi-compartment forward model while in LiFE the stick and ball model was used. Notwithstanding, both can be adapted to use any other model. Second, different solvers can be applied for solving the nonnegative least-squares problem of (2). Both COMMIT and LiFE use the subspace Barzilei-Borwein (SSB) solver proposed in [28]. Due to the nature of the problem, sparsity on weights is desirable. For this purpose, COMMIT proposes a basis pursuit de-noise (BPDN) formulation of (2) that actively considers sparsity by minimizing the 1-norm of **x**. Such a formulation can be written as:

$$\underset{\mathbf{x}\geq 0}{\text{argmin}} ||\mathbf{x}||\_1, \text{ subject to } ||\mathbf{A}(\mathbf{T})\,\mathbf{x}-\mathbf{y}||\_2^2 \leq \epsilon,\tag{3}$$

where is a parameter.

In order to reduce the inherent computational burden of these strategies, **A** is implemented in LiFE and COMMIT through a lookup table on a dictionary of precomputed estimations. Moreover, a GPU-based optimized version has recently been proposed for LiFE [29].

#### **2.1.2 SIFT and SIFT2**

Very similar to the techniques from the previous subsection is Sphericaldeconvolution Informed Filtering of Tractograms (SIFT) [55]. Instead of targeting the raw dMRI data, it aims at reconstructing the fiber orientation distribution function (fODF) in each voxel. First, the fODF is obtained with constrained spherical deconvolution (CSD) [63]. Second, the contribution of every streamline to the fODF is assessed. These contributions are used to determine whether a streamline is deemed redundant/noisy or not. Third, these contributions are sorted in order to remove the least relevant streamlines. Finally, the aforementioned two steps are iterated until either a target number of streamlines or a certain residual level is reached. Unlike LiFE and COMMIT, SIFT does not generate weights per streamline. Thus, SIFT2 was proposed as a slight modification of SIFT in which an additional regularization term is added and a weight per streamline is computed [56].

In order to compare the agreement between SIFT and SIFT2, we run them with their standard parameters on a whole-brain tractogram computed with anatomicallyconstrained tractography (ACT) [54] from MRrtix31 with one million streamlines obtained from one HCP subject. In this experiment, SIFT selected 34.6% of the streamlines in the original tractogram. Figure 3 shows the histograms of weights computed with SIFT2 where the individual histograms are obtained by separating the streamlines based on whether they were accepted or discarded by SIFT. It can be seen that the two histograms have a big overlapping region. A repetition of the experiment with 500k streamlines showed similar results. This means that it is difficult to reproduce the results from SIFT with the weights computed with SIFT2. Thus, while SIFT2 can be useful for describing the contributions of streamlines to the acquired data, unlike SIFT, its direct use for tractogram filtering is not straightforward.

<sup>1</sup>https://www.mrtrix.org/.

**Fig. 3** Histograms showing the frequency of different SIFT2 weights computed for streamlines that are filtered out (in blue) or not (in orange) with SIFT. Both methods were computed on an HCP subject with 1 million streamlines computed with ACT in MRtrix3

#### **2.1.3 Issues of dMRI Signal Fitting**

While dMRI signal fitting is appealing, it has not been able to significantly reduce the false positive rate [50, 52]. This fact can be attributed to various reasons. Daducci et al. [11] discuss a number of issues of dMRI signal fitting. First, most of the current dMRI signal fitting methods require a whole brain tractogram, even if a single white matter bundle is of interest. This results both in an unnecessary computational burden and a higher risk for false positives when targeting specific fiber bundles. For example, if a fiber bundle is not appropriately represented in the full tractogram, which is not uncommon, there is a higher risk for implausible streamlines from other bundles to take their place in the reconstruction. Second, the computed weights of streamlines tend to be inversely proportional to the number of similar streamlines in the tractogram. This effect is shown in Fig. 4 for the case of SIFT2 (a similar behavior is expected from LiFE and COMMIT). As shown, the SIFT2 weights are lower in the centerlines of the bundles, where tractography tends to yield more streamlines. Thus, thresholding of SIFT2 weights cannot be used for progressive filtering, since that might result in discarding the most important tracts very early. Moreover, some noisy streamlines might be classified as acceptable just because of the reward they get for reaching distant regions. This issue comes from the fact that weights of SIFT2 are designed for fitting the acquired data, but not for filtering. This could potentially be solved through an extra step of weight normalization that, to our knowledge, has not been proposed so far. Third, as discussed in [11], minimizing the residual from Fig. 2 does not guarantee that the solution is plausible as the current methods are very prone to overfit due to the large amount of unknowns that must be estimated. Finally, working with incomplete streamlines can lead to biased results in the uncovered regions, which might be especially problematic for structural connectivity.

**Fig. 4** Average weights computed through SIFT2 per voxel. Darker and brighter voxels correspond to lower and higher SIFT2 weights, respectively. The values are extracted from the same dataset of Fig. 3

**Fig. 5** The course of two close fiber bundles is depicted in dashed green. A local error (blue box) can lead to a noisy streamline (in red). Since most segments of its path are correct, it still might be classified as acceptable by most dMRI signal fitting methods

In addition to these issues, most dMRI signal fitting methods compute a single weight per streamline, which can lead to acceptance of implausible streamlines that are erroneous in a small region (cf. Fig. 5). As suggested in [42], this issue can be handled by having variable weights along the streamlines. However, this solution might come at a cost of numerical instability. Finally, an important aspect to consider is the applicability of these methods to diseased brains. For example, it was reported in [72] that using SIFT in certain types of illnesses (e.g. brain tumors) can lead to wrong conclusions of connectivity changes.

In [69], we tested the performance of SIFT2 in short synthetic streamlines. First, we generated dMRI data with Fiberfox [35] with and without noise from a set of straight and bent short streamlines. Second, we added a set of streamlines at a different angle. These added streamlines should ideally be classified as noisy as they do not comply with the generated dMRI data. Finally, SIFT2 was run in order to assess its ability to separate the 'signal-generating' streamlines from the added ones. The main result of these experiments is that SIFT2 is able to filter the implausible streamlines in this simplified problem only for straight streamlines with b-values below 3000 s mm−<sup>2</sup> and low levels of noise. Our hypothesis for this is that dMRI signal fitting methods might benefit from longer streamlines for increasing their stability. However, as already discussed, stability cannot guarantee a good solution as shown in Fig. 5.

It is worth to note that the filtered tractograms have been reported significantly different from the non-filtered ones [45]. However, it is difficult to assess whether or not the differences are due to the reduction of noise or to the aforementioned inherent problems of this methodology.

Finally, it is important to remark that global tractography [33, 46] is closely related to dMRI signal fitting. Both use the acquired diffusion data to assess the quality of the tractograms, one for generating streamlines, and the other to assess the plausibility of them. Thus, global tractography shares the same issues of dMRI signal fitting. On the other hand, the current dMRI signal fitting methods are more efficient than global tractography.

#### *2.2 Inclusion and Exclusion ROIs*

A different approach is to implement anatomical constraints in terms of using segmentation masks for filtering out implausible streamlines.

An intuitive way to employ such masks is atlas-based tractogram filtering. This strategy comprises two steps: first, the atlas of streamline bundles is registered to the subject data, and second, streamlines that are not fully contained within a single bundle mask after registration are filtered out. Despite its simplicity, this method has various shortcomings. First, compared to gray matter (GM), registration of WM in raw data is more challenging due to the relatively low contrast and the less convoluted structure [58]. This makes registration more prone to errors [22]. Thus, different methods have been proposed for the specific purpose of WM registration on both raw and derived features such as fractional anisotropy (FA) maps (cf. [38] for a review of methods). Second, atlas-based approaches are known for not being able to model the anatomical variability among subjects very well. Finally, the anatomical shape of fiber bundles can change due to illnesses (e.g. brain tumors), making the use of atlases difficult in such applications.

An interesting alternative to atlas-based filtering is to estimate segmentation masks of fiber bundles directly from dMRI data. By that, the problematic registration step is avoided. TractSeg [67, 68] follows this idea by training a neural network for segmenting 72 different fiber bundles. In this approach, fODF peaks are taken as input to a 2.5D U-Net architecture for segmentation [48]. Unlike atlas-based methods, TractSeg does not require registration and is subject-specific since it only uses the acquired data.

Alternatively, Wassermann et al. [66] proposed the white matter query language (WMQL) and its implementation TractQuerier, which can be used to define fiber bundles of interest using high-level relationships between GM and WM structures. The method uses any parcellation for locating regions of GM that can be used in the query. WMQL can be used for filtering out streamlines not compliant with any of the anatomical rules defined for extracting all bundles of interest. Notice that although neither TractSeg nor TractQuerier were proposed for tractogram filtering, they can be easily adapted for this purpose. Moreover, it is important to remark that TractQuerier itself is only a tool to define the mentioned rules. Thus, it can potentially be used for tractogram filtering either targeting TPs or FPs.

It is important to notice that methods using segmentation masks as inclusion or exclusion ROIs could be unable to detect certain implausible streamlines that are not rejected. That is, defining anatomy solely based on inclusion/exclusion ROIs is often only a necessary but not a sufficient condition for determining the plausibility of streamlines. Moreover, while some tractography methods do include anatomical priors in the formulation (e.g. [26, 54]), in practice, the resulting tractograms usually contain false positives.

#### *2.3 Streamline Geometry or Shape*

In this approach, the shape of streamlines is used for deciding whether or not they are plausible. Most tractography methods already include restrictions on the shape of streamlines, such as maximum local curvature or minimum and maximum length. Moreover, some tractography methods include geometry priors at a higher level [4, 15]. Since these rules are usually included in state-of-the-art tractography approaches, there has been less need for filtering methods implementing this idea.

A recurrent issue in probabilistic tractography is the existence of streamlines with unrealistic loops. In [3], changes in track density are used for detecting and removing such loops. Using a more general formulation, it is assumed in [65] that the relationship between a streamline and its neighbours should be similar along the path. Such relationships are modeled through graphs and are analyzed using spectral graph theory. This method is able to remove unrealistic loops while keeping the ones that comply with anatomy.

Recently, the usefulness of streamline geometry for filtering has been shown. For example, in ExTractor simple geometry priors are employed to assess streamline plausibility [43]. Surprisingly, the authors found that around half of the streamlines generated by many state-of-the-art methods do not comply with such priors. A graphbased neural network was trained in [1] to reduce the computational cost of ExTractor.

Additional geometry priors can potentially be useful for tractogram filtering. Geometrical features extracted from a population could be used to assess the plausibility of streamlines at a local level. For example, an atlas of local sheet probability index [61], statistical shape models of fiber bundles [15] or an atlas of local orientation and curvature [6] could be beneficial for removing false positives.

Also deep learning-based methods have been proposed for tractogram filtering, most of them relying mainly on streamline geometry as criteria. For instance, in FiberNet a convolutional neural network is used on the coordinates of the streamlines after a spatial normalization for bundle classification [24]. In [73], the authors extract a set of features from the shape of streamlines (named FiberMap) to train their bundle classifier. Streamlines not belonging to a known bundle are assigned to an additional class. Furthermore, TRAFIC trains a network using distances to five landmarks, curvature and torsion per tract as features for filtering [37]. Moreover, DeepBundle [30] used a graph convolutional neural network for extracting geometric features from the streamlines. Such learned features are then used to assign them to their more likely fiber bundle in an end-to-end fashion. The loss function can be designed to target false positives. Since these deep learning-based methods use streamlines of specific fiber bundles for training, their main target are the TPs in those bundles.

Similarly to the inclusion and exclusion of ROIs, geometry constraints of the streamlines are not necessarily sufficient criteria for deciding the validity of streamlines and must be combined with other priors. That does, however, not apply to the mentioned DL-approaches which—depending on the training labels—are able to learn a model of plausible streamline geometry.

#### *2.4 Streamline Similarity and Clustering*

With this strategy, streamlines are clustered in bundles before further analysis. Such clusters can be used as surrogates of the underlying structure ofWM. Fibers belonging to small clusters or that do not share similar properties of bundles of interest can be removed.

The only requirement for using standard clustering algorithms for streamline clustering is to define a distance metric between streamlines. While proposing distance metrics is straightforward, it is more difficult to find the most appropriate one for streamlines. Depending on the application, a tractogram can consist of millions of streamlines. Thus, it is critical to use efficient implementations. For example, Quickbundles [18] was proposed as a tool for performing clustering very efficiently.

Once clusters of streamlines are extracted, there are different alternatives for performing filtering. For example, Recobundles uses an atlas-based strategy in which the clusters are first registered to an atlas of streamline bundles, followed by a pruning procedure of streamlines lying far away from the registered centroids [19]. Following another strategy, in [74], 800 clusters of streamlines are computed for a number of subjects that, after manual curation performed by an expert, are used for creating an atlas of streamline clusters. This curated white matter atlas (WMA) is used for filtering out streamlines far away from any cluster in the atlas. Following a different idea, BundleMAP [27] uses support vector machines on the mean and covariance of the coordinates of the streamlines in a bundle to detect FPs.

Clustering methods based on deep learning have the potential to be computationally more efficient than classical approaches. In [40], the authors proposed FS2NET, a Siamese deep neural network that uses bi-directional long short term memory (LSTM) layers for learning a distance measure between streamlines. With this distance, the method can be used to assess if two streamlines should be clustered together or not.

Implausible streamlines tend to follow more erratic paths compared to plausible ones. Thus, using clustering is appealing for filtering, since it introduces strong requirements on smoothness of streamlines. Moreover, combined with atlas-based approaches, they are able to filter both whole brain or partial tractograms. Unlike the atlas-based approaches described in Sect. 2.2, the registration is performed on tractograms, which tends to be more accurate (cf. [38] for a review of methods). Still, the inherent issues of atlas-based approaches might have an impact on the accuracy of such methods.

Since the methods of this subsection are based on bundle similarity, they target only certain bundles and, by that, only TPs.

#### *2.5 Multiapproaches*

From the previous discussion, it is natural to devise methods taking advantage of different priors for increasing accuracy. Due to the fact that the research field is relatively new, only a few multiapproach methods have been proposed. In this line, COMMIT2 [50] adds anatomy priors to the original formulation of COMMIT in order to target the issues of dMRI signal fitting methods. Another example is anchorconstrained plausibility [36], which combines streamline clustering and dMRI signal fitting for performing filtering. In [23], FiberNet2.0 has been proposed as an extension of FiberNet in which inclusion/exclusion of ROIs are added to the processing pipeline.

We have recently used deep learning for combining two methods: RecoBundles and ExTractor [25] using the dMRI signal as the only input of the neural network. From our preliminary results, it is not obvious which method should be used as gold standard, as the choice of accuracy measurement depends very much on the application. Thus, while a perfect combination of priors is not straightforward, from our experience in [25], we expect new methods that combine two or more priors to perform better on average.

#### **3 Challenges and Perspective**

Machine learning and in particular deep learning has been very successful in many medical image analysis applications in the last few years. Preliminary efforts show the potential of this approach also for the specific problem of tractogram filtering [1, 25]. However, important challenges for methods following this approach remain open.

As mentioned in [44], there are important general challenges for tractography, which also apply for tractogram filtering. Specifically, machine learning and deep learning require large amounts of training data of good quality that are difficult to obtain for tractogram filtering. Moreover, the available training data is relatively scarce and difficult to combine. Also, inter observer variability of manual annotations is particularly severe in tractography [47, 52]. Furthermore, it is in general questionable if manual annotation of whole brain tractograms would ever become a feasible goal. For this reason, the definition of adequate training labels in absence of a ground truth or a strong gold standard can be expected to remain the main challenge for machine learning-based methods in this field in the future.

One way of addressing this issue is to combine different methods as automatic annotation tools in order to define a gold standard. Following this idea, in [25], we proposed a method that builds on top of two methods, namely Recobundles and ExTractor, for defining labels. However, this combination is not straightforward. As pointed out in Table 1 and in the previous sections, it must be considered that different methods assess different characteristics of the tractogram. For example, the rejection of a streamline based on geometry priors could have a higher confidence than basing that decision on a clustering argument. The reverse is also true: a streamline close to an anatomically plausible cluster might be accepted but a streamline compliant with a finite number of geometry-based rules could still violate other unchecked rules and therefore be implausible. Filtering also depends on the application. If the goal is to obtain segmentation masks, geometrical constraints could have a lower value.

In order to investigate the process of finding a good balance in the combination of different automatic annotation tools, we run Tractquerier (TQ), which is an implementation of WMQL, Recobundles (RB), TractSeg (TS) and COMMIT (CM) in a tractogram of 10 million streamlines computed with ACT [54] for one HCP subject. A naïve approach to combine the methods would be majority voting. Figure 6 (on the left) shows the acceptance rates of streamlines for the testing dataset after performing a majority voting with different thresholds. As shown, requiring at least three methods to accept a streamline would result in a massive filtering of 95.1% of the

**Fig. 6** Percentage of accepted streamlines obtained with Tractquerier (TQ), Recobundles (RB), TractSeg (TS) and COMMIT (CM) on a 10M tractogram of one HCP subject.**Left:** Thresholding the number of positive labels from the four methods per streamline (majority voting). **Right:** Individual methods

dataset. Even with a milder threshold of at least two methods, 79.1% of the dataset would be filtered out. While the amount of around 20% of accepted streamlines (i.e., two million streamlines in this example) could be enough to fill up the space of the WM with an improved TP-to-FP ratio, potential biases of such an *ad hoc* approach would need to be investigated. Also the low agreement for only 50.6% of the streamlines (0.2% for four positive votes and 50.4% for four negative votes) indicates that a more sophisticated strategy for the combination of different tools should be considered. This could maintain a higher acceptance rate of TP streamlines and by that potentially reduce the required number of streamlines in the tractogram.

Figures 6 (on the right) and 7 show the percentage of streamlines accepted by each method as well as their agreement, respectively. As shown, the most restrictive method is CM and the most relaxed one is TQ. While the other two methods are in the middle, they are also rather restrictive. Moreover, any pair of methods agrees in 60–80% of the streamlines. From these initial analyses, it is clear that more research is needed in order to find better ways of synthesising information from different methods for the purpose of tractogram filtering than just simple majority voting. Again, the final application must also be considered for assessing the ideal approach.

In addition to combining different methods, using other prior information can be potentially useful for tractogram filtering. For example, including specific microstructure information has been useful for tractography [20, 51], which can also be expected from tractogram filtering methods. Combining dMRI and functional information is promising to understand the mechanisms for brain connectivity [14]. Thus, it would be interesting to explore in the future the use of functional data for tractogram filtering.

#### **4 Conclusion**

Tractogram filtering is a relatively young but very active research area with a high potential of development. While the reviewed methods in this chapter show that there are a multitude of ways to obtain information related to the plausibility of streamlines, there is yet no holistic approach to separate the TP and FP streamlines in a tractogram in a fully satisfying way. In our opinion, machine learning-based methods have the potential to contribute substantially to tractogram filtering. However, at this moment the applicability of supervised approaches is tightly coupled to the proper definition of training labels, which is difficult to obtain in the absence of a ground truth. We see the combination of different automatic annotation tools, potentially complemented with manual annotations from neuroanatomists, as a promising avenue to address this problem, while developments are still needed in that line of research in the future.

**Acknowledgments** This work was partially supported by VINNOVA, through AIDA. We thank students Blanca Bastardés-Climent and Pehr Wessmark for their involvement in the project, and Jakob Wasserthal for computing and providing the tractogram we used in Sect. 3. We also thank the University of Sherbrooke Research Chair in Neuroinformatics and ComputeCanada for their invaluable computing resources. We used data from the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

**Modeling Anisotropy**

## **Single Encoding Diffusion MRI: A Probe to Brain Anisotropy**

**Maëliss Jallais and Demian Wassermann**

**Abstract** This chapter covers anisotropy in the context of probing microstructure of the human brain using single encoded diffusion MRI. We will start by illustrating how diffusion MRI is a perfectly adapted technique to measure anisotropy in the human brain using water motion, followed by a biological presentation of human brain. The non-invasive imaging technique based on water motions known as diffusion MRI will be further presented, along with the difficulties that come with it. Within this context, we will first review and discuss methods based on signal representation that enable us to get an insight into microstructure anisotropy. We will then outline methods based on modeling, which are state-of-the-art methods to get parameter estimations of the human brain tissue.

#### **1 Accessing Brain Anisotropy Using Diffusion MRI**

#### *1.1 Introduction*

Diffusion-weighted MR imaging is a non-invasive tool used to probe tissue microstructure. During a typical acquisition of tens of milliseconds in brain imaging, water molecules can displace up to tens of micrometers. Diffusion is therefore sensitive to a wide range of microstructural and physiological parameters in the tissue. The diffusing molecules get restricted by the boundaries of the underlying microstructure of tissues. Diffusion anisotropy corresponds to the hindrance of those molecules, otherwise free diffusing (i.e. isotropically). Changes in anisotropy have been related to brain diseases such as ischemia, multiple sclerosis, trauma, or brain tumors [2, 58]. Diffusion anisotropy is therefore considered as a potential biological marker for changes in tissue microstructure. A loss of anisotropy can also be the sign

M. Jallais (B) · D. Wassermann

Université Paris-Saclay, Inria, CEA, 91120 Palaiseau, France e-mail: maeliss.jallais@inria.fr

D. Wassermann e-mail: demian.wassermann@inria.fr

of an increasing isotropy, as neurons grow in size. Panagiotaki et al. [57] notably use this change to study the evolution of tumor cell size in response to a drug. An understanding of the origin of anisotropy change and a combined study of both anisotropy and isotropy can therefore lead to great discoveries on a tissue microstructure and its evolution.

An ultimate goal of a Magnetic Resonance (MR) diffusion theory is then to relate the microstructural and physiological parameters quantitatively to the diffusionweighted signal. This task appears to be complicated as deducing those parameters constitutes a complex inverse problem requiring careful modeling of the diffusion signal over a wide range of diffusion time and diffusion weightings (diffusion weightings will be further explained in Sect. 2.2).

Regarding its cellular composition, brain can be decomposed in two main parts: white matter and grey matter. The former designates regions that contain mainly long-range myelinated axons, which cross the brain connecting different parts of grey matter, and relatively few cell bodies. A method to study those connections is called tractography and has been well explored during the past few years [28]. Grey matter contains mainly cell bodies, connected by neurites, and relatively few myelinated axons. Anisotropy exists in both white matter and grey matter and is due to the presence of cells with long cylindrical processes (axons in white matter and neurites in grey matter). Its presence, or its absence, will provide us with key information about the tissue structure at the cellular level.

Fick et al. dedicated a review on existing diffusion anisotropy metrics [14], which includes Fractional Anisotropy (FA) [3], Generalized Fractional Anisotropy (GFA) [60], Propagator Anisotropy (PA) [50], Orientation Dispersion Index (ODI) [67], and microscopic Fractional Anisotropy (µFA) [33]. We present here a complementary approach, which considers anisotropy as a probe for accessing microstructure, either through signal representations or tissue modeling.

#### *1.2 Anisotropy as Reflected by Water Motion*

Particles suspended in a fluid are constantly undergoing small random movements, which is known as Brownian motion [42]. The physical process of a steadily spread of a substance is called diffusion. Diffusion can therefore be considered a macroscopic manifestation of Brownian motion on the microscopic level. When no barrier impedes diffusion preferentially in one direction over another, molecular displacements are equal in all directions. This is known as isotropic diffusion. However, in brain, molecule movements are hindered by cell membranes. Diffusion is then not equal along all directions anymore and has become anisotropic. The distance traveled by a water molecule depends on its interactions. Certain geometric characteristics of the underlying structure at the microscopic level can therefore be inferred from the molecule movements [32]. The further a molecule travels during the time of an acquisition, the greater signal attenuation we get. The objective is to use this attenuation to deduce the structure of the medium where the water molecules are trapped in.

Depending on the diffusion time, i.e. the amount of time between two gradient pulses (see Sect. 2.1), the information we get about the structure will be different. If the diffusion time was extremely short, only the local intrinsic diffusivity of the fluid, i.e. the rate at which particles can spread, would be measured. The hindrance effects would only become apparent at longer times. The degree of anisotropy hence also depends on the diffusion time.

#### *1.3 Structural Brain Anisotropy*

Brain tissue is very anisotropic due to the cylindrical shapes of axons and processes. Water molecules within those fibers will on average move further along them than across them due to their small diameter. Typical axon diameter in humans is of the order of 1–10 µm [5]. Process diameters in grey matter lie between 0.1 and 15 µm.

The strong anisotropy in white matter due to the axons encouraged its wide study over the past decades. The more complex tissue structure and weaker anisotropy in grey matter make its study harder. The presence of isotropy in grey matter is partly due to the numerous somas whose shapes resemble spheres (see Sect. 4.1). Soma diameters range between 20 and 120 µm. White matter models need to be adapted to account for the presence of somas in order to be applied to grey matter [38, 53]. Myelin also appears to modulate the degree of diffusion anisotropy between axons and processes (and so between white matter and grey matter), but has a smaller role in anisotropy than membrane [6].

Note that anisotropy is not only a property of neural fibers. Anisotropy has also been observed in liquid crystals, muscles and other tissues, even in fruits and vegetables [5]. The degree of fractional anisotropy (see Sect. 3.1) is however higher in healthy neural fibers than in other tissues such as skeletal muscle, kidney, and myocardium [11, 18].

#### *1.4 Measuring Anisotropy Using Diffusion MRI*

Using Diffusion Magnetic Resonance Imaging (dMRI) as a non-invasive probe in human brain, we aim at getting information about its structure. The acquired diffusion signal is a sum of the diffusion signals coming from each compartment weighted by their relative volume fractions [56], and is therefore modulated by the geometry of the tissue microstructure. Relevant information to infer from it are soma diameters, soma and process densities, and diffusivities. Two complementary approaches have emerged for extracting these information about the tissue microstructure from the diffusion signal: signal representation and tissue modelling (denomination from Novikov et al. [48]).

Signal representations aim at quantifying parameters and are model-independent mathematical expressions. Their parameters do not carry any particular physical meaning. Representations can be used to store, compress or compare measurements. There is an infinite way to represent a continuous function. One chooses a representation according to the need of a particular neuroimaging study [44]. Although signal representations are suited for all kind of tissues, they lack specificity and provide only an indirect characterization of the microstructure.

Biophysical tissue models rely on a schematic geometry of the underlying tissue. They are pictures representing a physical reality relying on assumptions meant to simplify the complexity of a biological tissue. A good model only keeps relevant features which characterize the tissue and discards irrelevant degrees of freedom. The designed analytical expression is then fit to the diffusion data in order to estimate these relevant features of the microstructure. This advantage of providing greater specificity and interpretation of biologically-relevant parameters appears to be the weakness of the method. Indeed the initial geometric assumption must be chosen as to accurately capture all of the features of the tissue that effectively impact the diffusion signal in a given acquisition range [48], but we also must be able to mathematically solve this inverse problem. Model validations are important because a wrong model could lead to wrong interpretations of a physical phenomenon.

Techniques from these two approaches, signal representations and tissue modelling, will be reviewed respectively in Sects. 3 and 4.

#### **2 Diffusion MRI: Introduction to a Non-Invasive Imaging Technique**

Nuclear Magnetic Resonance Imaging allows to non-invasively study the brain invivo, and in particular brain anisotropy, induced by its microstructure.

#### *2.1 Diffusion MRI Acquisition Sequence*

Consider an MRI acquisition sequence. After slice selection, all the nuclei on this plane are precessing at the same frequency. To obtain a diffusion MR image, two gradient pulses are added to the acquisition sequence. The first applied pulse is going to make the particles go off phase. We then apply a second gradient with the same strength in the opposite direction, during the same amount of time. If molecules stayed still between those two gradients, they would have all come back to their original phase, the two gradients cancelling each other. However, after turning the first gradient on, molecules are moving randomly (Brownian motion). After a certain evolution time, if molecules are not at the same location, the second gradient causes destructive interference, which results in a loss of signal. The further a molecule travels from its initial position during the time between the two diffusion gradients along the gradient direction, the greater signal attenuation we get. The ratio between the signal obtained with diffusion gradients and the one without them quantifies the amount of ongoing diffusion. The objective is then to deduce the structure of the medium where the water molecules are trapped in from those signal losses.

#### *2.2 Mathematical Foundations*

Stejskal and Tanner invented in 1965 the Pulsed Gradient Spin Echo (PGSE) sequence [59] to measure diffusion in a specific direction. In this sequence, two opposite diffusion gradients are applied during a time δ, separated by a time interval . The diffusion-weighting is globally encoded by the b-value [37], and reflects the strength and timing of the gradients used to generate the diffusion weighted images. This factor is computed as follow:

$$b = \wp^2 g^2 \delta^2(\Delta - \delta/3),\tag{1}$$

where γ ( MHzT−1) is the nuclear gyromagnetic ratio of the water proton <sup>1</sup>*H* and g is the strength of the diffusion gradient. In the following sections, **g** encodes the direction of the applied diffusion weighting in addition to its strength (*g* = ||**g**||), and **g**ˆ is the corresponding unit vector (**g**ˆ = **g**/||**g**||).

The quantity *E*(*b*) = *S*(*b*)/*S*<sup>0</sup> expresses, for each voxel, the attenuation of the diffusion-weighted signal along the selected gradient direction, *S*<sup>0</sup> being the image acquired without diffusion gradients. In the absence of restrictions (free diffusion), the signal attenuation can be expressed as:

$$E(b) = e^{-bD},\tag{2}$$

with D the diffusion coefficient.

If δ is assumed to be infinitely narrow, i.e. the diffusion during that time is negligible, the signal attenuation can be related to the ensemble average propagator (EAP) *P*(**r**,τ) via a Fourier relationship under the q-space formalism [9, 59]:

$$E(\mathbf{q}, \tau) = \frac{S(\mathbf{q}, \tau)}{S\_0} = \int\_{\mathbb{R}^3} P(\mathbf{r}, \tau) e^{-2\pi i \mathbf{q} \cdot \mathbf{r}} d\mathbf{r},\tag{3}$$

where **q** is the wave vector and τ the diffusion time, which, for the PGSE sequence, are expressed as

$$\mathbf{q} = \mathbf{y}\,\delta\mathbf{g}/2\pi \text{ and } \mathbf{r} = \boldsymbol{\Delta} - \delta/3. \tag{4}$$

The diffusion time τ expresses the time interval during which spins are allowed to diffuse before measurement. By increasing the spatial frequency *q* = ||**q**|| it is possible to achieve a higher spatial resolution of *P*(**r**,τ) in the displacement space described by **r**.

#### *2.3 Acquisition Strategies*

Experimental parameters, and especially *q* and τ , influence the diffusion signal attenuation along different gradient directions, and therefore the estimation of diffusion anisotropy. Ideally, many gradient directions, q-values and diffusion times would be required to completely characterize diffusion anisotropy in a tissue. In practice, the sampling strategy depends on the application and on the chosen signal representation. This way, only one shell of gradient directions and a single b-value are usually used in DTI (see Sect. 3.1). Also using only one shell at a higher b-value and more directions, are the High Angular Resolution Diffusion Imaging (HARDI) schemes, which aim at increasing the angular resolution of the diffusion signal with the intent of resolving crossing tissue configurations [61]. Different diffusion-weightings signal acquisitions are also needed for some signal representations. In that case, multi-shell acquisitions are set up using different q-shells with fixed diffusion time. Each shell then represents a collection of samples in the three-dimensional space with the same q-value. An optimal spatial coverage is important to measure the diffusion signal as efficiently as possible. Expansions have been proposed such that all the acquired samples lie on different non-collinear directions [10]. This multi-shell design can be extended to τ -shells, called *q*τ acquisitions [13] in order to exploit different values for both *q* and τ . In that case, a complete q-shell scheme is acquired for each desired diffusion time. Ning et al [44] reviews and compares 16 reconstruction algorithms (single and multi-shells) to help determine an appropriate acquisition protocol (number of b-values) and the analysis method to use for a particular neuroimaging study.

#### *2.4 Difficulties*

A main drawback to take into consideration is inherent to the dMRI acquisition process. Due to the acquisition device limitations and the mesoscopic size of neurons, one voxel, at the macroscopic scale, includes thousands of somas and processes. This means that the acquired signal is an average of the signal coming from all those cells. Several issues have then to be considered.

First, the acquired signal in a voxel will be an average of the signal of all the diffusing molecules within this voxel, which could correspond to not less than 3000 axons in white matter. Features that will then be computed from it, such as anisotropy, will be an average of all the components in the tissue. One needs to note that every tissue is made of several compartments and that the signals from each of these compartments where water molecules are present are averaged. Investigations using diffusion-weighted spectroscopy, an imaging technique with increased cellular specificity, are also led to try and target specific compartment(s) [38, 54]. This average problem leads to a second issue: a small change in anisotropy (or other features) can actually reflect greater pathological differences. It means that there needs to be a big change in the voxel to be able to detect it in the acquired diffusion signal. Third issue is that anisotropic cellular elements might be considered as isotropic due to the tree pattern of processes within grey matter [26] or to crossing fibers in white matter. At least, as expected from an acquisition, the signal is noisy. Low concentration of water molecules in some tissue (and thus long scan times) can lead to a poor signal-to-noise ratio.

In addition to those issues, we must recall that the spacing between axons, axon diameter, myelin thickness, etc are all also variables, even within the same tract, which adds to the complexity of the problem. The barriers to diffusion have also not a simple nor regular geometry. The correspondence between the biological features of the tissue and the non-invasive diffusion measure is therefore not straightforward.

#### **3 Quantifying Anisotropy via Signal Representation**

Signal representation is an indirect method that aims at describing the diffusion signal with no assumptions about the underlying structure. It can therefore be applied to healthy or diseased tissues. Several methods are described, with an emphasis made on the cumulant expansion, which is the most widespread signal representation.

#### *3.1 Cumulant Expansion*

Common signal representations are based on the cumulant expansion [39, 65], which corresponds to a development of the logarithm of the signal in polynomials up to a given order in b:

$$\ln\left(\frac{S(b)}{S\_0}\right) = -bD + \frac{1}{6}(bD)^2K + \dots \tag{5}$$

where D is the diffusion coefficient and K the kurtosis. This formula can also be written in the tensor form:

$$\ln \frac{S(b)}{S\_0} = -bC\_{i\_1i\_2}^{(2)}g\_{i\_1}g\_{i\_2} + b^2C\_{i\_1\dots i\_4}^{(4)}g\_{i\_1}\dots g\_{i\_4} - \dots \tag{6}$$

where *C*(*l*) are the cumulant tensors, and **g** is the direction of the applied diffusion weighting (see Sect. 2.2). Note that Einstein's convention of summation over repeated indices is used here.

An expansion in moments, which corresponds to a Taylor expansion of the signal, is also possible. While expansions in moments and in cumulants are mathematically equivalent, for a similar order truncation at some fixed (low) order, the cumulant series provides a more accurate estimation of the dMRI signal than a moment expansion. Moment expansion is more optimal for analytical treatments because contributions from different tissue compartments add up. A combinatorial relation exists between the two expansions [40, 65]. Computing the cumulant tensors and converting them into moments is promoted to be the most numerically stable methodology to adopt [49].

One of the most popular MRI techniques in brain research as well as in clinical practice is Diffusion Tensor Imaging (DTI) [3], based on the cumulant expansion up to the first order in b. This technique is valid for low diffusion weighting (*b* - (*DK*)−1). Note that this technique does not assume that the medium is homogeneous with unrestricted diffusion (K = 0), which appears to be not true for most biological tissues, but that it follows a Gaussian law when *b* -(*DK*)−1.

Using tensor decomposition, three eigenvalues (λ1, λ<sup>2</sup> and λ3) reflecting axial and radial diffusivity of molecules within fibers and in the extra-cellular space are computed (see Fig. 1). The difference between those two diffusivities enable to define variables such as Mean Diffusivity (MD) (the average of all the eigenvalues) or Fractional Anisotropy (FA), defined as follow:

$$FA = \frac{\text{std}(\lambda)}{\text{rms}(\lambda)} = \sqrt{\frac{1}{2}} \frac{\sqrt{(\lambda\_1 - \lambda\_2)^2 + (\lambda\_2 - \lambda\_3)^2 + (\lambda\_3 - \lambda\_1)^2}}{\sqrt{\lambda\_1^2 + \lambda\_2^2 + \lambda\_3^2}} \tag{7}$$

Those two measures are complementary, as they bring different information to the comprehension of a tissue (Fig. 2). Hofstetter et al. [19] used MD to hypothesize the presence of bigger cells in the brain after a learning session. Beaulieu et al. also investigated anisotropy in the human brain grey matter using DTI [4].

Diffusion Kurtosis Imaging (DKI) goes beyond DTI and its first order expansion by also estimating the kurtosis of the diffusion probability distribution function [25].

**Fig. 1** Isotropic diffusion in somas can be modeled by a sphere (left). Anisotropic diffusion in neurites can be represented by an ellipsoid reflecting axial (λ1) and radial (λ<sup>2</sup> and λ3) diffusion. This image has been inspired by the book chapter written by Christian Beaulieu [4]

**Fig. 2** Mean diffusivity (MD) and fractional anisotropy (FA) are two complementary measures. Here are three examples of ellipses ranging from isotropic to anisotropic that have the same mean diffusivity (0.<sup>7</sup> <sup>×</sup> <sup>10</sup>−<sup>3</sup> mm2s−1). This image has been inspired by the book chapter written by Christian Beaulieu [4]

The kurtosis quantifies the non-Gaussianity of a distribution. The information that it provides is complementary to DTI metrics. Fitting the kurtosis tensor significantly improves the accuracy of the diffusion tensor estimation [64]. In a same way, extending the series to the sixth order cumulant (in *b*3) also increases the accuracy of the kurtosis estimation, albeit with a penalty on precision.

In order to estimate the six independent components of the diffusion tensor, the minimal required data is one b = 0 (unweighted) image and six non-collinear directions on a single diffusion weighting, or "shell". The additional estimation of the 15 independent components of the kurtosis tensor requires a minimal acquisition of one b = 0 image and one or two nonzero shells with 15 non-collinear gradient directions, so that a total of 22 diffusion-weighted images are acquired [64]. The choice of the shell b-values is a trade-off between accuracy and precision. The b-values should be as low as possible to respect the validity of the cumulant expansion, but slightly higher values enable to limit the impact of noise [12]. Jelescu et al. [21] suggest a typical value around b = 1 ms µm−<sup>2</sup> for DTI and 2 ms µm−<sup>2</sup> for DKI in vivo. For further details on the optimization of acquisition parameters for precise measurement of diffusion in anisotropic systems, we invite the reader to have a look at the work of Jones et al. [31].

#### *3.2 Other Representations*

Yablonskiy et al. [66] hypothesize that the acquired diffusion signal is a sum of signals originating from many spin packets, present in different cell types, at different positions. Each spin packet having then different trajectories and facing different hindrances, they make the assumption that they all have a different apparent diffusion coefficient (ADC). Hence, they introduced a distribution of diffusion coefficients ρ(*D*), and expressed the diffusion signal as following:

$$\frac{S}{S\_0} = \int\_0^\infty \rho(D) e^{-bD} dD\tag{8}$$

Theoretically, the distribution of diffusion coefficients can be estimated using the inverse Laplace transform. In practice, some functional form needs to be assumed for ρ(*D*) due to the mathematical ill-conditioning of the inverse Laplace transform. In addition, a very strong diffusion weighted regime is needed for the estimated distribution to accurately reflect the tissue distribution of diffusion coefficients [34, 47].

Jian et al. [29] propose a statistical method to infer connectivity patterns based on the characterization of the water molecule diffusion by a continuous distribution of diffusion tensors. They described the MR signal attenuation as the Laplace transform of this probability distribution defined on the manifold of symmetric positive-definite tensors. Combined with a spherical deconvolution approach, displacement probability functions and distinct fiber orientations can be estimated in each voxel in a HARDI dataset.

The multi-shell Mean Apparent Propagator (MAP)-MRI method, as proposed by Özarslan et al. [50], expands the signal using harmonic oscillator basis functions. It indeed represents the diffusion-weighted signal by an anisotropic Gaussian modulated by a series of Hermite polynomials. This method allows the estimation of three-dimensional EAP, where both restricted (non-Gaussian) diffusion and crossing axons can be represented. However, according to [44], this method fails to estimate crossing angles correctly. The strength of the method resides on its capacity to accurately estimate diffusion properties such as return-to-origin probability, and meansquared displacement. The propagator anisotropy (PA) metric was derived from this method, which is a measure of dissimilarity between the reconstructed EAP and its closest isotropic approximation EAP.

Hanyga et al. [17] proposed a new space-fractional diffusion model based on an anomalous anisotropic diffusion equation that preserves posivity. This method seems well-suited for applications to DTI [42].

Other representations exist, but have not been included in this chapter.

#### *3.3 Limitations*

The validity and therefore the usefulness of the cumulant expansion depends on its convergence towards the acquisition signal, characterized by the convergence radius *bc* [35]. If *b* < *bc*, then the series can be approximated using a couple of low terms in Eq. 6, higher order terms being flooded by the noise, i.e. small contributions to the signal can not be decoupled from the noise in experimental data. The number of parameters to estimate is then reduced, but a good accuracy does not assure its validity. Otherwise, if *b* > *bc*, the series in Eq. 6 diverges which means that the model function cannot be reduced to a polynomial. A good quality fitting gives then more credit to the underlying model.

Hutchinson et al. [20] compare the DTI, DKI, MAP-MRI and NODDI (see Sect. 4.4.1) methods in different experimental conditions to study the influence of noise and sampling (among others) on parameter estimations. All methods proved to be influenced by the acquisition parameters such as the b-values, the resolution, the SNR and the diffusion time. The need of DKI to fit a higher order tensor explains its high sensitivity to noise.

Regional issues are also to be noted, related to crossing fibers, which can be detected as isotropic zones [1]. Indeed, several diffusion directions are possible in that case. The angular resolution needs to be high enough and the model designed to take this particular case into account.

#### **4 Biophysical Modeling to Measure Anisotropy**

This second approach is based on a biophysical model designed for a particular tissue geometry. This model is fit to the diffusion signal acquired, which allows the estimation of the relevant parameters of the microstructure. While it can provide a greater specificity of biological parameters, the design of the model remains difficult, as it needs to accurately capture all the features that effectively and substantially impact the diffusion signal in a given acquisition range (the coarse graining problem, see [46]).

Another big challenge of this approach comes from the number of unknowns to estimate after the definition of all effective parameters. To estimate them all we would need a lot of different b-values. This is unfeasible in clinical applications, because first the gradients used in clinical MRIs are not strong enough, and secondly it would require a patient to stay in the MRI device for a very long time. Some methods rely on constraint to bypass this problem, as presented in Sect. 4.4.

#### *4.1 Multi-compartmental Model*

Tissue in the brain can generally be decomposed into four compartments. The first one corresponds to the somas, which are the central part of the neurons, mainly present in grey matter. Glial cells are also comprised in this compartment, as done by Palombo et al. [54]. However, their possible high exchange rate with the extracellular space (ECS) is still a matter of discussion and this argument would argue in favor of their better modeling in the ECS compartment [15]. Somas can be modeled as spheres of different diameters. Neurites, the second compartment, connect those neurons together, either in short distances in grey matter (they are called dendrites), or long distance connections in white matter (axons). The diffusivity across the processes is considered zero due to the restriction implied by the fixed small diameter. Processes can therefore be modeled by cylinders with zero-radius ("sticks") (see part Sect. 4.2 below). The orientation of a collection of processes within a voxel is characterized by an orientation distribution function (ODF) [60]. The third compartment corresponds to extra-cellular space (ECS) and is modeled as Gaussian anisotropic. The last one is the cerebrospinal fluid (CSF), which could contribute if a voxel contains part of a ventricle, and corresponds to free diffusing molecules. It is hence modeled as free diffusion.

The acquired water signal originates from these four compartments and are weighted according to their relative signal fraction *f* :

$$S(b) = f\_{\text{soma}} \cdot S\_{\text{soma}}(b) + f\_{\text{neurites}} \cdot S\_{\text{neurites}}(b) + f\_{\text{ECS}} \cdot S\_{\text{ECS}}(b) + f\_{\text{CSF}} \cdot S\_{\text{CSF}}(b), \tag{9}$$

with *f*somas + *f*neurites + *f*ECS + *f*CSF = 1. Remark that *f*somas, *f*neurites, *f*ECS and *f*CSF are not the relative volume fractions due to the T2 differences between the compartments. In the following models presented, a combination of those compartments is used to model particular tissues and keep only the relevant compartments. Note that a common assumption is made that the exchanges between the compartments can be neglected at the time scales of clinical dMRI, at least in white matter. The estimation of exchange rate in vivo is challenging and more investigations are needed to validate this hypothesis in white and gray matter.

#### *4.2 Neurites as Sticks*

Neurites have been modeled by zero-radius impermeable cylinders, characterized by their longitudinal diffusivity, the transverse diffusivity being considered zero. These neurites are called "sticks" and correspond to the most anisotropic Gaussian compartment possible [7, 26, 36].

The intra-neurite response function, i.e. the diffusion signal from water inside a stick of diffusivity *Da* pointing in the unit direction **n**ˆ, is defined as:

$$G\_{\hat{\mathbb{H}}}(\hat{\mathbf{g}}, b) = \mathbf{e}^{-bD\_{\mathbf{a}}(\hat{\mathbf{g}} \cdot \hat{\mathbf{n}})^2},\tag{10}$$

with **g**ˆ being the unit gradient direction of the measurement. It is determined by cos θ ≡ **g**ˆ · **n**ˆ, where θ is the angle between **n**ˆ and **g**ˆ.

The signal, after being isotropically averaged over multiple gradient directions **g**ˆ, is the following [8, 24, 30]:

$$
\bar{S} \cong \beta \cdot b^{-1/2},
\tag{11}
$$

with <sup>β</sup> <sup>=</sup> <sup>π</sup> <sup>4</sup> · *f*neurites/(*D*|| *<sup>a</sup>* )<sup>1</sup>/2.

In brain tissue, at sufficiently large b-values, the extra-axonal space signal is exponentially suppressed, its diffusivity being non-zero in any direction. The only remaining signal in white matter comes from the axons (*S*neurites in Eq. 9), and follows the power law from Eq. 11 [41, 62]. This equation captures the very anisotropy of white matter.

Veraart et al. [63] recently proved that the radius of the axons can be estimated for very high b-values, where the transverse diffusivity is not considered null anymore. The direction-averaged DWI signal then follows the following law:

$$
\bar{S} \cong \beta e^{-bD\_a^\perp} \cdot b^{-1/2} \tag{12}
$$

Such law however does not hold in gray matter, which indicates that white matter and grey matter require different models in order to accurately capture their microstructure. Several hypothesis have been elaborated to explain the different behaviour of gray matter DWI signal. McKinnon et al. [41] and Veraart et al. [63] suggest that an increased permeability in cell membranes of neurites in gray matter might be the cause of an increased exponent, while Palombo et al. [52] advocate the abundance of cell body in gray matter. Özarslan et al. [51] suggest that curvy projections, along with longer pulse duration, lead to a disappearance of the *b*−1/<sup>2</sup> decay.

#### *4.3 Standard Model of Diffusion in Neural Tissue*

The measured diffusion signal in brain white matter is a sum of anisotropic compartments. It can be modeled as a convolution between a response kernel K from a perfectly aligned fascicle pointing in the direction **n**ˆ and the fiber orientation distribution (ODF) P(**n**ˆ) normalized to *d***n**ˆP(**n**ˆ) ≡ 1.

$$S\_{\hat{\mathbb{H}}}(b) = \int\_{|\hat{\mathbb{H}}| = 1} \mathcal{P}(\hat{\mathbb{n}}) \mathcal{K}(b, \hat{\mathbf{g}} \cdot \hat{\mathbb{n}}) d\hat{\mathbb{n}},\tag{13}$$

**g**ˆ being defined in Sect. 2.2.

In the case of white matter, the kernel can be written as:

$$\mathcal{K}(b,\xi) = \mathcal{S}\_0 \left[ f \mathbf{e}^{-bD\_{\mathbf{z}}\xi^2} + (1 - f - f\_{\rm CSF}) \mathbf{e}^{-bD\_{\mathbf{z}}^\perp - b\left(D\_{\mathbf{z}}^\parallel - D\_{\mathbf{z}}^\perp\right)\xi^2} + f\_{\rm CSF} \mathbf{e}^{-bD\_{\rm CF}} \right],\tag{14}$$

with ξ = **g**ˆ · **n**ˆ. Those exponential contributions correspond to the intra-axonal space modeled by a stick compartment (Eq. 10), the extra-axonal space modeled by an axially symmetric Gaussian compartment with transverse and longitudinal diffusivities *D*<sup>⊥</sup> *<sup>e</sup>* and *D*|| *<sup>e</sup>* , and the cerebrospinal fluid (CSF) compartment. All those compartments are represented in Fig. 3. This decomposition has been widely used in white matter

**Fig. 3** Standard Model of diffusion in neuronal tissue. Two-compartment model (intra- and extraneurite spaces) described by 4 independent parameters: *f* , *Da*, *D <sup>e</sup>* and *D*<sup>⊥</sup> *<sup>e</sup>* and a fiber orientation distribution P(*n*ˆ). This figure is reproduced from Novikov et al. [49]

by the community. As a consequence, Novikov et al. suggested to call it the Standard Model (SM) [46]. For the sake of reference, we will also refer to it as the SM in this chapter.

#### *4.4 Standard Model Parameter Estimation Using Constraints*

In the previous sections we presented the SM of diffusion in neural tissue as a sum of anisotropic Gaussian compartments, as defined by Novikov et al. [46]. We will now introduce some methods based on the SM that rely on constraints to overcome the challenge of estimating many biological parameters of interest.

#### **4.4.1 Neurite Orientation Dispersion and Density Imaging**

In order to reduce the number of parameters that need to be estimated, Zhang et al. [67] proposed to impose restrictions on the intrinsic diffusivities. They introduced a method called Neurite Orientation Dispersion and Density Imaging (NODDI), which relies on a three-compartment SM (intra-axonal space, extra-axonal space and CSF), described by seven parameters: volume fractions *fintra* and *fiso*, diffusivities *D*|| *a* , *D*|| *<sup>e</sup>* , *D*<sup>⊥</sup> *<sup>e</sup>* and *Diso*, and the orientation dispersion modeled by a Watson distribution of concentration parameter κ. By fixing the diffusivities to the following values:

$$D\_a^\parallel = D\_e^\parallel = 1.7 \,\mu\text{m}^2/\text{ms} \tag{15}$$

$$D\_e^\perp = (1 - f\_{\text{intra}}) \cdot D\_e^\parallel \tag{16}$$

$$D\_{iso} = 3 \,\mu m^2/\text{ms} \tag{17}$$

only the two volume fractions and the orientation dispersion need to be estimated.

Although this method allows to estimate the parameters, the validity of those constraints need to be questioned. To begin with, if we admit that the equalities are correct, they imply that a small deviation from the fixed values, as occurs in cerebral ischemia, will induce a non-negligible bias in the other parameters estimation, leading to false interpretations. However, studies using Diffusion weighted spectroscopy MR which can quantify the diffusion of specific metabolites (e.g. [55]), suggest, through the study of metabolites specifically found on different sections of the neurons and extra cellular tissue, that such diffusivity is not constant across the whole brain. Whether and how these findings can be used to shed light on water diffusion in the brain, is an open question.

Another drawback of this method is that it leads to indetermination, which means that NODDI returns one possible result among a multiplicity of mathematical solutions by fixing *D <sup>a</sup>* = *D <sup>e</sup>* [23, 49]. If we consider the case where all the parameter constraints are released and the CSF compartment neglected (called NODDIDA [22], which stands for NODDI with diffusivity assessment), two distinct solutions to the estimation problem exist: *D <sup>a</sup>* > *D <sup>e</sup>* and *D <sup>a</sup>* < *D <sup>e</sup>* (see Figs. 8 and 9 in Jelescu et al. [23]). Both solutions lie within biologically plausible ranges, and determining which solution is biologically correct is an active field of research, although most studies are suggesting *D <sup>a</sup>* > *D <sup>e</sup>* . At least, the tortuosity approximation that relates *D*<sup>⊥</sup> *<sup>e</sup>* and *D <sup>e</sup>* has been invalidated in the case of tight packings of axons [45].

#### **4.4.2 White Matter Tract Integrity Metrics**

Another approach to estimate the relevant features of interest in a tissue proposes to relate the scalar parameters to the DKI components. Called White Matter Tract Integrity (WMTI) [16], it is a two-compartment SM that relies on the assumption that sticks are highly aligned within a voxel.

The tissue is described as a sum of two Gaussian compartments (intra- and extraaxonal space, Eq. 9 with *f*somas = 0 and *f*CSF = 0), where axons are modeled as sticks embedded in a Gaussian anisotropic extra-axonal medium. Each compartment is characterized by a tensor (*Da* and *De*) derived from the kurtosis tensors *D* and *K*. In any direction j:

$$D\_j = f\_{\text{intra}} \, D\_{\text{a,j}} + (1 - f\_{\text{intra}}) \, D\_{\text{e,j}},\tag{18}$$

$$K\_j = \Im f\_{\rm intra} \cdot (1 - f\_{\rm intra}) \frac{\left(D\_{\rm e,j} - D\_{\rm a,j}\right)^2}{D\_j^2} \tag{19}$$

We retrieve the two possible mathematical solutions mentioned before, as demonstrated by the square in Eq. 19. The solution chosen in this method is *D <sup>a</sup>* < *D e* , which leads to:

$$f\_{\text{intra}} = \frac{K\_{\text{max}}}{K\_{\text{max}} + 3},\tag{20}$$

$$D\_{\rm e,j} = D\_{\rm j} \left[ 1 + \sqrt{\frac{K\_j \cdot f\_{\rm intra}}{3 \left( 1 - f\_{\rm intra} \right)}} \right],\tag{21}$$

$$D\_{\rm a,j} = D\_{\rm j} \left[ 1 - \sqrt{\frac{K\_{\rm j} \left( 1 - f\_{\rm intra} \right)}{\Im f\_{\rm intra}}} \right]. \tag{22}$$

Although WMTI enables to capture the changes of diffusivities, it has two main limitations. First, this approach is limited to regions of highly aligned single fiber bundles, which are only present in some white matter regions. Jespersen et al. suggested a method that alleviates this assumption by assuming a Watson distribution of the axons (like in NODDI) [27]. Second, as it relies on the DKI decomposition, this method is only restricted to the low b-value regime, which could lead to some bias.

#### *4.5 Lemonade*

As explained in Sect. 4, estimating both compartment diffusivities and orientation dispersion of neurites simultaneously is problematic and tends to be biased. Some methods suggest fixing some parameters such as NODDI or to limit its application to coherent fibers only as WMTI to work around these problems. Releasing these constraints necessitates to estimate a larger number of parameters.

A very recent method in white matter estimates the scalar parameters of a twocompartment kernel separately from the ODF without any constraints. The method developed by Novikov et al. [49] is based on the modeling of the diffusion signal as a convolution of the ODF and the response kernel from a perfectly aligned fiber segment, as presented in Sect. 4.3. It can be decomposed into two steps. A first step solves an algebraic system of equations that relates the kernel parameters to the signal moments for low b-values. This part was called LEMONADE, which stands for Linearly Estimated Moments provide Orientations of Neurites And their Diffusivities Exactly. It requires at least 3 non-zero b-shells inferior to 2.5 ms µm−<sup>2</sup> and returns estimates for *fintra*, *D <sup>a</sup>* , *D <sup>e</sup>* , *D*<sup>⊥</sup> *<sup>e</sup>* and *<sup>p</sup>*<sup>2</sup> <sup>=</sup> <sup>3</sup>(cos ψ)<sup>2</sup> −<sup>1</sup> <sup>2</sup> , which gives an estimate of the orientation dispersion. In a second step, a rotationally invariant energy function of the system is minimized exploiting all available data and using the first estimates as initialization values.

This method emphasizes the existence of the two mathematical solutions as introduced before, and shows that, in principle, the degeneracy can be avoided using measurements up to the 3rd order of b-values. However, due to noise in the data, the solution selection remains difficult in practice and individual validation should be carried out.

The assumptions made in this approach are, as in the other methods previously presented, the existence of only two compartments, the uniformity of diffusivities across all axons in the voxel, and axial symmetry of the kernel. These assumptions are also the limitations of the model used. Validation in the case of pathological tissue also needs to be investigated.

We refer the reader to Jelescu and Budde's review on the accuracy and validation of biophysical parameters of different diffusion models in white matter, which includes the ones presented before [21].

#### **5 Summary and Above**

We have shown two main approaches to describe microstructure anisotropy using diffusion MRI: signal representation and biophysical modeling. While the former are general and make no assumptions about the underlying tissue, models are designed for a particular tissue and therefore provide greater specificity and interpretation of the estimated biological parameters. The difficulties in modeling reside in accurately capturing the features that effectively and substantially impact the diffusion signal in a given acquisition range, and being able to correctly fit the model (inverse problem).

Anisotropy provides great insight into a tissue structure, and its evolution can enlighten the progression of certain pathologies. The presence of isotropy must not be neglected either, as it can be a great marker of other microstructures, such as in grey matter where it denotes the presence of somas.

Although great progresses have been made during the last decade, some questions remain unresolved. To cite a few, we can wonder to which extent we can consider compartments as non-exchanging. Diffusion time, brain region and myelination of the tissue will most likely impact the answer of this question. Can we also come up with methods less sensitive to the signal-to-noise ratio or a way to disentangle thermal noise and artifacts from the signal of interest?

**Acknowledgments** This work was supported by the ERC-StG NeuroLang and the ANR/NSF NeuroRef grants.

#### **References**


Estimation of Diffusion in Neural Tissue. NMR Biomed. **29**(1), 33–47 (2016). https://doi.org/ 10.1002/nbm.3450


for non-monoexponential attenuation: modeling diffusion of brain metabolites in vivo up to very high diffusion weighting. Magn. Reson. Med. **77**(1), 343–350 (2017). https://doi.org/10. 1002/mrm.26548


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Conceptual Parallels Between Stochastic Geometry and Diffusion-Weighted MRI**

**Tom Dela Haije and Aasa Feragen**

**Abstract** Diffusion-weighted magnetic resonance imaging (MRI) is sensitive to ensemble-averaged molecular displacements, which provide valuable information on e.g. structural anisotropy in brain tissue. However, a concrete interpretation of diffusion-weighted MRI data in terms of physiological or structural parameters turns out to be extremely challenging. One of the main reasons for this is the multi-scale nature of the diffusion-weighted signal, as it is sensitive to the microscopic motion of particles averaged over macroscopic volumes. In order to analyze the geometrical patterns that occur in (diffusion-weighted measurements of) biological tissue and many other structures, we may invoke tools from the field of stochastic geometry. Stochastic geometry describes statistical methods and models that apply to random geometrical patterns of which we may only know the distribution. Despite its many uses in geology, astronomy, telecommunications, etc., its application in diffusionweighted MRI has so far remained limited. In this work we review some fundamental results in the field of diffusion-weighted MRI from a stochastic geometrical perspective, and discuss briefly for which other questions stochastic geometry may prove useful. The observations presented in this paper are partly inspired by the Workshop on Diffusion MRI and Stochastic Geometry held at Sandbjerg Estate (Denmark) in 2019, which aimed to foster communication and collaboration between the two fields of research.

T. Dela Haije (B)

A. Feragen Department of Applied Mathematics and Computer Science, Technical University of Denmark, Lyngby, Denmark e-mail: afhar@dtu.dk

Department of Computer Science, University of Copenhagen, Copenhagen, Denmark e-mail: tom@di.ku.dk

#### **1 Introduction**

*Diffusion-weighted magnetic resonance imaging* (MRI) [22] is one of the few imaging modalities that is capable of mapping the immensely complex *micro-structural* architecture of the human brain in a non-destructive manner. This is achieved by applying a specific sequence of diffusion-sensitizing magnetic field gradients during the MRI acquisition, producing a signal with a decay rate that is dependent on the relative mobility of water molecules in the tissue [30]. As the overall mobility of the molecules is decreased by the presence of any material barrier, the acquired signal effectively provides an indirect probe of the ambient structure. At the micrometer length scales accessible in current scanners, the dominant barriers to the diffusing molecules in the brain are the fiber-like *neurites* that transmit information between different regions of the brain [3], and whose properties are (naturally) of profound importance in neurology. Aside from the anticipated and present clinical value of this modality, diffusion-weighted MRI also stands to provide unique information about the evolution, morphogenesis, and function of the brain, already being pursued through the tracking of macroscopic fibers in *tractography* [2, 4, 20]. The challenge faced by the diffusion-weighted MRI community is to identify the relevant structural parameters determining the signal decay, and—to the extent that this is possible—to invert the relation between them. This is where we believe *stochastic geometry* could play a role.

Stochastic geometry [29] is an area of statistics that provides modeling and inference techniques for complex spatial objects whose structure can be described effectively as random patterns. One of the first cases studied in what is now called stochastic geometry—the problem of *stereology* [1]—included the inference of geometric properties of 3D objects from their intersections with a small number of 2D planes. This topic gained traction in the 1960s, as its solutions alleviated the challenging and computationally demanding task of actually computing 3D reconstructions. Stereology found a wealth of applications ranging from geology to for example microscopy of neuroanatomy. Since then stochastic geometry has evolved into a mature field of mathematics that offers a rich toolbox of rigorously developed techniques, including models with potential relevance for diffusion-weighted MRI such as random tessellations and *fiber processes*. Skimming the stochastic geometry literature one quickly comes across a number of concepts that have obvious analogues independently developed in the diffusion-weighted MRI literature, and yet the two research domains have had very limited contact so far.

This chapter is written from the perspective of diffusion-weighted MRI as a field working on a set of challenging modeling problems, focusing on the potential utility of stochastic geometry in addressing them. Our aim is not to provide an exhaustive overview of relevant theory in either field, but simply to highlight some well-known results where the conceptual links between the two become apparent. A comprehensive introduction to stochastic geometry can be found in the book by Stoyan et al. [29], while the books by Jones [16] and by Johansen-Berg and Behrens [15] provide a good entryway to diffusion-weighted MRI.

To initiate an exchange of ideas between these two fields, the first *Workshop on Diffusion MRI and Stochastic Geometry* was co-organized by the authors and Eva B. Vedel Jensen at Sandbjerg Estate (Denmark, January 20–24, 2019). The observations presented in this chapter partly inspired this workshop, but the chapter likewise builds on the valuable discussions held at the lively and interactive workshop. We are not aware of any previous works pointing out the parallels described here, although some of them will undoubtedly have been noticed before.

#### *Overview*

The molecular dynamics relevant for diffusion-weighted MRI are conveniently described by a displacement probability density function—the *diffusion propagator P*(*r*, *t*). The propagator *P*(*r*, *t*) represents the probability of a displacement *r* at a *diffusion time t*, and the diffusion-weighted signal is related to the characteristic function *S* of the propagator given by

$$S(q) = \int \mathbf{e}^{-iqr} P(r, t) \,\mathrm{d}r,\tag{1}$$

where *qr* denotes the inner product between *q* and *r*. The Fourier parameter *q* can be considered an experimental parameter, determined in practice by the diffusionsensitizing gradients of the acquisition.

In the following sections we give examples of connections to stochastic geometry for three different 'limiting regimes' of the diffusion-weighted signal, where expressions for the relevant molecular dynamics can be simplified significantly. Sections 2 and 3, which feature the short and long diffusion time limits respectively, can be well-understood in terms of the time-dependent *diffusion coefficient D*(*t*), which represents the mean squared molecular displacement at a time *t*. This diffusion coefficient appears as the first non-trivial coefficient in the Taylor series of log *S*, the cumulant expansion.<sup>1</sup>

$$\log S(q) \cong -D(t) \, q^2 + \dots \tag{2}$$

From this expansion it can already be gathered that results based on the diffusion coefficient are mostly applicable when *q* is relatively small—i.e., when the gradients are weak enough for the *D*-dependent term in this expansion to dominate—and in Sect. 4 we consider instead the limit where the gradient strength parameter ∼ *q* becomes large. In this regime the diffusion coefficient no longer provides an adequate vehicle for the description and analysis of the diffusion process, and we have to rely on other descriptors. Where necessary, additional details on these concepts will be given in the text, although technicalities will be skipped in favor of accessibility. To simplify the exposition further, we will make implicit use of the narrow

<sup>1</sup>We restrict ourselves in Eq. (2) to the one-dimensional case, to keep technicalities to a minimum.

pulse approximation [6, 27] throughout. The paper concludes with an outlook on the possible future uses of stochastic geometry in Sect. 5.

#### **2 Specific Volumes and the Short-Time Limit**

…the time-dependent diffusion coefficient *D*(*t*) of mobile molecules confined in pores or cells carries information about the confining geometry. At early times, [a perturbative expansion of *D*(*t*)] gives, irrespective of details, the pore surface to volume ratio …, which is a measure of microscopic length.

Sen (2004)

The first regime we consider is the *short-time limit*, described in the seminal works by Mitra, Sen, and others [18, 19, 26], and with important insights dating back to Kac [17]. The situation analyzed in these works is the diffusion of molecules in the neighborhood of obstructive geometrical features, Fig. 1. At the boundaries between the diffusive medium and the geometry, the diffusion is prescribed by boundary conditions that can depend on e.g. the *permeability*, leaving the spatially averaged, time-dependent diffusion coefficient *D*(*t*) to be solved. While this is a very challenging problem for any non-zero, finite time *t*, the limiting behavior for *t* → 0 can be expressed in terms of a relatively small set of practically useful structural parameters. As *D*(*t*) can be measured in the scanner, we can use diffusion-weighted measure-

**Fig. 1** A schematic showing a particle (black solid line) diffusing in a volume *V*, marked by the blue hatch pattern. The gray boundary *S* acts as a barrier to the diffusion, resulting in a decreased time-dependent diffusion coefficient *D*(*t*) compared to the free diffusion coefficient *D*0. At short times only particles in a small neighborhood of the boundary (within a distance <sup>∼</sup> <sup>√</sup>*D*<sup>0</sup> *<sup>t</sup>*) are significantly impacted by this effect, resulting in a characteristic decrease of *D*(*t*) proportional to the ratio between the surface area *S* and the volume *V*. Note that the time-dependent diffusion coefficient measured in diffusion-weighted MRI is acquired by averaging over volumes much larger than the typical diffusion length <sup>√</sup>*D*<sup>0</sup> *<sup>t</sup>*

ments to obtain estimates of these parameters. The emergence of these structural parameters in the diffusion-weighted signal can be understood as follows.

In the limit *t* → 0 the diffusing molecules do not have enough time to interact with the geometry, and the diffusion coefficient naturally approaches the medium's free diffusion coefficient *D*(0) = *D*0—the diffusion coefficient of the medium in the absence of any geometry. At times close to the limit, the particles move a typical distance in the order of <sup>√</sup>*D*<sup>0</sup> *<sup>t</sup>* (by virtue of the definition of *<sup>D</sup>* in terms of the mean squared displacement), and so roughly speaking only the fraction of particles within some distance <sup>∼</sup> <sup>√</sup>*D*<sup>0</sup> *<sup>t</sup>* can 'see' the geometry. As the geometry essentially acts as a barrier to the diffusion, the time-dependent diffusion coefficient decreases from its free diffusion limit, and this decrease is more significant if a larger fraction of the total number of particles can interact with the boundaries. For diffusion times approaching 0, this fraction becomes exactly proportional to the surface area of the geometry. Formalizing this notion, Mitra et al. [19] then showed that the first order correction describing the approach of the limit becomes proportional to the *surfaceto-volume ratio S*/*V* of a smooth geometry, cf. Fig. 1, according to

$$D(t) \sim D\_0 - \frac{4}{3\,d\,\sqrt{\pi}} \frac{S}{V} \sqrt{D\_0 t} \, D\_0 \quad (t \to 0), \tag{3}$$

where *d* is the spatial dimension. From this relation, the surface-to-volume ratio can be estimated reliably from the diffusion-weighted measurements [13]. The order *t* terms in this expansion depend to varying degrees on the boundary condition parameters such as the permeability, as well as on the average curvature of the geometry, while higher order terms depend on even more intricate details of the environment.

Although the complex interactions between particles undergoing random displacements and their surrounding structures have not been considered in stochastic geometry as such—this problem is closer to mathematical physics [5]—the surfaceto-volume ratio uncovered in Eq. (3) is a commonly estimated quantity in for example stereology. The surface-to-volume ratio is also referred to as the *surface density* or as the *specific surface area* in stereology and stochastic geometry. An example of a question in this scenario for which stochastic geometry could be helpful is: "what is the smallest observation window (voxel) that has an acceptable error when estimating a given characteristic?" [29, Sect. 6.4.6].

#### **3 Stationarity and the Long-Time Limit**

That is, we assume here that the voxel is *statistically homogeneous*. This macroscopic uniformity allows us to go from averaging over the contributions from all parts of the system … to ensemble averaging over all disorder realizations, leading to the description of the signal in terms of the statistical properties embodied in the correlators …

In the short-time limit of the previous section, particles could not explore their surroundings outside a vanishingly small window, leading to the simple relation between the surface-to-volume ratio and the time-dependent diffusion coefficient. In the *longtime limit*, on the other hand, we can assume that particles in fact see *all* the surrounding structures. Under fairly general conditions, the diffusion process in this limit can still be described in terms of a single diffusion coefficient—the limiting coefficient *D*<sup>∞</sup> = lim*<sup>t</sup>*→∞ *D*(*t*). While determining *D*<sup>∞</sup> for a given sample is very difficult, Novikov et al. [21, 23, 24] showed that once again there are basic geometrical properties of the structure that completely determine the limiting behavior of *D*(*t*). In this case, the diffusion coefficient approaches its limit as

$$D(t) \sim D\_{\infty} + c \, t^{-\vartheta} \quad (t \to \infty), \tag{4}$$

where *c* is a constant, and where the exponent ϑ = (*p* + *d*)/2 is determined by the spatial dimension *d* and a *structural exponent p*. The structural exponent *p* is the exponent of the limiting power law behavior of the structure correlation function, which roughly speaking quantifies the presence of long-range correlations in the diffusion barriers. While the constant *c* varies significantly between different configurations of barriers, we know of only a few possible values for the exponent *p*, with many vastly different configurations of barriers producing the same critical exponent in experiments. The structural exponent *p* effectively distinguishes between different *structural universality classes* [23].

A central concept in the cited works by Novikov et al. is that *because* the diffusing particles can explore their entire surroundings, the exact configuration of the barriers becomes irrelevant. Consequently, one can theoretically replace the complicated original structure with a simpler-to-analyze *effective medium*, permitting the derivation of e.g. Eq. (4). As explained in the quote above, the practical application of this simplifying methodology requires that we assume that the subsets of a sample explored by different particles can be viewed as random samples ('disorder realizations') generated by some basic statistical properties of the global structure. In the context of stochastic geometry, this same assumption is more commonly called (spatial) *stationarity*, and it is a key assumption in many classical proofs in the field. In particular, the assumption of stationarity enabled the first proofs of the *fundamental formulas of stereology*. In a number of stochastic geometry results it is now known that a weaker 'first order stationarity' assumption is sufficient [1], and it might be interesting to see if the same is possible for the long-time limit results discussed here.

#### **4 Directional Measures and the Strong-Gradient Limit**

The concept of disordered media and statistical averaging can be particularly valuable to deal with the geometric complexity of biological tissues. We believe that further progress in the field can be achieved by merging microscopic geometric models, statistical [descriptions] of disordered media, and high-gradient features of the signal formation.

The gradient strength parameter set during a diffusion-weighted MRI experiment determines—roughly speaking—the scanner's sensitivity to diffusive motion. The mobility of the water molecules in the sample affects the decay rate of the diffusionweighted signal, cf. Sect. 1, and stronger gradients make this effect correspondingly stronger. Although stronger gradients thus produce weaker, harder to detect signals, these signals contain information about interesting features of the diffusion that cannot be observed at lower gradient strengths. The results for the short-time and long-time limits described in the previous subsections are mainly used at low to moderate gradient strength acquisitions.

When we increase the gradient strength in a diffusion-weighted MRI experiment we first notice that the *anisotropy*, i.e., the orientation-dependence, becomes more significant. While we omitted this before, anisotropy is already a factor at the lower gradient strength experiments used for the concepts discussed in Sects. 2 and 3. In the general case, the long-time limit *D*<sup>∞</sup> of *D*(*t*) considered in Eq. (3) is, for example, in fact a *tensorial quantity* related to the mean squared molecular displacements along different orientations. This anisotropy in the diffusion reflects the anisotropy in the sample's micro-structure, so for example in the brain it is predominantly determined by the orientation distribution of neurites, cf. Sect. 1. At higher gradient strengths, the orientation-dependence of the diffusion can no longer be represented by a simple tensor, and a more complete set of orientational features becomes accessible.

The neurite orientations are generally characterized by a so-called *fiber orientation distribution function* (ODF), which specifies the likelihood that a neurite is locally tangent to a given orientation, and the estimation of this object from strong-gradient experiments is a common problem considered in diffusion-weighted MRI. A large number of techniques have been proposed to deal with this question [14, 31, 32], but we will not discuss them in detail here. Instead we will point out that the ODF also has a stochastic geometrical analogue: the *rose of directions* [29]. The rose of directions is defined for *fiber processes*—a well-defined (*locally finite*) collection of randomly placed fibers. The rose of directions is the distribution of the direction tangent to a typical point on a fiber, where the meaning of a 'typical point' can be made explicit using *Palm distribution theory*. A stationary fiber process is an example of a stochastic geometrical model in the spirit of the work by Novikov et al. [21] that could be useful to model biological tissues. Furthermore, the rose of directions has a dual—the *rose of intersections*. These two objects are related through the *Funk–Radon* integral transform [9], which also appears naturally in for example the diffusion-weighted signal expression for narrow cylinders (the 'fiber ball' ODF) [14].

At even stronger gradients we move to another limiting regime—the stronggradient limit, or *localization regime* [11]. Here we can still make use of the intuition developed in Sect. 2: any structure in the sample will act as a barrier for particles in its vicinity, and thus slow down the average motion in that region. Slower diffusion in turn implies less signal decay, and so the main contributions to the signal now come from particles *localized* near the barriers [28]. The localization effect carries non-trivial consequences for the diffusion time-dependence of the signal, but for this chapter we are more interested in the orientation dependence in this regime.

A simple way to understand the impact of the geometry's anisotropy on the signal in the strong-gradient limit, is to look at a diffusion propagator with a *compact support* -. At strong gradients the integrand in Eq. (1) oscillates rapidly, and as a result the integral indeed tends to 0. However, the rate at which it decays is in fact governed by the support -, which is of course determined by the geometry. We can consider as an illustration the 'indicator' propagator which is 1 everywhere inside a sufficiently smooth and convex -, and 0 elsewhere. The signal decay is then exponential with an exponent sup*<sup>r</sup>*∈ *qr* [12], where the exponent, viewed as a function in *q*, determines completely. We could for example use this observation to recover the shape of a convex pore enveloping a diffusing substance, provided that the pore is much smaller than the typical diffusion length <sup>√</sup>*D*<sup>0</sup> *<sup>t</sup>* [6], but this approach is not limited to closed pores.

The function *q* → sup*<sup>r</sup>*∈ *qr* is called the *support function* of in stochastic geometry, where it can be associated with the rose of directions of a fiber process [29]. A similar interpretation can be given to the support function as it occurs here, leading to the definition of the *barrier ODF* [7, 8]. It must be mentioned that as these developments are very recent, their practical utility is currently still being investigated.

#### **5 Perspectives**

We have given three examples of concepts in diffusion-weighted MRI that also occur in stochastic geometry: the surface-to-volume ratio, which is related to the specific surface area commonly estimated in stereology; statistical homogeneity, which is related to stationarity; and the fiber orientation distribution function, whose stochastic geometry analogue is the rose of directions. In the context of neuroimaging, the common thread between them appears to be the concept of stationary fiber and surface processes from stochastic geometry, which are completely characterized by the specific length/area and the rose of directions.

We believe further investigations in this direction could lead to new and useful methods for the analysis of diffusion-weighted MRI data, as well as novel research problems in stochastic geometry. In particular, we hope that the link to the mature statistical theory found in stochastic geometry may offer practical tools for the analysis of uncertainty and variation in applications of diffusion weighted MRI: How much faith can we put in estimated anatomical quantities? Which quantities can we expect to be able to derive from given data? What are the conditions we should put on our data acquisition in order to be able to draw sound conclusions for the hypotheses that we ultimately hope to investigate?

**Acknowledgments** The authors thank the Villum Foundation for funding this work through a block stipendium and a Villum Experiment grant, as well as for funding the Centre for Stochastic Geometry and Advanced Bioimaging (CSGB) which fostered 10 years of interactions between the authors and experts in stochastic geometry, including the 2019 Workshop on Diffusion MRI and Stochastic Geometry. In particular, the authors wish to thank Eva B. Vedel Jensen and Markus Kiderlen, for their engagement, input, and encouragement in our ideas for collaboration, as well as their kind patience with us when beginning to engage with stochastic geometry. We are also indebted to Andrea Fuster and Luc Florack for their contributions to the PhD project of Tom Dela Haije, which included the introduction of the barrier ODF. The authors would like to acknowledge Schloss Dagstuhl and the organizers and participants of Dagstuhl seminars 16142 and 18442 for facilitating discussions that further supported developing the ideas discussed in this work. Finally, the authors wish to thank all the participants in the 2019 Workshop on Diffusion MRI and Stochastic Geometry.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Magnetic Resonance Assessment of Effective Confinement Anisotropy with Orientationally-Averaged Single and Double Diffusion Encoding**

#### **Cem Yolcu, Magnus Herberthson, Carl-Fredrik Westin, and Evren Özarslan**

**Abstract** Porous or biological materials comprise a multitude of micro-domains containing water. Diffusion-weighted magnetic resonance measurements are sensitive to the anisotropy of the thermal motion of such water. This anisotropy can be due to the domain shape, as well as the (lack of) dispersion in their orientations. Averaging over measurements that span all orientations is a trick to suppress the latter, thereby untangling it from the influence of the domains' anisotropy on the signal. Here, we consider domains whose anisotropy is modeled as being the result of a Hookean (spring) force, which has the advantage of having a Gaussian diffusion propagator while still confining the spatial range for the diffusing particles. In fact, this confinement model is the effective model of restricted diffusion when diffusion is encoded via gradients of long durations, making the model relevant to a broad range of studies aiming to characterize porous media with microscopic subdomains. In this study, analytical expressions for the powder-averaged signal under this assumption are given for so-called single and double diffusion encoding schemes, which sensitize the MR signal to the diffusive displacement of particles in, respectively, one or two consecutive time intervals. The signal for one-dimensional diffusion is shown to exhibit power-law dependence on the gradient strength while its coefficient bears signatures of restricted diffusion.

C. Yolcu (B) · E. Özarslan

E. Özarslan e-mail: evren.ozarslan@liu.se

M. Herberthson Department of Mathematics, Linköping University, Linköping, Sweden e-mail: magnus.herberthson@liu.se

C.-F. Westin Department of Radiology, Brigham & Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA e-mail: westin@bwh.harvard.edu

Department of Biomedical Engineering, Linköping University, Linköping, Sweden e-mail: cem.yolcu@liu.se

E. Özarslan Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden

#### **1 Introduction**

Magnetic resonance has proved to be an extremely effective tool to peer into materials and tissues noninvasively. It manipulates the magnetic orientation of molecules pervading the material into a natural precession which emits radio waves. By imposing a spatially-varying magnetic field (hence frequency of precession), the emitted radio frequency signal is made to encode in its spectrum the coordinates of the molecules that emit it. This way, the signal can be spectrally decomposed to trace back what proportion of it originates from where, that is, from which 'voxel.' Commonly achieved voxel sizes are in the neighborhood of a millimeter.

Another use of a spatially-varying precession rate involves sensitizing the signal to the motion of the molecules; diffusion in particular. When the molecules trace out random (Brownian) paths where different locations they visit impart different precessional angles on them, their precessions lose coherence, attenuating the sum of their emitted radio waves. This attenuation of MR signal, specifically its response to the direction of the gradient in precession rate, can reveal or quantify how mobile the molecules are along different directions. The difference of mobility can arise from pore boundaries, impurities, cell membranes, etc. Hence the diffusion-attenuated signal encodes the influence of structures that can be significantly smaller than voxel dimensions. Tracing out axon bundles in human brain white matter is for instance a widely employed application of this principle. This modality of magnetic resonance imaging, which we refer to as diffusion MR, is the subject of this contribution.

While the anisotropy (i.e., variance under a rotation transformation) of a single pore or a cell may be easily visualized, one would be mistaken to make a one-toone connection with that and the anisotropy of the signal (i.e., the response of the signal to orientations of the specimen or the apparatus). For instance, the signal of a voxel consisting of an unaligned mixture of cylindrical aqueous compartments will be less sensitive to rotations than one consisting of an aligned bundle of cylinders. The anisotropy of the individual compartments is common in the two examples, but the aligned case has more ensemble anisotropy.

In some sense, then, ensemble anisotropy confounds compartment anisotropy at the signal level, and eliminating it pronounces features at the subvoxel level. One way to achieve this is to take an average of the signal over all orientations. If the material allows it, it can be ground into a powder to that effect; hence the term *powder* averaging. However this is generally impossible in bio-medical applications. Then, repeated applications of a measurement protocol in different orientations is the avenue to follow.

In this contribution, we are concerned with two particular diffusion MR schemes, single and double diffusion encoding (SDE and DDE), orientationally averaged in the aforementioned fashion to eliminate ensemble anisotropy. The single encoding scheme [44] is the bread and butter of most applications, employing a magnetic field gradient that remains on for a specified duration in one direction, and then the opposite direction after a specified delay (Fig. 1). The signal then encodes the probability of Brownian displacement between the application of the two pulses.

**Fig. 1** The pulse sequence for single diffusion encoding, with time running along the horizontal direction

**Fig. 2** The pulse sequence for double diffusion encoding

SDE has been the encoding scheme employed since some of the earliest studies of the powder averaged signal [5]. However, very recent theoretical [13] as well as experimental [1] studies have considered the orientationally-averaged signal for more general gradient waveforms.

The double diffusion encoding (DDE) scheme employs two single-encoding blocks in succession [9], in different directions in general, before the signal is read (Fig. 2). This method has been studied extensively in recent years mostly because it allows the anisotropy of the microdomains to be quantified [8], which has important implications for medical imaging. The reader is referred to the reviews [4, 11, 27, 40, 42] for in-depth presentation of the method. In a nutshell, the DDE technique encodes into the signal the joint probability of two Brownian displacements taking place between the two pulses of each block. For freely diffusing molecules, the two displacements are uncorrelated [36]. However, when restrictions, and arguably inhomogeneities and forces, are present, this is no longer the case [24], hence imparting signatures of compartment size onto the signal. As mentioned above, DDE employing gradient blocks in different directions is sensitive to anisotropy of subdomains inside the voxel [6, 8], but not independently of ensemble anisotropy [29].

While we do not consider restricted diffusion in the strict sense of restriction by hard walls in this article, we do respect the finite range of motion of the molecules by the aid of a harmonic attractive force [7, 20, 25, 46, 49]. This is an *effective* model which can mimic restrictive walls with reasonably tractable mathematics, and is actually approached when the gradient pulses are long [34].

This chapter is organized as follows. We first present the signal arising from an effectively confined domain under double diffusion encoding. Afterwards we derive analytical expressions for its powder average and consider the "stick" geometry, wherein diffusion is fully confined in two dimensions. We then treat the case of single diffusion encoding as an extreme case of double encoding and give its corresponding powder average expressions. The special cases of "stick" and "pancake" geometry are compared to their counterparts for free diffusion. The article is concluded after discussions based on the results of the previous sections.

#### **2 Double Diffusion Encoding at the Compartment Level**

Here we derive the double-encoded diffusion MR signal arising from a compartment which is characterized by an effective spring force attracting the molecules toward the center. Such a force mimics the confining effect of walls, membranes, etc., with minimal mathematical burden.

The spring (Hookean) force influences the motion of the molecules via a quadratic potential<sup>1</sup>

$$V(\mathbf{x}) = \frac{1}{2}\mathbf{x}^{\mathsf{T}}\mathbf{C}\mathbf{x}\,,\tag{1}$$

defining the *confinement tensor* **C**, which can be taken to have **C**- = **C** without loss of generality. However, in order for the steady state molecule number density *p*st(*x*) ∝ e−*V*(*x*) to be normalizable, **C** must be positive (semi)definite.2 Under this (or any) potential, the magnetization density ρ(*x*, *t*) evolves according to

$$\partial\_t \rho(\mathbf{x}, t) = (\mathbf{D} \nabla) \cdot \mathbf{e}^{-V(\mathbf{x})} \nabla \mathbf{e}^{V(\mathbf{x})} \rho(\mathbf{x}, t) - \mathbf{i} \mathbf{g}(t) \cdot \mathbf{x} \rho(\mathbf{x}, t) \,. \tag{2}$$

Here, we have assumed that diffusion is governed by a spatially-uniform, possibly anisotropic diffusivity tensor **D** and that diffusion encoding is achieved by a magnetic field gradient waveform *g*(*t*). Note that we absorb the gyromagnetic ratio γ into *g*(*t*) so that it has dimensions of time−<sup>1</sup> length−1. 3

The signal *S*<sup>c</sup> = - d<sup>3</sup>*x* ρ(*x*, *t*) arising from a single such confined compartment under a general encoding waveform *g*(*t*) can be found thanks to the Brownian paths having a Gaussian probability measure under the potential (2) [49]:

$$\mathbf{L}\mathbf{S}\_{\mathbf{c}} = \exp\left(-\frac{1}{2}\mathbf{q}\_{\mathbf{c}}^{\mathsf{T}}(\mathbf{0})\mathbf{D}\mathbf{Q}^{-1}\mathbf{q}\_{\mathbf{c}}(\mathbf{0}) - \int\_{0}^{t\_{\mathbf{c}}} \mathbf{d}\tau \,\mathbf{q}\_{\mathbf{c}}^{\mathsf{T}}(\tau)\mathbf{D}\mathbf{q}\_{\mathbf{c}}(\tau)\right),\tag{3}$$

<sup>1</sup>In units of the thermal energy scale *k*B*T* , where *k*<sup>B</sup> is the Boltzmann constant and *T* is the absolute temperature.

<sup>2</sup>Vanishing confinement (i.e., free diffusion) has an unnormalizable steady state, but it can be handled.

<sup>3</sup>Alternatively, *g*(*t*) can be called a precession rate gradient waveform.

with the generalized encoding wave vector

$$\mathbf{q}\_{\mathbf{c}}(t) = \int\_{t}^{t\_{\varepsilon}} \mathbf{d}\tau \,\mathbf{e}^{-\mathfrak{A}(\tau-t)} \mathbf{g}(\tau). \tag{4}$$

Here, *t*<sup>e</sup> is the duration of the encoding protocol, and

$$
\mathfrak{Q} = \mathbf{D} \mathbf{C} \tag{5}
$$

is a matrix of equilibration rates. According to an equivalence between the signals of diffusion under a Hookean confinement and under restricting walls [34], a restriction of linear size *L* should have a confinement value of around *C* ≈ 11/*L*2. For instance, the confinement value within walls 5 µm apart would be about *C* ≈ 0.44 µm−2. With water diffusing at *D* = 3 µm<sup>2</sup>/ms, this would imply an equilibration rate of - ≈ 1.3 ms−1. In other words, it would take random walkers a few milliseconds to spread out roughly to their eventual distribution.

As depicted in Fig. 2, double-diffusion encoding is achieved by a gradient waveform consisting of two pairs of bipolar rectangular pulses, each with a given duration δ and separation , and magnitudes *g*<sup>1</sup> and −*g*2; the minus sign is customary. The time between the leading edges of the second pulse of the first pair and the first pulse of the second pair, *t*m, is called the mixing time. The calculation of the signal (3) therefore entails very simple integrals, but in a cumbersome piecewise fashion. Upon significant simplification one finds

$$\mathbf{S}\_{\mathbf{c}}(\mathbf{g}\_1, \mathbf{g}\_2) = \exp\left(-\mathbf{g}\_1^{\mathsf{T}} \mathbf{T}\_{\circ} \mathbf{g}\_1 - \mathbf{g}\_2^{\mathsf{T}} \mathbf{T}\_{\circ} \mathbf{g}\_2 - 2\mathbf{g}\_1^{\mathsf{T}} \mathbf{T}\_{\times} \mathbf{g}\_2\right) \,. \tag{6}$$

One may refer to the tensors

$$\mathbf{T}\_{\circ} = \mathbf{D}\mathbf{\mathcal{Q}}^{-3} \left[ (\mathbf{l} - \mathbf{e}^{-\mathbf{\mathcal{Q}}\Delta})(\mathbf{l} - \mathbf{e}^{-\mathbf{\mathcal{Q}}\delta})^2 \mathbf{e}^{\mathbf{\mathcal{Q}}\delta} - (\mathbf{l} - \mathbf{e}^{-2\mathbf{\mathcal{Q}}\delta}) \mathbf{e}^{\mathbf{\mathcal{Q}}\delta} + 2\mathbf{\mathcal{Q}}\delta \right], \quad \text{(7a)}$$

$$2\mathbf{T}\_{\times} = \mathbf{D}\mathbf{Q}^{-3}\mathbf{e}^{-\mathbf{Q}(t\_{\rm in} - \delta)}(1 - \mathbf{e}^{-\mathbf{Q}\delta})^2(1 - \mathbf{e}^{-\mathbf{Q}\Delta})^2,\tag{7b}$$

respectively, as the *self-coupling* and *cross-coupling* tensors between encoding blocks.<sup>4</sup> For the orientational average below, we consider |*g*1|=|*g*2| = *g*, with a fixed angle ψ between them.

As the free diffusion limit is approached, one can see that the cross-coupling of encoding blocks vanishes, as **T**<sup>×</sup> ∼ (**D**/2)δ<sup>2</sup><sup>2</sup>(1 + δ − *t*m), due to displacements in separate time intervals being uncorrelated in pure Brownian motion [36]. The dependence on the mixing time *t*<sup>m</sup> does not enter until second order in confinement, in line with approximate calculations done for a spherical wall [29].

<sup>4</sup>**T**◦ is the same tensor that appears in the single-encoding signal *<sup>S</sup>*<sup>c</sup> <sup>=</sup> <sup>e</sup>−*g*-**<sup>T</sup>**◦ *<sup>g</sup>* [49]. In the free diffusion (*-* <sup>→</sup> 0), it is easily shown that **<sup>T</sup>**◦ <sup>→</sup> **<sup>D</sup>**δ2( <sup>−</sup> <sup>δ</sup>/3), which recovers the free diffusion signal [43, 44]. .

#### **3 Double Diffusion Encoding: Powder Average**

The orientational (powder) average of the compartment signal (6) is performed as follows: The vectors *g*<sup>1</sup> and *g*<sup>2</sup> form a plane with unit normal vector

$$
\hat{\mathfrak{n}} = \frac{\mathbf{g}\_1 \times \mathbf{g}\_2}{|\mathbf{g}\_1 \times \mathbf{g}\_2|}. \tag{8}
$$

One constructs the average by integrating over all orientations of (the plane normal to) *n*ˆ, and for each of these integrate over all orientations of the pair {*g*1, *g*2} within the plane, with their relative angle ψ fixed. The procedure hence described can be written as5

$$\bar{S} = \int \frac{\mathrm{d}\hat{\boldsymbol{\mu}}}{4\pi} \int \frac{\mathrm{d}\boldsymbol{\beta}}{2\pi} \, \mathrm{S}\_{\mathfrak{c}}(\mathbf{g}\_{\hat{\mathfrak{h}}}(\boldsymbol{\beta}), \, \mathbf{g}\_{\hat{\mathfrak{h}}}(\boldsymbol{\beta} + \boldsymbol{\psi})) \,. \tag{9}$$

Here, *gn*ˆ(β) is a vector of magnitude *g* in the plane normal to *n*ˆ, whose orientation is parameterized by the (in-plane) azimuthal angle β, according to which we can identify *g*<sup>1</sup> → *gn*ˆ(β) and *g*<sup>2</sup> → *gn*ˆ(β + ψ).

We first take the in-plane integral

$$\tilde{S}(\hat{\mathfrak{n}}) \stackrel{\text{def}}{=} \int \frac{\mathbf{d}\beta}{2\pi} S\_{\mathbf{c}}(\mathbf{g}\_{\hat{\mathfrak{n}}}(\beta), \mathbf{g}\_{\hat{\mathfrak{n}}}(\beta + \psi)) \stackrel{\text{def}}{=} \int \frac{\mathbf{d}\beta}{2\pi} \mathbf{e}^{-\sigma\_{\hat{\mathfrak{n}}}(\beta)},\tag{10}$$

which serves as the definitions for the intermediate quantities *S*˜(*n*ˆ) and σ*<sup>n</sup>*ˆ(β). According to Eq. (6), the latter is given by

$$\sigma\_{\hat{\mathbf{n}}}(\boldsymbol{\beta}) = \mathbf{g}\_{\hat{\mathbf{n}}}^{\mathsf{T}}(\boldsymbol{\beta})\mathbf{T}\_{\diamond}\mathbf{g}\_{\hat{\mathbf{n}}}(\boldsymbol{\beta}) + \mathbf{g}\_{\hat{\mathbf{n}}}^{\mathsf{T}}(\boldsymbol{\beta} + \boldsymbol{\psi})\mathbf{T}\_{\diamond}\mathbf{g}\_{\hat{\mathbf{n}}}(\boldsymbol{\beta} + \boldsymbol{\psi}) + 2\mathbf{g}\_{\hat{\mathbf{n}}}^{\mathsf{T}}(\boldsymbol{\beta})\mathbf{T}\_{\times}\mathbf{g}\_{\hat{\mathbf{n}}}(\boldsymbol{\beta} + \boldsymbol{\psi}) \,. \tag{11}$$

To anchor the angle coordinate β, we define the in-plane cartesian coordinates *u*, *v* such that *u*ˆ *gn*ˆ(0) and *v*ˆ = *n*ˆ **×** *u*ˆ, yielding

$$\mathbf{g}\_{\hat{\mathbf{n}}}(\beta) = \hat{\mathbf{u}}\,\mathbf{g}\,\cos\beta + \hat{\mathbf{v}}\,\mathbf{g}\,\sin\beta\,. \tag{12}$$

After an exercise in trigonometric simplification, and denoting *u*ˆ - **T**◦*v*ˆ = *T* ◦ *uv* etc., one finds

$$
\sigma\_{\hat{\mathfrak{n}}}(\beta) = \varsigma\_{\hat{\mathfrak{n}}} + \rho\_{\hat{\mathfrak{n}}} \cos 2\beta - \varrho\_{\hat{\mathfrak{n}}} \sin 2\beta \,\,,\tag{13}
$$

<sup>5</sup>We suppress the obvious limits of these integrals.

with

$$\begin{aligned} \frac{\delta \mathfrak{k}}{\delta g^2} &= \left( T\_{uu}^{\circ} + T\_{vv}^{\circ} \right) + \left( T\_{uu}^{\times} + T\_{vv}^{\times} \right) \cos \psi \,, \end{aligned} \tag{14a}$$

$$\begin{aligned} \frac{\rho\_{\tilde{\mathbf{n}}}}{g^2} &= \left( T\_{uu}^{\circ} - T\_{vv}^{\circ} \right) \frac{1 + \cos 2\psi}{2} + T\_{uv}^{\circ} \sin 2\psi + \left( T\_{uu}^{\times} - T\_{vv}^{\times} \right) \cos \psi + 2T\_{uv}^{\times} \sin \psi \,, \end{aligned} \tag{14b}$$

$$\frac{\varrho\_{\text{fl}}}{\varrho^2} = \left(T\_{\text{uu}}^{\circ} - T\_{\text{vv}}^{\circ}\right) \frac{\sin 2\psi}{2} - T\_{\text{uv}}^{\circ} \left(1 + \cos 2\psi\right) + \left(T\_{\text{uu}}^{\times} - T\_{\text{vv}}^{\times}\right) \sin \psi - 2T\_{\text{uv}}^{\times} \cos \psi \,. \tag{14c}$$

These depend on *n*ˆ through *u*ˆ and *v*ˆ, of course. Hence via Eq. (10),

$$\begin{split} \tilde{S}(\hat{\mathfrak{n}}) &= \mathrm{e}^{-\varsigma\_{\hat{\mathfrak{n}}}} \int \frac{\mathrm{d}\beta}{2\pi} \mathrm{e}^{-\rho\_{\hat{\mathfrak{n}}}\cos 2\beta + \varrho\_{\hat{\mathfrak{n}}}\sin 2\beta} \\ &= \mathrm{e}^{-\varsigma\_{\hat{\mathfrak{n}}}} I\_0 \left( \sqrt{\rho\_{\hat{\mathfrak{n}}}^2 + \varrho\_{\hat{\mathfrak{n}}}^2} \right) \,, \tag{15} \end{split} \tag{15}$$

where an integral representation of the modified Bessel function of order 0 was recognized [10]. The argument of the square root can be found after a semi-tedious calculation as

$$\rho\_{\hat{\mathfrak{h}}}^2 + \varrho\_{\hat{\mathfrak{h}}}^2 = \operatorname{g}^4 \left[ \left( T\_{\text{uu}}^\circ - T\_{\text{vv}}^\circ \right) \cos \psi + \left( T\_{\text{uu}}^\times - T\_{\text{vv}}^\times \right) \right]^2 + 4 \operatorname{g}^4 \left( T\_{\text{uv}}^\circ \cos \psi + T\_{\text{uv}}^\times \right)^2, \tag{16}$$

and we rewrite the powder-averaged double-encoding signal (9) via Eqs. (10) and (15) as

$$\bar{S} = \int \frac{\mathbf{d}\hat{\mathbf{n}}}{4\pi} \,\mathrm{e}^{-\sphericalangle} I\_0 \left(\sqrt{\rho\_{\hat{\mathbf{n}}}^2 + \varrho\_{\hat{\mathbf{n}}}^2}\right) \,. \tag{17}$$

For the actual evaluation of the integral, explicit expressions in terms of the polar and azimuthal angles (θ, ϕ) of *n*ˆ need to be substituted,6 which are steps we omit here. Eyeballing how entries of **T**◦ and **T**<sup>×</sup> appear in Eq. (17) via Eqs. (14a) and (16), one notes that it is useful to define the intermediate tensors

$$\mathbf{M} = g^2 \left( \mathbf{T}\_\circ \cos \psi + \mathbf{T}\_\times \right) \tag{18a}$$

$$
\tilde{\mathbf{M}} = g^2 \left( \mathbf{T}\_\circ + \mathbf{T}\_\times \cos \psi \right). \tag{18b}
$$

Referring to their eigenvalues, in any preferred order (but the same for both), as *mi* and *m*˜ *<sup>i</sup>* , and using the following shorthand

<sup>6</sup>*n*<sup>ˆ</sup> <sup>=</sup> (sin <sup>θ</sup> cos <sup>ϕ</sup>,sin <sup>θ</sup> sin <sup>ϕ</sup>, cos <sup>θ</sup>), *<sup>u</sup>*<sup>ˆ</sup> <sup>=</sup> (cos <sup>θ</sup> cos <sup>ϕ</sup>, cos <sup>θ</sup> sin <sup>ϕ</sup>, <sup>−</sup> sin <sup>θ</sup>), and *<sup>v</sup>*<sup>ˆ</sup> <sup>=</sup> (<sup>−</sup> sin <sup>ϕ</sup>, cos ϕ, 0).

$$m\_{ij}^- = m\_i - m\_j \,, \tag{19a}$$

$$
\tilde{m}\_{ij}^- = \tilde{m}\_i - \tilde{m}\_j \; \; , \tag{19b}
$$

$$
\tilde{m}\_{ij}^+ = \tilde{m}\_i + \tilde{m}\_j \; \; , \tag{19c}
$$

the powder-averaged signal (17) attains the form

$$\begin{split} \bar{S} &= \mathbf{e}^{-\tilde{m}\_{12}^{+}} \int \frac{\mathbf{d} \cos \theta \, \mathbf{d} \, \mathbf{\bar{\boldsymbol{\sigma}}}}{4\pi} \mathbf{e}^{-\left(\tilde{m}\_{31}^{-} + \tilde{m}\_{12}^{-} \sin^{2} \varphi\right) \sin^{2} \theta} \\ &\times I\_{0} \Big( \sqrt{\left(m\_{12}^{-}\right)^{2} + 2m\_{12}^{-} \left[m\_{31}^{-} + \left(m\_{13}^{-} + m\_{23}^{-}\right) \sin^{2} \varphi\right] \sin^{2} \theta + \left(m\_{13}^{-} + m\_{21}^{-} \sin^{2} \varphi\right)^{2} \sin^{4} \theta} \Big), \end{split} \tag{20}$$

upon substantial algebraic manipulation.

Reference [13] found series expansions for this form in a different context. We can use its results. Namely,

$$\tilde{S} = \mathbf{e}^{-\tilde{m}\_{12}^{+}} \sum\_{k=0}^{\infty} \sum\_{m=0}^{k} q\_{mk} Y\_{mk} \,, \quad \text{where} \tag{21a}$$

$$Y\_{mk} = \sum\_{n=0}^{\infty} \sum\_{l=0}^{n} \frac{\sqrt{\pi}(-1)^n}{l!(n-l)!} \frac{\left(\tilde{m}\_{31}^{-}\right)^{n-l} \left(\tilde{m}\_{12}^{-}\right)^l}{2^{2(m+l)+1}} \binom{2m+2l}{m+l} \frac{(n+k)!}{\left(n+k+\frac{1}{2}\right)!}, \quad \text{and} \tag{21b}$$

$$q\_{mk} = \frac{\left(m\_{31}^{-}\right)^{k-m}}{(k-m)!} \sum\_{j=0}^{k/2} \frac{\left(m\_{12}^{-}\right)^{j} I\_{k-j}\left(m\_{12}^{-}\right)}{2^{j} j! \left(m\_{13}^{-} + m\_{23}^{-}\right)^{2j-m}} \,\_2\tilde{F}\_1(m-k, -2j; m+1-2j; \frac{m\_{13}^{-}+m\_{23}^{-}}{m\_{12}^{-}})\,\_1,\tag{21c}$$

with the following three alternatives for Eq. (21b):

$$Y\_{mk}^{(1)} = \frac{\sqrt{\pi}}{2^{2m+1}} \binom{2m}{m} \sum\_{n=0}^{\infty} \frac{\left(\tilde{m}\_{13}^{-}\right)^{n} (k+n)!}{n! (k+n+\frac{1}{2})!} \,\_2F\_1 \left(m+\frac{1}{2}, -n; m+1; \frac{\tilde{m}\_{21}^{-}}{\tilde{m}\_{31}}\right), \tag{22a}$$
 
$$Y\_{mk}^{(2)} = \frac{\sqrt{\pi}}{2^{2m+1}} \sum\_{n=0}^{\infty} \frac{\left(\tilde{m}\_{21}^{-}\right)^{n} (k+n)! \binom{2n+2m}{n+m}}{2^{2n} n! (k+n+\frac{1}{2})!} \,\_1F\_1 \left(k+n+1; k+n+\frac{3}{2}; \tilde{m}\_{13}^{-}\right), \tag{22b}$$

*Y* (3) *mk* = √π 22*<sup>m</sup>*+<sup>1</sup> 2*m m* <sup>∞</sup> *n*=0 *m*˜ <sup>−</sup> 13*n* (*k*+*n*)! *n*! *<sup>k</sup>*+*n*+<sup>1</sup> 2 ! <sup>2</sup>*F*<sup>2</sup> *<sup>m</sup>*+<sup>1</sup> <sup>2</sup> , *<sup>k</sup>*+*n*+1; *<sup>m</sup>*+1, *<sup>k</sup>*+*n*+<sup>3</sup> <sup>2</sup> ; ˜*m*<sup>−</sup> 21 . (22c)

Here, *<sup>i</sup> Fj*(. . .) are the confluent hypergeometric functions [3], tilde denoting regularization. Note that these expressions apply to the most general case, in which the confinement tensor (and thus the tensors that are functions of it) have three *distinct* eigenvalues. Special cases associated with coinciding eigenvalues are discussed in the appropriate occasion later.

Which alternative among Eq. (22) yields better convergence is a matter of how the eigenvalues *m*˜ *<sup>i</sup>* are chosen to be ordered, by way of the sizes and signs of *m*˜ <sup>−</sup> *i j* . As a general guideline it would be wise to order the eigenvalues so as to avoid a sequence that alternates in sign, and with a large expansion parameter. Take Eq. (22b) for instance. Given that <sup>1</sup>*F*1(*k*+*l*+1; *<sup>k</sup>*+*l*+<sup>3</sup> <sup>2</sup> ; ˜*m*<sup>−</sup> <sup>13</sup>) > 0 and increasing for all *m*˜ <sup>−</sup> 13, it would be beneficial to make *m*˜ <sup>−</sup> <sup>13</sup> negative (and large if possible), while keeping *m*˜ <sup>−</sup> 21 positive (and small if possible). For a given set of eigenvalues *m*˜ *<sup>i</sup>* , ordering them in the fashion *m*˜ <sup>1</sup> < *m*˜ <sup>2</sup> < *m*˜ <sup>3</sup> would be along this guideline, whereas the ordering *m*˜ <sup>2</sup> < *m*˜ <sup>3</sup> < *m*˜ <sup>1</sup> would result in a sequence with larger terms and alternating sign.

Note however that while **T**◦ is a monotonic (decreasing) function of *-*, **T**<sup>×</sup> is not; see Eq. (7). Through Eq. (18) this means that their mixtures **M** and **M**˜ are not necessarily monotonic in the confinement*-*. Hence what ordering of the confinement eigenvalues *<sup>i</sup>* achieves what ordering in the eigenvalues *mi* and *m*˜ *<sup>i</sup>* is a question which has an answer only on a case-by-case basis. Furthermore, the ordering of *i* that yields a desirable ordering of *m*˜ *<sup>i</sup>* for a particular one of Eq. (22) may not produce an ordering of *mi* as desirable for the convergence of *qmk* in Eq. (21c).

#### *3.1 Axisymmetric Confinement*

We refer to the condition when two of the eigenvalues *<sup>i</sup>* of the confinement tensor coincide as *axisymmetric* confinement. Under this condition, the series expansions above undergo simplifications.

The most drastic simplification occurs when -<sup>1</sup> = -2. That is, given that two eigenvalues coincide, assigning first and second place to them is the wisest choice as far as the evaluation of the series expansions (21) is concerned.

First, the coefficient *qmk* simplifies as<sup>7</sup>

$$I\_0(\dots) = \sum\_{k=0}^{\infty} \sum\_{m=0}^{k} q\_{mk} \sin^{2m} \varphi \sin^{2k} \theta \dots$$

Upon all coefficients in the argument except for *m*− <sup>13</sup> vanishing due to axisymmetry, one has

$$I\_0(m\_{13}^-\sin^2\theta) = \sum\_{k=0}^{\infty} \frac{(m\_{13}^-)^{2k}}{(k!)^{2}2^{2k}}\sin^{2k}\theta,$$

which, comparing to the previous (double) expansion, implies Eq. (23).

<sup>7</sup>For this, it needs to be noted [13, supplementary information] that the way *qmk* arises in the calculation—before ever arriving at Eq. (21c)—is that it is the coefficient in the (double) series expansion of the Bessel function in Eq. (20):

212 C. Yolcu et al.

$$q\_{mk} = \begin{cases} \frac{\delta\_{m0} (m\_{13}^{-})^k}{(\frac{k}{2})! (\frac{k}{2})! 2^k}, & \text{even } k\\ 0, & \text{odd } k \end{cases} . \tag{23}$$

The coefficient *Ymk* , on the other hand, loses the interior summation in Eq. (21b) due to *m*<sup>−</sup> <sup>12</sup> vanishing. The remaining summation can be identified according to the definition of hypergeometric functions [3] as

$$Y\_{mk} = \frac{\sqrt{\pi}}{2^{m+1}} \binom{2m}{m} \frac{k!}{\left(k + \frac{1}{2}\right)!} \,\_1F\_1\left(k+1; k+\frac{3}{2}; \tilde{m}\_{13}^-\right),\tag{24}$$

which yields via Eq. (23) and Eq. (21a)

$$\bar{S}\_{\text{axy}} = \sqrt{\pi} \mathbf{e}^{-\tilde{m}\_{l2}^{+}} \sum\_{n=0}^{\infty} \binom{2n}{n} \frac{\binom{m}{13}^{2n}}{2^{2n+1}} \frac{{}\_1F\_1(2n+1; 2n+\frac{3}{2}; \tilde{m}\_{13}^{-})}{\left(2n+\frac{1}{2}\right)!} . \tag{25}$$

This expansion is not so sensitive to the ordering of eigenvalues number 1 and 3, as the expansion parameter (*m*<sup>−</sup> 13) is squared, and the hypergeometric function <sup>1</sup>*F*1(2*n*+1; 2*n*+3/2; ˜*m*<sup>−</sup> <sup>13</sup>) > 0 for all arguments.

We depict in Fig. 3 the evaluation of the powder averaged signal (25) for axisymmetric confinement for representative values of encoding parameters. In Fig. 3a, the confinement is anisotropic (prolate), whereas in Fig. 3b, it is (nearly) isotropic. The bell-shaped dependence on the relative angle ψ between the gradient directions is seen in both cases, which is a sign that diffusion is not free [29]. This dependence is due mainly to the exponential prefactor in Eq. (25) that has nothing to do with the difference between the confinement eigenvalues (anisotropy). When the mixing time is increased, the bell-shaped modulation, indicating confinement regardless of anisotropy , stops overwhelming the relatively smaller influence of the rest of the expression (25): see Fig. 3a where the confinement is anisotropic. In an isotropic confinement, on the other hand, angular modulation simply disappears when the mixing time is increased (Fig. 3b), illustrating that the angular modulation that survives the increase in mixing time is due only to compartmental anisotropy, and not due to the fact of confinement (or to ensemble anisotropy, which was already eliminated by powder averaging).

#### *3.2 Insights from Two Dimensions*

The signal expressions for double encoding are a bit unwieldy to get a conceptual handle on. However, some insight can be gleaned from considering the orientational averaging in two dimensions.

Obviously, in the spirit of Eq. (9), the 2D orientationally averaged signal can be written as

**Fig. 3** Dependence of powder averaged signal (25) on the angle ψ between double-encoding gradient directions. **a** Prolate confinement *-*, with eigenvalues {1, 1, 0.1} in arbitrary units. = 10 and δ = 1, in units of -−1 <sup>1</sup> . The mixing time *t*<sup>m</sup> was set to the following multiples {1, 2, 6, 30} of δ. The gradient strength is set such that *D*0*g*2/-3 <sup>1</sup> = 0.16, with *D*<sup>0</sup> being the unit of diffusivity. **b** Isotropic confinement *-*, with eigenvalues {1, 1, 1} in arbitrary units. = 3 and δ = 2/3, in units of -−1 <sup>1</sup> . The mixing time *t*<sup>m</sup> was set to the following multiples {1, 3/2, 3, 6} of δ. The gradient strength is set such that *D*0*g*2/-3 <sup>1</sup> = 1. **c** Free diffusion with varying anisotropy (prolate) for comparison, with diffusivity **D** eigenvalues {*D*0, *eD*0, *eD*0}. Across the three groups of plots, diffusion weighting *<sup>b</sup>* <sup>=</sup> *<sup>g</sup>*2δ2( <sup>−</sup> <sup>δ</sup>/3) takes the values {0.4, <sup>1</sup>.4, <sup>10</sup>} in units of *<sup>D</sup>*−<sup>1</sup> <sup>0</sup> , whereas each group of plots have the eccentricity parameter set to {0.01, 0.1, 0.2}

$$\bar{S}\_{\text{2D}} = \int \frac{\mathbf{d}\beta}{2\pi} \, \mathcal{S}\_{\text{c}}(\mathbf{g}(\beta), \mathbf{g}(\beta + \psi)) \,. \tag{26}$$

That is, there is no normal vector to integrate over; everything takes place in an (*x*, *y*) plane. The same steps Eq. (10) through Eq. (15) apply, and one has

$$\bar{S}\_{2D} = \mathbf{e}^{-g^2 \text{Tr}\left(\mathbf{T}\_\circ + \mathbf{T}\_\times \cos \psi\right)} I\_0(g^2 (T\_1^\circ - T\_2^\circ) \cos \psi + g^2 (T\_1^\times - T\_2^\times)) \,. \tag{27}$$

The dependence on the relative angle ψ of the gradient directions occurs both in the exponential attenuation factor and in the Bessel function. The angular dependence in the Bessel function has to do with anisotropy (*T* ◦ <sup>1</sup> − *T* ◦ <sup>2</sup> ) which the exponential factor is insensitive to due to the trace. In the exponent, on the other hand, the angular dependence is controlled by Tr**T**×, whose presence is due to confinement (since **T**<sup>×</sup> → 0 in the free diffusion limit). For large mixing times (*t*m*<sup>i</sup>* 1) the latter drops out and the angular modulation due to anisotropy is liberated. However, for smaller mixing time, it turns out that the angular dependence of the exponential factor dominates, which is due to confinement, suppressing the signature of anisotropy.

An interesting double-encoding scheme is its "symmetrized" version [35], which fixes ψ = π/2, but varies the magnitudes *g*<sup>1</sup> = *g* cos α and *g*<sup>2</sup> = *g* sin α as a function of some parameter α. In that scenario, one can easily calculate that

$$\bar{S}\_{\text{2D}}^{\text{sym}} = \text{e}^{-\frac{\omega^2}{2}\text{Tr}(\mathbf{T}\_\circ)} I\_0 \left( \frac{\text{g}^2}{2\sqrt{2}} \sqrt{\left(\Delta T\_\circ^2 + \Delta T\_\times^2\right) + \left(\Delta T\_\circ^2 - \Delta T\_\times^2\right) \cos 4\alpha} \right), \qquad (28)$$

showing that the modulation due to confinement in the exponential factor drops out, and one of pure anisotropy remains (with *T*◦ denoting *T* ◦ <sup>1</sup> − *T* ◦ <sup>2</sup> etc.). The result is a 'cleaner' version of the signal modulation wherein the confinement anisotropy is the only source of angular modulation characterized by the cos 4α dependence. It should be remembered though that the confinement model is purely Gaussian, as such does not account for compartmental kurtosis. When truly restricted diffusion is considered, the compartment anisotropy and compartmental kurtosis both yield the same type of angular dependence compromising the interpretation of such angular dependence except when the compartments are isotropic [35].

#### *3.3 One-Dimensional Diffusion Under High Gradient: g***−<sup>2</sup>** *Scaling*

A special case of interest is when the compartment has an extremely elongated shape, resembling a "stick". Such extremely anisotropic shapes combined with an asymptotically large gradient strength (*g* → ∞) tend to exhibit power-law dependence on the gradient strength *g* or the wave vector *q* = δ*g*. In what follows, we confirm that the orientationally averaged signal (25) is no exception. Physical interpretation of the limits or extremes involved is remarked on at the end.

The two transverse confinement eigenvalues -<sup>1</sup> = -<sup>2</sup> = -<sup>⊥</sup> approach infinity while -<sup>3</sup> = is finite. Accordingly, *m*<sup>1</sup> = *m*<sup>2</sup> = *m*<sup>⊥</sup> = 0 and *m*<sup>3</sup> = *m* is finite (similarly for *m*˜ *<sup>i</sup>*). Combined with a gradient *g* → ∞, noting that *m* ∼ *g*<sup>2</sup> and *m*˜ ∼ *g*<sup>2</sup> due to the definition (18), the orientationally averaged signal (25) assumes the asymptotic form

$$\bar{S}\_{\rm stick} \sim \left( 2\tilde{m}\_{\parallel} \sqrt{1 - \left( \frac{m\_{\parallel}}{\tilde{m}\_{\parallel}} \right)^2} \right)^{-1};\tag{29}$$

see footnote 8 for details.<sup>8</sup> When we substitute from Eq. (18), the power-law dependence (∼ *g*−2) becomes obvious:

$$\bar{\mathbf{S}}\_{\rm stick} \sim \left( 2g^2 \left( T\_{\parallel}^{\circ} + T\_{\parallel}^{\times} \cos \psi \right) \sqrt{1 - \left( \frac{T\_{\parallel}^{\circ} \cos \psi + T\_{\parallel}^{\times}}{T\_{\parallel}^{\circ} + T\_{\parallel}^{\times} \cos \psi} \right)^2} \right)^{-1} . \tag{30}$$

This result should be compatible with the findings of Ref. [13] in the free diffusion limit (*-* → 0). It is easy to see that Eq. (7) implies *T* <sup>×</sup> → 0 and *T* ◦ <sup>→</sup> *<sup>D</sup>*δ<sup>2</sup>( <sup>−</sup> δ/3) as *-*→ 0, yielding

$$\bar{S}\_{\rm stick}^{\rm free} \sim \left( 2g^2 \delta^2 D\_{\parallel} \left( \Delta - \frac{\delta}{3} \right) \sin \psi \right)^{-1},\tag{31}$$

which is in fact the powder averaged signal of a rank-1 diffusivity tensor *<sup>D</sup>* 0 0 0 00 0 00 under the rank-2 measurement matrix *g*<sup>2</sup>δ<sup>2</sup> <sup>−</sup> <sup>δ</sup> 3 1+cos <sup>ψ</sup> 0 0 0 1−cos ψ 0 0 00 of DDE, as predicted by Ref. [13]. This scaling (∼ *g*−2) of the signal of stick-compartments under rank-2 measurements (planar encoding) has been recently confirmed in vivo [1]. Note that the expression is invalid for ψ = 0 and ψ = π where the two encoding directions coincide, and the corresponding measurement matrix is rank-1.

Lastly, we address the physical interpretation of the limits of *large* confinement (-) and gradient (*g*) values. Note that the final expressions such as Eqs. (21)–(22) and (25) come out in terms of the eigenvalues of the matrices **M** and **M**˜ , which are in turn defined in Eqs. (18) and (7). The sense in which a confinement eigenvalue - and gradient strength *g* is large follows in particular from Eq. (7): In the same order as the limits have been carried out, infinite (transverse) confinement (-<sup>⊥</sup> → ∞) is to be interpreted physically as *g*2*D*⊥-−3 <sup>⊥</sup> 1, while infinite gradient (*g* → ∞)

8At the extreme of *<sup>m</sup>*<sup>1</sup> <sup>=</sup> *<sup>m</sup>*<sup>2</sup> <sup>=</sup> *<sup>m</sup>*<sup>⊥</sup> <sup>=</sup> 0, the signal (25) becomes

$$\bar{S}\_{\text{stack}} \sim \sqrt{\pi} \sum\_{n=0}^{\infty} \binom{2n}{n} \frac{m\_{\parallel} ^{2n}}{2^{2n+1}} \frac{\,\_1F\_1\left(2n+1; 2n + \frac{3}{2}; -\tilde{m}\_{\parallel}\right)}{\left(2n + \frac{1}{2}\right)!}.$$

The hypergeometric function can be rewritten by a so-called Kummer transformation as

$$\,\_1F\_1(2n+1;2n+\frac{3}{2};-\tilde{m}\,\mathbb{I}) = \mathfrak{e}^{-\tilde{m}\,\mathbb{I}}\,\_1F\_1(\frac{1}{2};2n+\frac{3}{2};\tilde{m}\,\mathbb{I}) \sim \left(2n+\frac{1}{2}\right)!/\sqrt{\pi}\,\tilde{m}\,\mathbb{I}^{2n+1},$$

where the second step uses the asymptotic form (*m*˜ → ∞) of the Hypergeometric function [10]. Hence, the signal attains the form

$$
\bar{S}\_{\rm stick} \sim \sum\_{n=0}^{\infty} \binom{2n}{n} \frac{m\_{\parallel}^{2n}}{2^{2n+1} \bar{m}\_{\parallel}^{2n+1}},
$$

converging to Eq. (29) for |*m*/*m*˜ | < 1, which can be verified by carefully considering the functional dependences arising from the explicit form (18) of the matrices **M** and **M**˜ (with the exception of the encoding angles ψ = 0 and ψ = π where the two encoding directions coincide).

is to be taken to mean *g*2*D*-−3 1. One may write *<sup>D</sup>*−<sup>1</sup> -3 *<sup>g</sup>*<sup>2</sup> *<sup>D</sup>*−<sup>1</sup> <sup>⊥</sup> -3 <sup>⊥</sup> in short. It should be noted that this order-of-magnitude treatment ignores potential exceptions or extremes which may arise from very particular combinations of the protocol parameters δ, , *t*m, and ψ.

#### **4 Single Diffusion Encoding**

The powder-averaged single-encoding signal can be obtained by way of "hacking" its double-encoded counterpart. One notes that *<sup>S</sup>*<sup>c</sup> <sup>=</sup> <sup>e</sup>−*g*-**<sup>T</sup>**◦ *<sup>g</sup>* is the single-encoding signal at the compartment level [49]. Then the features of the form (6) to get rid of are (*i*) the presence of a second vector *g*<sup>2</sup> unequal to the first, (*ii*) the presence of cross-coupling (**T**×), and (*iii*) the double occurrence of self-coupling. The first is hacked away by setting ψ = 0. For the second, one simply sets **T**<sup>×</sup> = 0.<sup>9</sup> The result is *<sup>S</sup>*<sup>c</sup> <sup>=</sup> <sup>e</sup>−2*g*-**<sup>T</sup>**◦ *<sup>g</sup>*, suffering from the third problem above, which is fixed by replacing *g* → *g*/ <sup>√</sup>2. All this yields via Eq. (18)

$$\mathbf{M} = \frac{1}{2} \mathbf{g}^2 \mathbf{T}\_\circ = \mathbf{\tilde{M}}\,,\tag{32}$$

which is the substitution that converts the powder-average expressions for doubleencoding into those of single-encoding.

With this condition applied, the *m*˜ <sup>−</sup> *i j*'s appearing in Eq. (21) turn into *m*<sup>−</sup> *i j*'s. However this alone does not produce drastic simplifications such as reducing summations. Rather than using Eq. (21), in fact, it is better to note that other results in Ref. [13] are quite suitable in the case of single-encoding. Since the compartment signal has the form *<sup>S</sup>*<sup>c</sup> <sup>=</sup> <sup>e</sup>−Tr(**<sup>T</sup>**◦ *g g*-), with the matrix *g g* axisymmetric by dint of being rank-1, the results of Ref. [13] for axisymmetric diffusion or measurement tensor can be applied. Written in terms of the parameters of the present discussion, the relevant formulas of Ref. [13] indicate the following alternative expressions

$$\bar{S}^{(1)} = \frac{\sqrt{\pi}}{2} \mathrm{e}^{-g^2 T\_3^\circ} \sum\_{n=0}^\infty \frac{g^{2n} \left(T\_3^\circ - T\_1^\circ\right)^n}{\left(n + \frac{1}{2}\right)!} \,\_2F\_1\left(\frac{1}{2}, -n; 1; \frac{T\_1^\circ - T\_2^\circ}{T\_1^\circ - T\_3^\circ}\right),\tag{33a}$$

$$\bar{S}^{(2)} = \mathbf{e}^{-g^2 T\_3^\circ} \sum\_{n=0}^\infty \frac{g^{2n} \left(T\_1^\circ - T\_2^\circ\right)^n}{n!(2n+1)!} \,\_1F\_1\left(n+1; n+\frac{3}{2}; \,\mathrm{g}^2(T\_3^\circ - T\_1^\circ)\right),\tag{33b}$$

$$\bar{S}^{(3)} = \frac{\sqrt{\pi}}{2} \text{e}^{-g^2 T\_3^\circ} \sum\_{n=0}^\infty \frac{g^{2n} \left(T\_3^\circ - T\_1^\circ\right)^n}{\left(n + \frac{1}{2}\right)!} \,\_2F\_2\left(\frac{1}{2}, n+1; 1, n+\frac{3}{2}; \, g^2 \left(T\_1^\circ - T\_2^\circ\right)\right),\tag{33c}$$

<sup>9</sup>Physically, this can be imagined as the limit *<sup>t</sup>*<sup>m</sup> → ∞. However, physics is not necessary. We simply have a set of expressions, e.g. Eq. (21), containing **T**◦ and **T**<sup>×</sup> via Eq. (18) via Eq. (19), and we want to remove instances of **T**×.

Magnetic Resonance Assessment of Effective Confinement Anisotropy … 217

$$\bar{S}^{(4)} = \frac{\sqrt{\pi}}{2} \text{e}^{-\text{g}^2 T\_3^\circ} \sum\_{n=0}^\infty \frac{\binom{2n}{n} \text{g}^{4n} \left(T\_1^\circ - T\_2^\circ\right)^{2n}}{16^n \left(2n + \frac{1}{2}\right)!} \,\_1F\_1(2n+1; 2n + \frac{3}{2}; \frac{\text{g}^2}{2}(2T\_3^\circ - T\_2^\circ - T\_1^\circ)),\tag{33d}$$

for fully anisotropic *-* (hence **T**◦); the axisymmetric case is taken up below. These summations are subject to the same guidelines that followed Eq. (22) except that the caveats about conflicting eigenvalue ordering do not apply here. The single-encoding expressions here only involve the self-coupling tensor **T**◦ which is a monotonically decreasing function of *-*; see Eq. (7). Therefore the ordering of the eigenvalues *T* ◦ *i* and *<sup>i</sup>* are certain to be in exactly the opposite sense of each other.

#### *4.1 Axisymmetry and the Power-Laws for Confined diffusion*

The first alternative in Eq. (33) needs special care when *T* ◦ <sup>1</sup> = *T* ◦ <sup>3</sup> and the argument of the hypergeometric function <sup>2</sup>*F*1(. . .) diverges. Invoking a property of the hypergeometric function,10 and renaming *T* ◦ <sup>1</sup> = *T* ◦ <sup>⊥</sup> and *T* ◦ <sup>2</sup> = *T* ◦ , one finds<sup>11</sup>

$$\bar{S}\_{\text{axy}} = \frac{\sqrt{\pi}}{2} \text{e}^{-g^2 T\_{\perp}^{\circ}(\delta, \Delta)} \frac{\text{erf}\left(g\sqrt{T\_{\parallel}^{\circ}(\delta, \Delta) - T\_{\perp}^{\circ}(\delta, \Delta)}\right)}{g\sqrt{T\_{\parallel}^{\circ}(\delta, \Delta) - T\_{\perp}^{\circ}(\delta, \Delta)}}. \tag{34}$$

Here, we have also explicitly denoted the dependence on the encoding protocol's timing parameters as per Eq. (7). Note that in the free diffusion limit (*-* → 0, see footnote 4), the known single-encoding powder-average signal [2, 18, 48]

10

.

.

$$\lim\_{x \to 0} x^n \,\_2F\_1\left(m+\frac{1}{2}, -n; m+1; \frac{\chi}{x}\right) = (-\chi)^n \frac{\left(m+n-\frac{1}{2}\right)!}{\left(m-\frac{1}{2}\right)! \left(m+n\right)!} \,\_2F\_1\left(m+\frac{1}{2}, -n+1; \frac{\chi}{x}\right)$$

11Alternatively to using the results of Ref. [13] here, one may go back to the integral expression (20). First, we note that axisymmetry, with the choice -<sup>1</sup> = -2, makes the integrand independent of ϕ, yielding

$$\tilde{S} = \frac{1}{2} \mathbf{e}^{-\tilde{m}\_{12}^{+}} \int \mathbf{d} \cos \theta \,\mathbf{e}^{\tilde{m}\_{13}^{-} \sin^{2} \theta} I\_0 \left( m\_{13}^{-} \sin^{2} \theta \right) \,\mathrm{J}$$

We are not aware of a closed-form evaluation of this integral. However, its special case (32) relevant here, making the arguments of the exponential and Bessel function match, has the result [12, 6.625– 4]

$$\bar{S} = \frac{\sqrt{\pi}}{2} \mathbf{c}^{-2m\_1} \frac{\operatorname{erf}\left(\sqrt{2m\_{31}^-}\right)}{\sqrt{2m\_{31}^-}} = \frac{\sqrt{\pi}}{2} \mathbf{c}^{-g^2 T\_1^\circ} \frac{\operatorname{erf}\left(\operatorname{g}\sqrt{T\_3^\circ - T\_1^\circ}\right)}{\operatorname{g}\sqrt{T\_3^\circ - T\_1^\circ}}.$$

218 C. Yolcu et al.

$$\bar{S}\_{\text{axy}} = \frac{\sqrt{\pi}}{2} \text{e}^{-q^2(\Delta - \frac{\delta}{3})D\_\perp} \frac{\text{erf}\left(q\sqrt{\left(\Delta - \frac{\delta}{3}\right)\left(D\_\parallel - D\_\perp\right)}\right)}{q\sqrt{\left(\Delta - \frac{\delta}{3}\right)\left(D\_\parallel - D\_\perp\right)}},\tag{35}$$

with *q* = *g*δ, is recovered.

Two special limiting cases follow.

#### **4.1.1 Stick:** *S***¯ ∝** *g***−<sup>1</sup> Scaling**

When particles have negligible latitude to move in the transverse direction, the orientationally-averaged signal (34) assumes a special form. In terms of the confinement model, this corresponds to -<sup>⊥</sup> → ∞, which implies *T* ◦ <sup>⊥</sup> <sup>→</sup> 0 via Eq. (7).<sup>12</sup> The signal (34) then becomes

$$\bar{S}\_{\rm stick} = \frac{\sqrt{\pi}}{2} \frac{\mathrm{erf}\left(\mathrm{g}\sqrt{T\_{\parallel}^{\circ}(\delta,\Delta)}\right)}{\mathrm{g}\sqrt{T\_{\parallel}^{\circ}(\delta,\Delta)}} \stackrel{\mathcal{S}\to\infty}{\sim} \frac{\sqrt{\pi}}{2\mathrm{g}\sqrt{T\_{\parallel}^{\circ}(\delta,\Delta)}} \stackrel{\mathcal{Q}\_{\parallel}\to 0}{\sim} \frac{\sqrt{\pi}}{2q\sqrt{(\Delta-\frac{\delta}{3})D\_{\parallel}}} \cdot \tag{36}$$

The large gradient regime has been important in identifying stick-like compartments via the *q*−<sup>1</sup> scaling, which has been observed in white-matter areas of the brain and has been interpreted with the assumption of a free one-dimensional diffusion [23], which is adequate for channels of straight long channels of infinitesimal diameter. A notable exception is Ref. [33], which has incorporated the effects of finite size and curvature to offer an explanation for any deviation from the *q*−<sup>1</sup> scaling, most apparent within gray matter. Ref. [33] also pointed out that such a scaling is not the true asymptotic behavior of the signal; the latter is rather dictated by the Debye-Porod law yielding *q*−<sup>4</sup> scaling for narrow pulses [38], while an even steeper attenuation is predicted when the pulses are wide.

The above expression suggests that a similar decay is expected for the gradient magnitude rather than the *q*-value. The crucial difference is in the dependence on the timing parameters {δ, } of the SDE sequence, see Fig. 1. It would thus be interesting, e.g. in white-matter, to investigate whether the dependence on the timing parameters is more like 1/ <sup>√</sup> <sup>−</sup> <sup>δ</sup>/3 (free difusion along the fiber) or 1/ *T* ◦ (δ, ) (confined diffusion along the fiber), which would inform about the diffusion process along the axons.

<sup>12</sup>The limits -<sup>⊥</sup> → ∞ and *g* → ∞ are to be interpreted in the same sense as Sect. 3.3.

#### **4.1.2 Pancake:** *S***¯ ∝** *g***−<sup>2</sup> Scaling**

For completeness, we consider the opposite case where particles are able to spread in a plane, but not along the normal. Here, - → ∞, implying *T* ◦ <sup>→</sup> 0 via Eq. (7).<sup>13</sup> One then has<sup>14</sup>

$$\bar{S}\_{\text{puncake}} = \frac{\sqrt{\pi}}{2} \text{e}^{-g^2 T\_{\perp}^s(\delta, \Delta)} \frac{\text{erfi}\left(\text{g}\sqrt{T\_{\perp}^s(\delta, \Delta)}\right)}{\text{g}\sqrt{T\_{\perp}^s(\delta, \Delta)}} \stackrel{\mathcal{S} \to \infty}{\sim} \frac{1}{2g^2 T\_{\perp}^o(\delta, \Delta)} \stackrel{\Omega\_{\perp} \to 0}{\sim} \frac{1}{2q^2(\Delta - \frac{\delta}{3})D\_{\perp}}.\tag{37}$$

Thus, the orientationally-averaged signal attenuates at a faster rate than in the case of sticks. Similarly though, the dependence on the timing parameters are different in free and confined diffusion scenarios.

#### **5 Discussion**

We have provided, for the first time, explicit expressions for the orientationallyaveraged SDE and DDE MR signal intensity for structures represented by confinement tensors [49]. The latter is the effective model of restricted diffusion when pulses are long enough for the diffusing particles to traverse distances larger than the pore size [34]. As such, our findings are relevant for a broad range of porous materials featuring isolated, small pores.

The counterpart of these results for compartments of free anisotropic diffusion were given in Ref. [13] for arbitrary encoding waveforms. Reference [13] showed *inter alia* that the absence of axisymmetry could be discerned from the decay of the orientationally-averaged signal by employing the most general formulas. Though not rigorously studied here, we expect such arguments to be valid in the case of confined geometries as well by employing Eqs. (21)–(22) and (33).

For the time being, considering arbitrary waveforms for confined compartments seems extremely challenging. However, taking confinement into account is important, since the free diffusion model lacks features (such as the dependence on relative angle in double encoding) that a realistic signal will bear, see Fig. 3. Such bell-shaped angular modulation was related to the radius of gyration of the pores [24] and has since been used to estimate the apparent size of pores [4, 17, 29, 30, 41] via DDE measurements. The underlying reason was thought to be a restriction effect [11] that not only leads to an anisotropy of the diffusion process at a length scale smaller than the pore size [26, 29, 31, 32], but also makes the apparent diffusion coefficient depend on the diffusion time [15]. Thus, this effect is absent when free diffusion is thought to take place within individual pores, see Fig. 3c. Our results indicate that

<sup>13</sup>The limits - → ∞ and *g* → ∞ are to be interpreted in a sense analogous to Sect. 3.3, with ⊥ and interchanged.

<sup>14</sup>The imaginary error function has the properties i erfi(*z*) <sup>=</sup> erf(i*z*), and erfi(*z*) <sup>∼</sup> <sup>e</sup>*z*<sup>2</sup> /*z* <sup>√</sup>π.

the confinement tensor framework is capable of capturing such angular modulation, which makes it a suitable representation of diffusion within microdomains [22, 34, 49]. This is important in applications like *q*-space trajectory imaging [47] and diffusion tensor distribution imaging [45], which aim to characterize the structure of subdomains using general gradient waveforms.

Another angular modulation that is apparent in Fig. 3 is w-shaped, which was pointed out by Mitra [24] for randomly distributed sticks. Later, it was proved that the dominant contribution to such angular modulation had the functional form cos 2ψ for fully restricted structures and cylinders of finite diameter [28]. Such modulation that manifests itself at twice the "angular frequency" [19, 24, 28] is present even when diffusion within the subdomains is envisioned to be free as long as it is anisotropic, see Fig. 3c. Thus, such modulation is truly indicative of the anisotropy of the subdomains, be it free (Fig. 3c), confined (Fig. 3a, b), or truly restricted [28, 32]. For more information on such anisotropy, the reader is referred to the review on this topic by Ianu¸s et al. [14] in this book series.

From a mathematical point of view, the expressions in Ref. [13] provided the Laplace transform of a tensor distribution, which includes rotated copies of a given diffusion tensor wherein all orientations are equally likely, thus extending the modeling approach that employs parametric diffusion tensor distributions [16, 21, 37, 39] to a new type of tensor distribution. Evaluating the signal, in a similar fashion, for confinement rather than diffusion tensors can be regarded as the evaluation of a transform whose kernel is the compartmental signal given in (6) instead of the kernel of the matrix Laplace transform e−**BD**.

#### **6 Conclusion**

We have given analytical expressions for the orientationally averaged diffusion MR signal originating from confined anisotropic compartments for two relatively simple encoding schemes. A number of observations related to signal modulation and powerlaw tails were made for such confined pores. These findings complement and extend the exact expressions for locally free diffusion provided in Ref. [13] to confined diffusion albeit for SDE and DDE measurements.

**Acknowledgments** We acknowledge the following sources for funding: Swedish Foundation for Strategic Research AM13-0090, the Swedish Research Council 2016-04482, Linköping University Center for Industrial Information Technology (CENIIT), VINNOVA/ITEA3 17021 IMPACT, and National Institutes of Health P41EB015902 and R01MH074794.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Riemann-DTI Geodesic Tractography Revisited**

**Luc Florack, Rick Sengers, Stephan Meesters, Lars Smolders, and Andrea Fuster**

**Abstract** Clinical tractography is a challenging problem in diffusion tensor imaging (DTI) due to persistent validation issues. Geodesic tractography, based on a shortest path principle, is conceptually appealing, but has not produced convincing results so far. A major weakness is its rigidity with respect to candidate tracts it is capable of producing given a pair of endpoints, showing a tendency to produce false positives (such as shortcuts) and false negatives (e.g. if a shortcut supplants the correct solution). We propose a new geodesic paradigm that appears to overcome these problems, making a step towards semi-automatic clinical use. To this end we couple the DTI tensor field to a *family* of Riemannian metrics, governed by control parameters. In practice these parameters may allow for edits by an expert through manual selection among multiple tract suggestions, or for bringing in a priori knowledge. In this paper, however, we consider an automatic, evidence-driven procedure to determine optimal controls and corresponding tentative tracts, and illustrate the role of edits to remediate erroneous defaults.

A. Fuster e-mail: A.Fuster@tue.nl

L. Florack · R. Sengers (B) · S. Meesters · L. Smolders · A. Fuster Eindhoven University of Technology, Department of Mathematics & Computer Science, NL-5600 Eindhoven, MB, The Netherlands e-mail: H.J.C.E.Sengers@tue.nl

L. Florack e-mail: L.J.M.Florack@tue.nl

S. Meesters e-mail: S.P.L.Meesters@tue.nl

L. Smolders e-mail: L.Smolders@student.tue.nl

#### **1 Introduction**

Tractography aims at reconstructing bundles of nerve fibers in the brain (aka tracts) from diffusion weighted magnetic resonance imaging (DWI), the only neuroimaging technique enabling non-invasive in vivo imaging of the brain's fibrous structure. Unfortunately, persistent issues curb clinical progress [14, 53]. Decades after its inception [5, 7, 23, 30, 38, 47], lack of consensus and even skepsis as to its clinical feasibility prevail. This has sparked new incentives to specifically address current limitations in quantitative evaluation studies and international competitions [9, 12, 13, 37, 39, 51].

The geometric rationale for Diffusion Tensor Imaging (DTI), in its original form proposed by O'Donnell et al. [43] and, from a somewhat different perspective, by Lenglet et al. [31], and subsequently adapted by Fuster et al. [21], Hao et al. [24, 25], and several others, stipulates that one can 'geometrize away' local diffusivity patterns inside the brain. The idea is to incorporate anisotropic diffusivity of water in brain white matter, viewed as a porous medium with an orientational preference along axon bundles [57], into the intrinsic geometry of a suitably defined, curved manifold (akin to the geometrization of gravitational forces in general relativity theory). A similar approach is taken by Aumentado-Armstrong et al. [4], where a curved manifold is used to model conductivity of electrical signals in the heart muscles. In the case of DTI a Riemannian manifold presents itself, since its defining metric represents a positive definite quadratic form (or inner product) that can be formally mapped one-to-one onto the DTI tensor. Due to its modest performance, however, geodesic tractography has been largely abandoned in return for other approaches. A notable exception is the probabilistic approach by Hauberg et al. [26] and Schober et al. [55].

Our goal is to provide a versatile deterministic Riemann-DTI geometric paradigm for DTI geodesic tractography, revisiting original ideas (*loc. cit.*), which overcomes some of the main weaknesses, such as producing false positives (e.g. shortcuts) and false negatives (e.g. a shortcut is obscured by the correct solution). Since the mapping of DTI data to biologically meaningful tracts is generally ill-posed, we aim for a flexible metric equipped with control parameters. This admits adaptation to fiducial 'ground truth' tracts, in which data-extrinsic knowledge may be incorporated (e.g. via manual edits by an expert or via machine learning). The parameters control a locally smooth, spatially varying so-called '3-bein', or*triad*, detailed in Sect. 2 (cf. Savadjiev et al. [54] and Piuze et al. [48] for a similar idea in the context of myofiber geometry from DTI, based on Cartan's method of moving frames). In Sect. 3 we perform experiments to illustrate the theory, provide proof of principle, and present results in the context of a simple DTI-tractography phantom for the sake of illustration. There are no obstacles for application on more sophisticated simulated or real data.

#### **2 Theory**

Differential geometric approaches in DWI are not new [1, 3, 15, 20–22, 24, 25, 31– 33, 43, 46, 48]. The premise underlying the Riemann-DTI paradigm is that tissue microstructure imparts non-random barriers to water diffusion [8, 57]. The fibrous nature of brain white matter, comprising bundles of elongated axons connecting nerve cells in surrounding grey matter regions, facilitates mobility of water molecules along fiber directions. Since DTI captures the main diffusion anisotropy, it is natural to stipulate a Riemannian metric proportional to the inverse of the diffusion tensor. In this way, a relatively large mean free path is tantamount to a relatively short Riemannian distance, so that the problem of tractography can be related to a geodesic ('shortest path') problem. Candidate tracts can then be obtained by direct integration of the geodesic equations, by functional minimization of the Riemannian length (or related cost) functional for curves with fixed endpoints, or (with some care) by inference from the Hamilton-Jacobi equation [28, 45, 50, 52].

Despite the appealing heuristics supporting the geometric paradigm, there are serious caveats we need to take into consideration:


We will address both concerns and outline our strategy towards an improved framework.

Ad caveat 1: Geodesic completeness , tantamount to (huge) redundancy, may be used to our advantage, provided two conditions are met:


The first condition is our main hypothesis, the second one is a constitutional part of the geodesic tractography problem. The problem then boils down to finding 'the right' metric together with effective connectivity criteria for pruning its geodesics [2, 50, 56]. Streamline tractography, in its simplest form a singular limit of geodesic

<sup>1</sup>This fact also implies that 'ground truth' simulations must be interpreted with great care in order not to penalise experimental results that deviate from the stipulated ground truth used to create a phantom if such results are equally compatible with data evidence.

tractography based on a degenerate metric, lacks completeness, a direct consequence of its first order nature, which prohibits generic endpoint constraints. As a result, odds are that two fiducial endpoints (picked by an authoritative expert, say) fail to be connected, leaving us in a quandary how to repair for 'false negatives'. Clearly, geodesic completeness requires (at least) second order schemes.

Ad caveat 2: To allow for prior knowledge or retrospective corrections we do not determine a unique metric in terms of DTI data evidence a priori. Instead we aim for optimal control parameters for a *family* of metrics and induced geodesics in a joint (semi-)automated procedure. The premise is that, due to unknown microstructural factors, apparent diffusivities reflect fiber orientations at best qualitatively. By investigating the parameter unfolding of the family of metrics along with geodesic pruning we may investigate whether a stable result compatible with ground truth can be achieved (bearing in mind footnote 1). To this end we have conducted a feasibility study on the Fibercup [13, 49] simulator to clarify all conjectured features of our approach in a simplified context. There are no fundamental obstructions for application to more sophisticated phantoms or to real data, but this elaboration is left for future work.

For computational reasons one could employ any coordinate basis on the tangent bundle *T M* induced by an arbitrary coordinate map. Such a basis is commonly denoted by {∂*<sup>i</sup>* . =∂/∂*x<sup>i</sup>* }*i*=1,2,3. (A natural choice would be to employ the same coordinates as those given by a Cartesian coordinate frame of the associated Euclidean space.) However, given a suitable metric *gi j* , we may instead opt for a special, *g*-orthogonal (non-coordinate, or anholonomic) basis {*ea*}*a*=1,2,3. This is the triad alluded to in Sect. 1. If we write the new basis vectors in terms of linear combinations of the coordinate vectors,2

$$e\_a = e\_a^i \partial\_i \,, \tag{1}$$

then the coefficients *e<sup>i</sup> <sup>a</sup>* define the transformation matrix relating the general coordinate basis to the triad. We may define a new metric holor with entries *hab* relative to the triad by the standard change-of-basis formula:

$$
\hbar g\_{ij} e^i\_a e^j\_b = h\_{ab} \, . \tag{2}$$

The triad can be chosen so as to put *hab* into a convenient form. In order to illustrate this 'gauge fixing', let us assume that the dual metric *g*inv is identified with the DTI matrix *D*, as originally proposed. Take *h* =diag(1/λ<sup>1</sup>, 1/λ<sup>2</sup>, 1/λ<sup>3</sup>), in which the λ*<sup>a</sup>* are the eigenvalues of *D*. In the parlance of classical matrix theory [27], *ea* is then the eigenvector of *D* associated with λ*<sup>a</sup>*, which, in turn, defines a rank-one matrix *Za* =*ea* ⊗ *ea* ('Frobenius covariant'). Equation (2) is the geometrical counterpart of the classical Lagrange-Sylvester matrix decomposition, *D*=-3 *<sup>a</sup>*=<sup>1</sup> <sup>λ</sup>*<sup>a</sup> Za*, and the eigenvalues are our controls. We may vary the λ*<sup>a</sup>* and observe the effect on the

<sup>2</sup>The Einstein summation convention for identical upper/lower index pairs applies throughout in tensorial equations.

metric, induced geodesics, and ultimately fiber tracts. This parameter freedom unifies originally proposed models for the connection between DTI and Riemannian metric [21, 24, 25, 31, 43], since all instances can be obtained by slick (local or global) choice of λ*<sup>a</sup>*. This includes streamline tractography via singular perturbation theory.

To find a geodesic we fix seed and target points, *A* and *B* say, and minimize the length functional for curves connecting these points. This can be extended to any pair of regions provided one employs an efficient algorithm. Solving the Hamilton-Jacobi equation (with the help of fast marching methods) seems attractive in this respect, but, by design, provides only global minimizers, and is therefore not likely to solve the shortcut problem. Instead we opt for direct, coarse-to-fine minimization of the parametrization invariant functional

$$L\_g(\boldsymbol{\eta}) = \int\_{t\_A}^{t\_B} \sqrt{g\_{ij}(\mathbf{x}(t)) \dot{\mathbf{x}}^i(t) \dot{\mathbf{x}}^j(t)} \, dt \,, \tag{3}$$

in which

$$\gamma: [t\_A, t\_B] \to \mathbb{R}^3: t \mapsto \mathfrak{x}(t) \tag{4}$$

is an arbitrarily parametrized curve connecting *<sup>A</sup>*=*x*(*tA*) and *<sup>B</sup>* <sup>=</sup>*x*(*tB*) in <sup>R</sup><sup>3</sup> and *<sup>x</sup>*˙ denotes its derivative. We embed the metric in a multiresolution family, with scale parameter <sup>σ</sup> <sup>∈</sup>R+, according to the multiplicative scheme proposed by Florack et al. for positive symmetric matrices [16, 18], viz. if *g* is the Gram matrix with entries *gi j* , then

$$g(\mathbf{x}, \sigma) = \exp\left( (\phi\_{\sigma} \* \ln g)(\mathbf{x}) \right), \tag{5}$$

in which exp and ln are matrix-exp and matrix-log, φσ is the *L*1-normalised isotropic Gaussian kernel of scale σ [17, 19, 29, 34], and ∗ denotes entry-wise convolution. (This definition ensures that the dual of a blurred metric equals the blurred dual metric.) Since the metric can be shown to become Euclidean in the σ →∞limit under suitable, weak conditions, the asymptotic minimizer is the straight line connecting *A* and *B*. As one gradually decreases σ, this minimizer (geodesic at resolution 1/σ) is expected to deform likewise gradually, while adapting to the refined metric (at this level of rigor we ignore the problem of scale space bifurcations, cf. Damon [10, 11]). A higher resolution geodesic can then be found by adding a few control points roughly equidistantly along the curve in proportion to resolution, and minimizing the multivariate function of the control points representing our discretization of Eq. (3). Now suppose we have a (local) minimizer at some scale σ, then along with a scale refinement σ →σ −*d*σ (we take *d*σ ∝σ) we increment the number of control points along the curve, and seek a new minimizer in the vicinity of the previous one. In this way one can arbitrarily refine the geodesic curve (until grid scale, if needed), cf. Fig. 1.

The multiresolution scheme for geodesics sketched above applies to each fixed member of the (λ<sup>1</sup>, λ<sup>2</sup>, λ<sup>3</sup>)-family of metrics, and should not be confused with optimization with respect to the unfolding of this family as a function of (λ<sup>1</sup>, λ<sup>2</sup>, λ<sup>3</sup>). The details of this are given in the next section, notably Eq. (6).

**Fig. 1** Coarse-to-fine minimization of the length functional in Eq. (3) is implemented as an iterative multivariate function minimization in which the number of variables gradually increases in proportion to resolution. The control points represent a Bezier curve (blue) which by default we initialize by the unique Euclidean geodesic (dotted line), although this initialising curve may be manually overruled. In this example there are two control points, *P*<sup>2</sup> and *P*<sup>3</sup> (recall that the endpoints, *P*<sup>1</sup> and *P*<sup>4</sup> in this case, remain fixed). Note that the control points in general do not lie on the curve itself. These points are part of the 'behind-the-scenes' machinery in the minimization procedure and in themselves do not have any anatomical meaning. However, given *k* points in space, we may construct a Bezier curve through these points (although not uniquely). This may be useful for manual initialisation or correction by an expert, based on anatomical landmarks.

#### **3 Experiments**

In the following experiments we use an operational procedure to automatically select 'the right' (anisotropic) scaling of the control parameters λ*<sup>a</sup>* for the triad, relative to the original eigenvalues of the diffusion tensor. To begin with, we have opted for a single, global parameter ε∈(0, 1], introduced so as to rescale the diagonal metric *h*inv (the dual of *h* in Eq. (2)) by3 diag(1, ε, ε). The global character of ε allows us to leave λ<sup>1</sup> unscaled, since any scaling of λ<sup>1</sup> can be absorbed into ε. Effectively this yields an ε-parametrized family of Riemannian metrics *g*ε, replacing the unscaled metric *g*=*g*<sup>ε</sup>=<sup>1</sup> in Eqs. (2), (3) and (5). However, we only use this scaling whenever the fractional anisotropy (FA) [6] of the diffusion tensor is large enough (here we choose FA =0.15 as the ad hoc threshold) to ensure a well-defined main eigendirection.<sup>4</sup> Below this threshold the diffusion tensor is kept unscaled. As a result we obtain an ε-parametrized family of geodesics γε, cf. Eq. (4). These geodesics are computed one

<sup>3</sup>In principle, the control parameter triple (λ1, λ2, λ3) permits local adaptation leading to a 3 parameter family in every voxel. The singular limit ε=0 corresponds to streamline tractography and is excluded for its geodesic incompleteness. Values ε >1 may affect the ordering of the eigenvalues of *h* inv.

<sup>4</sup>This ad hoc anisotropy threshold calls for a more rigorously motivated alternative, but it serves our purpose in this feasibility study, viz. to ensure well-posedness.

by one by the coarse-to-fine scheme outlined at the end of Sect. 2 for each fiducial pair of endpoints and each setting of ε.

Subsequently each geodesic thus obtained is quantitatively evaluated in terms of the tract length unbiased, nonlinear connectivity functional [2, 50, 56]

$$C(\boldsymbol{\chi}\_{\boldsymbol{\epsilon}}) = \frac{L\_{\eta}(\boldsymbol{\chi}\_{\boldsymbol{\epsilon}})}{L\_{g}(\boldsymbol{\chi}\_{\boldsymbol{\epsilon}})} \,, \tag{6}$$

in which η denotes the standard Euclidean metric (so that *L*η(γ )is the usual length of γ ), recall Eq. (3). Note that in the denominator we use the unscaled diffusion metric *g*=*g*<sup>ε</sup>=1, for which the curve γε is, in general, not a geodesic (unless ε=1). Using the scaled length *g*<sup>ε</sup> in the denominator would entail a bias towards ε = 1. Each geodesic γε will almost surely not correspond to a streamline (unless both endpoints are on the streamline), which implies that there exists a *t* ∈ (0, 1) such that the tangent vector γ˙ε(*t*) has a component in the direction of a non-principal eigenvector of the diffusion tensor. Since this component scales with ε−1, the connectivity would vanish for ε → 0 even if the geodesic remains (approximately) the same curve. Instead, the ε control parameter merely serves to single out an optimal, metric-compatible geodesic through a suitable anisotropic scaling of the metric tensor. The connectivity criterion for the latter is based on data evidence, viz. average apparent diffusivity, which should not involve ε. Connectivity *C*(γε) is thus some average measure of diffusivity along γε. Parameter values

$$
\varepsilon^\* \in \operatorname\*{argmax}\_{\iota \in \{\delta, 1\}} C(\chi\_{\iota})\,,\tag{7}
$$

for some fixed 0< δ 1, at which *locally* <sup>5</sup> optimal connectivity is attained, are tractspecific sharpening parameters, affected in a still unknown manner, by underlying microstructure. The corresponding tracts γε<sup>∗</sup> are called *optimal tracts*.

As illustrated in Fig. 2 for the Fiberfox-reconstructed Fibercup phantom with added Riccian noise [40] (cf. Fillard et al. for a detailed description [13] of the original Fibercup) only optimal tracts are retained. Since it is known that the inverse metric has certain shortcomings (e.g. see [24]), we also employed our automated procedure in combination with one of the other metric proposals in the literature, namely the adjugate metric [21, 22]. We chose this one, because it is derived from first principles of the underlying diffusion process and in addition it's very easy to implement. In Fig. 2, the optimal tracts found for the inverse (bottom-left) and adjugate DTI metric (bottom-right) emerge for different optimal parameter values, but are, remarkably, virtually indistinguishable. As the adjugate DTI metric is essentially a *local* isotropic scaling of the inverse DTI metric, it is a noteworthy feature that we are able to obtain essentially the same tracts in both cases with a family of metrics characterised by a single, *global* parameter. After all, the governing geodesic equations for the inverse and adjugate are not the same. In practice, a broad range of parameter values (the high plateaux in the graphs) can be used to single out essentially the same tracts, making this procedure robust with respect to the precise choice of ε. The experiment shows that, in all cases and by virtue of proper placement of endpoints, correct tracts

<sup>5</sup> We define argmax*<sup>x</sup> <sup>f</sup>* (*x*) . = {*x* | *f* (*y*) ≤ *f* (*x*) for all *y* in a neighbourhood of *x*}.

**Fig. 2** (Top-left) Ground truth tracts (for various seed points); illustration adapted from Fillard et al. [13]. (top-right) Tracts obtained by using the unscaled inverse (red), respectively adjugate (blue) DTI tensor as the default metric for geodesic tractography. Both metrics produce shortcuts (false positives) instead of following the U-shape fibers, leaving false negatives. (middle-left) Pertract connectivity for the <sup>ε</sup>-parameter unfolding of the inverse DTI metric, *<sup>g</sup>*<sup>=</sup> *<sup>D</sup>*inv. (middle-right) Idem for the adjugate DTI metric, *<sup>g</sup>*=det *DD*inv. Colors correspond to those used for the tracts in the bottom two figures. The red dots indicate the tract-specific parameter values ε<sup>∗</sup> ∈(0, 1] for the metrics used for each optimal tract. This optimal parameter is not affected by a suitable (global) normalization of *g*, meaning that the absolute value of the connectivity is inherently meaningless. It is, however, the relative value that determines the optimal parameter. Note the logarithmic scale for ε. (bottom-left) Optimal results obtained by automatic optimization for inverse DTI metric ansatz. (bottom-right) Optimal results obtained by automatic optimization for adjugate DTI metric ansatz. The latter two results are virtually indistinguishable, but optimality is obtained at different control parameter settings.

**Fig. 2** (continued)

are found automatically, and, in particular, that shortcuts do not occur. To assess the robustness to placement of endpoints, in Fig. 3 the optimal tracts are computed for pairs of perturbed endpoints, demonstrated for the adjugate DTI metric.

Low connectivity typically corresponds to shortcuts. Figure 4 shows optimal tracts for each pair of corresponding (ground truth) seed and target points (top), as well as connectivities for optimal tracts between a selection of seed point and all remaining target points (bottom). For each pair of possible seed/target points, the optimal tract was determined by Eq. (7). Figure 4 shows two representative cases: one in which the stipulated ground truth yields the highest connectivity and a second one in which it is among the top but is still dominated by another tract.

Multiple local maxima of *C*(γε) generally exist, occurring at ε∈ {ε<sup>∗</sup> <sup>1</sup> ,...,ε<sup>∗</sup> *<sup>k</sup>* } say, corresponding to *k* plausible tracts (recall Eq. (7) and footnote 5) that invariably include the stipulated 'ground truth', although the latter does not necessarily correspond to the *global* maximum, recall Fig. 4b. Figure 5 illustrates this case by another simulation (based on Fiberfox [40]), emblematic of an ambiguous configuration of kissing or crossing fiber tracts . Both interpretations are indeed confirmed, in the sense of corresponding to local optima of *C*(γε). Our current coarse-to-fine implementation, however, cannot handle scale space bifurcations of geodesics automatically, and, due to our default initialization by the Euclidean geodesic, automatically zooms in on the kissing tract. The crossing tract requires a (simple) manual edit to overrule the default initialization and encourage convergence to another solution. This edit and its effect is shown in Fig. 5, and further illustrated in Fig. 6. Here the default initialization by the Euclidean geodesic has been replaced by a quadratic Bezier curve forced to pass through a manually selected inclusive ROI called an AND gate (green circle), while connectivity maximization for this control point has been constrained to this neighbourhood. Note that, for the inverse as well as the adjugate diffusion tensor ansatz, Fig. 6, boundary extrema γε are found near ε=1, illustrating the same inclination to produce shortcuts as γ<sup>1</sup> for the unscaled metric. Closer to the optimal ε<sup>∗</sup> ∈argmax<sup>ε</sup>∈[δ,1] *C*(γε) we observe correct convergence, though.

**Fig. 3** For each endpoint of the optimal green curve (cf. Fig. 2) the red points are generated from a multivariate normal distribution centered around the endpoints with covariance matrix Σ = diag(0.5, 0.5). The blue tracts are the optimal tracts for pairs of such perturbed endpoints by using the adjugate DTI metric ansatz. The green curve has a connectivity value of 2.749 and the mean connectivity of all perturbed tracts is 2.708 with a standard deviation of 0.037. Visually, the green curve is a good representative of this bundle of tracts as a whole and its connectivity is approximately one standard deviation away from the mean.

In Fig. 7, the left Cortical Spinal Tract of the ISMRM 2015 Tractography challenge data [35] is used to illustrate the operational procedure on a 3D dataset. Quantitative measures used in the challenge are the overlap (OL), overreach (OR) and *F*<sup>1</sup> scores, see Côté et al. [9]. OL is defined as the number of voxels through which both the reconstruction and the ground truth pass, normalized by the number of ground truth voxels. OR is the number of voxels through which the reconstruction passes but the ground truth does not, normalized by the number of reconstructed voxels. These two quantities can be interpreted as the true positive rate and the false discovery rate. The *F*<sup>1</sup> score is the harmonic mean of OL and 1 − OR. Our tractography results in OR = 0.324, OL = 0.375, leading to *F*<sup>1</sup> = 0.427, ranking it in the top 40% of the submitted challenge results based solely on *F*<sup>1</sup> scores. It is particularly noteworthy that this DTI-based procedure has a performance comparable to HARDI type methods, outperforming other DTI-based methods applied to the CST [36].

**Fig. 4** (Top) Seed and target point labeling: s*<sup>i</sup>* = start of tract, e*<sup>i</sup>* = end of tract; the attached number *i* ∈ {1,..., 7}, refers to the tract label. (bottom) Box-and-whisker plot of connectivity in the adjugate DTI metric versus seed point. For the seed points s1 and s2 the boxplot above it shows the optimal connectivity of that seed point to every other target point s*j* and e *j* . Endpoints that yield the highest connectivity are indicated on top on the boxplot. The red dots indicate the connectivity value for the stipulated ground truth, i.e. the tract running from s*i* to e*i* , for each tract label.

**Fig. 5** Simulation with kissing/crossing fibers. Seed and target points are invariably taken to lie in upper and lower left corner. Note that there are two plausible solutions. The left two figures pertain to the (connectivity pruned) ε-family induced by the inverse diffusion metric, the right two figures to that of the adjugate diffusion metric. In either case, our automatic procedure only finds one solution (blue curves). (top) The equally viable crossing scenario is due to the coarse-to-fine scheme with default initialization (red dotted Euclidean geodesic in the upper two figures). (bottom) By employing a restricted neighbourhood search region (AND gate, green circle) with suitable initialization (red dotted quadratic Bezier curve), the algorithm converges to the blue crossing geodesics in the bottom two figures, resolving the false negative result.

**Fig. 6** Geodesics found by unfolding of the Riemannian metric family *g*ε, Eq. (7) in combination with connectivity optimization, Eq. (6) for inverse (left) and adjugate (right) diffusion tensor ansatz *g*=*g*1, the AND gate of Fig. 5 (green circle). Initialization is given by the quadratic Bezier curve in Fig. 5. Red-to-blue rainbow color coding shows the connectivity evolution as a function of ε, with red indicating low connectivity tracts and blue indicating the (unique) connectivity optimized result for ε<sup>∗</sup> =argmaxε∈[δ,1] *C*(γε).

#### **4 Conclusion and Discussion**

We have proposed a modified Riemann-DTI geodesic tractography framework. The modification entails the embedding of a DTI induced metric into a family of metrics designed for the purpose of tractography, furnished with an operational scheme for tract-specific optimization of metric and geodesic(s) for any given pair of endpoints based on a nonlinear connectivity functional.

In a simple experiment based on the Fibercup simulator we conclude that all local connectivity maxima correspond to plausible tracts that consistently follow high diffusivity pathways and avoid, as much as logically possible, inconsistent regions. For instance, despite a rather simplistic connectivity measure, all ground truth tracts are reconstructed provided seed and target points are (roughly) correctly placed, i.e. in such a way that a plausible fiber does indeed exist, recall Fig. 2. The robustness of our procedure with respect to the placement of endpoints is demonstrated in Fig. 3, where tracts reconstructed using slightly perturbed seed and target locations, remained in close proximity to the original one.

It is essential, in this respect, to appreciate the ill-posed nature of tractography. Although the generation of diffusivity patterns from a given tract configuration is robust (albeit model-dependent), this process is not invertible. The equivalence class of tract configurations consistent with a given diffusivity pattern may contain many 'metamers', so to speak. The actual power of the proposed method is precisely its ability to generate multiple representatives from this equivalence class, so that no a priori bias is introduced. In clinical practice this could help an expert to reject

**Fig. 7** Comparison of the Left Cortical Spinal Tract (CST) between the ground truth (red) and connectivity-optimal geodesics according to Eqs. (6) and (7) in which the base metric is the adjugate DTI-metric. The geodesics are color-coded according to their connectivity value, which increases from green to blue. The depicted views are labeled with S(uperior), A(nterior) and L(eft) to determine the plane and orientation of the cross-section. On visual inspection of the multiple views of the CST, the optimal geodesics lie close to the ground truth. Using more refined connectivity measures or a criterion to remove single tracts, the results can be improved. This can relieve us from those spurious tracts that are seemingly leaving the bundle and clearly have a lower connectivity than the rest of the tracts. A selection of 3000 ground truth tracts were extracted from the CST to determine seed and target points for the geodesics.

false positives (if not eliminated automatically), based on data-extrinsic anatomical insight, in favour of selecting one of the remaining plausible candidate tracts, i.e. without being confronted with false negatives. Ideally such candidates present themselves, viz. as one of the local optima actually found by our automatic scheme.

False negatives may nevertheless occur if local maxima are missed due to inadequate numerical optimization. We have indicated how this problem may be mitigated through a simple manual edit by an expert. The delineation of an AND gate through which one believes tentative tracts of interest should pass, or the sketch of a coarse initialization curve between the endpoints, overruling the default straight line segment, both make it more likely to drive numerical optimization to the correct local maximum.

To prevent scaling of the diffusion tensors in a non-sensical manner, their eigenvalues are only affected whenever an FA threshold was exceeded in order to ensure a well-defined main eigendirection of the diffusion tensor. This treshold was based on experiments with the Fiberfox-reconstructed Fibercup phantom, but needs to be adjusted for tractography on real data. This ad hoc approach should be considered a first stepping stone towards a more refined technique combining the scaling of eigenvalues with the anisotropy of the tensor and the noise level of the data. Highly anisotropic tensors have a relatively large first eigenvalue and may tolerate a more severe scaling, since they provide more evidence for an articulated orientation. Since many low FA regions contain complex fiber structures, not scaling the DTI tensors there may influence the shape of the geodesics. In these areas we do not expect to get accurate results, since DTI is not able to resolve complex fiber structures adequately. However, scaling of the tensors enhances any noise present in the data and may increase the number of false positives, especially when the signal-to-noise-ratio is very low.

We stress that 'most likely' fiber candidates are obtained for *any* pair of endpoints in terms of local optimality of connectivity. Recalling Fig. 4, note that even though tract s2-e2 has a high connectivity value, maximal connectivity is attained by the tract s2-s5. This effect can be ascribed to the construction of the family of metrics *g*<sup>ε</sup> as well as the conservative nature of the connectivity measure in Eq. (6). At the intersection of two bundles the diffusion tensors are (nearly) isotropic, leading to similar diffusivities in all directions. This isotropy (FA ≤ 0.15) prevents anisotropic rescaling of the metric, as explained. Moreover, as the connectivity measure only accounts for average diffusion along a tract and does not penalize curvature, a bend in the tract does not decrease its connectivity, as the latter is based on diffusivity, not geometry. These effects combined result in a connectivity-optimal tract which differs from the ground truth one. This needs to be remedied by using more sophisticated connectivity measures, which will be application dependent and may require prior knowledge about the bundles of interest. Our generic connectivity measure has been motivated mainly by our desire to preserve true positives, not as a criterion to remove false positives, allowing a modular approach. Additional criteria for pruning false positives connecting a fiducial seed point to a volumetric target region (possibly the whole brain) will be studied in future work, including appropriate extensions of the proposed coarse-to-fine tractography algorithm.

In the ISMRM Tractography challenge reconstructed tracts are interpreted as centerlines of a fiber with a thickness/resolution of 1 mm isotropic voxel. The OL, OR and *F*<sup>1</sup> scores were calculated based on such 1 mm voxels [36]. The tractography, however, was performed on the diffusion weighted images consisting of 2 mm isotropic voxels. This way we can only resolve detail up to the grid scale of 2 mm and quantification measures should take this resolution difference into account when using the affine transformation to convert between the two different spaces. By downscaling this minimal resolution of 1 mm to 2 mm we increase the *F*<sup>1</sup> score to 0.526, ranking in the top 20% (as the only DTI based algorithm).

In this experiment details could be resolved up to grid scale, but if one would consider the effects of noise (e.g. in the acquisition process), this may lead to a different effective minimal resolution, determined by grid scale and noise level. The influence of different kinds of noise (e.g. acquisition noise, seeding errors) needs to be addressed in future studies in which the control-triad framework is combined with uncertainty quantification.

These experiments confirm our main conjecture that the potential of DTI has not yet been fully exploited in tractography.

**Acknowledgments** This work is part of the research programme 'Diffusion MRI Tractography with Uncertainty Propagation for the Neurosurgical Workflow' with project number 16338, (partly) financed by the Netherlands Organisation for Scientific Research (NWO).

The work of Andrea Fuster is part of the research programme of the Foundation for Fundamental Research on Matter (FOM), which is financially supported by the Netherlands Organisation for Scientific Research (NWO).

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

**Measuring Anisotropy**

## **Magnetic Resonance Imaging of** *T***2- and Diffusion Anisotropy Using a Tiltable Receive Coil**

#### **Chantal M. W. Tax, Elena Kleban, Muhamed Barakovi´c, Maxime Chamberland, and Derek K. Jones**

**Abstract** The anisotropic microstructure of white matter is reflected in various MRI contrasts. Transverse relaxation rates can be probed as a function of fibre-orientation with respect to the main magnetic field, while diffusion properties are probed as a function of fibre-orientation with respect to an encoding gradient. While the latter is easy to obtain by varying the orientation of the gradient, as the magnetic field is fixed, obtaining the former requires re-orienting the head. In this work we deployed a tiltable RF-coil to study *T*2- and diffusional anisotropy of the brain white matter simultaneously in diffusion-*T*<sup>2</sup> correlation experiments.

C. M. W. Tan and E. Kleban share first authorship.

C. M. W. Tax (B) · E. Kleban · M. Chamberland · D. K. Jones Cardiff University Brain Research Imaging Centre (CUBRIC), Cardiff University, Cardiff, UK e-mail: TaxC@cardiff.ac.uk

E. Kleban e-mail: KlebanE@cardiff.ac.uk

M. Chamberland e-mail: ChamberlandM@cardiff.ac.uk

D. K. Jones e-mail: JonesD27@cardiff.ac.uk

M. Barakovi´c

Cardiff University Brain Research Imaging Centre (CUBRIC), Cardiff University, Cardiff, UK e-mail: muhamed.barakovic@epfl.ch

Signal Processing Laboratory 5, Ecole Polytechnique Federale de Lausanne, Lausanne, Switzerland

Translational Imaging in Neurology Basel, Department of Biomedical Engineering, University Hospital Basel, Basel, Switzerland

#### **1 Introduction**

#### *1.1 Background*

**Magnetic Resonance Imaging (MRI)** allows us to probe structural anisotropy of tissue *in vivo* by studying the magnetic properties and translational motion of for instance hydrogen protons. In the human body, hydrogen is naturally abundant in various compounds, with the highest MR signal amplitude detected from hydrogen in water molecules. Hydrogen protons possess spin angular momentum, which is an intrinsic quantum property that allows the occurrence of magnetic interactions and resonance. When placed in a magnetic field, there is a slight preference for spins to be aligned with the field, resulting in a net magnetisation **M** aligned with the main magnetic field **B**<sup>0</sup> [1]. Upon the application of a radiofrequency field at the Larmor frequency **B**1, **M** can be tipped out of alignment with **B**0; most commonly into the perpendicular plane. The measured signal is the result of the ensemble of spins precessing coherently in the plane, and signal loss (decay) occurs when such coherence reduces.

This signal decay, which results from a progressive loss of coherence of precessional phase, i.e. *dephasing*, is also called spin-spin relaxation. The spin-spin relaxation rate is usually denoted by *R*<sup>2</sup> = 1/*T*2, where *T*<sup>2</sup> is the time taken for the magnetization to decay to 1/*e* of its initial value [2, 3]. In addition to the irreversible spin-spin relaxation, dephasing can be caused by local variations in the magnetic field, a reversible process if the spins are static. Such local variations in the magnetic field arise from a difference in interaction of different substances with the magnetic field (*susceptibility effects*). If the spins experience Brownian molecular motion, they will experience various magnetic field strengths, and their dephasing due to local field changes will be effectively irreversible. Hence, the measured apparent *T*<sup>2</sup> of signal decay will no longer be purely induced by dephasing due to spin-spin interactions, but will depend on the amplitude and spatial characteristics of the local field variations, and the mean displacement of molecules per unit of time due to incoherent motion.

In diffusion-weighted MRI, additional (and typically much stronger) magnetic field variations are induced intentionally by applying *magnetic field gradients* which cause the strength of the main magnetic field **B**<sup>0</sup> to vary linearly in space [4–6]. As such, the Brownian molecular motion can be encoded in the signal in a controlled way.

The interplay of susceptibility and diffusion effects leads to the anisotropy of tissue being reflected in different MRI contrasts and hence the combination of contrasts can give a more complete picture of tissue microstructure. In the following paragraphs we will describe anisotropy in the brain and these processes in more detail.

**Structural anisotropy of human brain tissue**. The dominant tissue exhibiting anisotropy in the brain is the *white matter* (WM). It is predominantly composed of the long extensions of neuronal cells—the axons, which are are grouped into *fibre bundles* and inter-connect different areas of the brain. The main function of axons is to transmit electric impulses between and within brain areas. Axons can be insulated by a *myelin sheath* which is formed of lipid chains and allows a faster signal transmission. The size, density, length of the axons and their myelination levels vary with age and may alter with pathology. Therefore, investigation of the white matter microstructure is of high importance in understanding the functionality of the healthy brain, but also in studying the mechanisms of normal/abnormal development and pathology.

**Diffusion effects**. Measurements of Brownian molecular motion in biological tissue can reflect not only the temperature and the viscosity of the medium it is occurring in, but most importantly will be sensitive to the underlying geometry and anisotropy of the tissue. For instance, in the brain white matter water molecules can propagate much easier along fibre bundles than perpendicular to them because of obstacles such as the cell membrane and myelin [7]. This property can be used to estimate the main orientation of fibre bundles and their virtual reconstruction by means of *fibre tractography* [8, 9]. Additionally, if water molecules are trapped inside the axon and cannot penetrate the boundaries (restricted diffusion), the mean displacement perpendicular to the axon will be similar to its diameter at long diffusion times. The diffusion of water molecules which reside outside axons is commonly thought of as not being fully restricted but hindered. An example of the differences between the movement of water molecules residing inside and outside of axons is visualised in Fig. 1a.

In MRI, Brownian motion of water molecules is most commonly encoded using a pair of pulsed magnetic field gradients, a dephasing and a rephasing gradient [6] (Fig. 2). If spins are stationary, these gradients would have no additional effect on the signal decay. However if spins change their positions during or between the application of the gradients, the rephasing will be incomplete which will result in signal loss. The signal loss due to diffusion can be enhanced by increasing the magnitude of the gradients, the time during which the gradients are on, and/or the time between the gradients. The strength of the diffusion weighting is described by the *b*-value; a parameter which combines the information on the diffusion gradient strength and timings.

**Magnetic susceptibility effects**. Any material placed inside a strong magnetic field interacts with it—it can become magnetised itself. The proportionality constant linking magnetic field strength and the magnetisation induced inside the material is called *magnetic susceptibility*. At the boundaries between materials with different magnetic properties the magnetic flux density is spatially inhomogeneous and its distribution will depend on the boundary orientation to the magnetic field and the difference in susceptibility between the materials. Additionally, the magnetisation induced in some materials may also depend on the orientation of the sample to the magnetic field, i.e. the magnetic susceptibility of those materials is anisotropic.

As mentioned above, the nerve fibres in human brain are insulated by myelin sheath. Myelin is more diamagnetic than water, i.e., it is repelled more strongly by a magnetic field. Additionally, several studies suggested that the magnetic suscepti-

**Fig. 1** A white matter nerve fibre can be modelled as a hollow cylinder composed of a myelin sheath. **a** Brownian motion prependicular to the nerve fibre is restricted inside the cylinder (the molecules are "trapped" inside the cylinder) and hindered outside of it (the mean displacement of the molecules per time-point is larger, but nevertheless lower than 'free' water). **b** The myelin sheath is more diamagnetic than the surrounding tissue and perturbs the local magnetic field. The field perturbation is inhomogeneous and depends on the nerve fibre orientation to the magnetic field **B**<sup>0</sup>

**Fig. 2** Pulsed-gradient spin-echo sequence: the echo time TE is defined as the time between the radio-frequency pulse and the centre of the spin-echo. The diffusion-weighting strength, the *b*-value, can be calculated as *<sup>b</sup>* <sup>=</sup> <sup>γ</sup>2*g*2δ2(-− δ/3)

bility of the myelin sheath has an anisotropic component [10–12]. It has been shown that the signal decay from white matter varies as a function of orientation to **B**<sup>0</sup> [13–20] and can be well explained using a hollow cylinder fibre model (Fig. 1b) [17] and is often represented by a 3-pool model [18, 21, 22].

The strongest contribution to this signal anisotropy arises from the water trapped between the layers of myelin sheath, called the myelin water. However, the myelin water signal decays very quickly, on account of its short *T*<sup>2</sup> (∼10–30 ms), and is usually negligible at the echo times used in a typical diffusion-weighted MRI experiment. Nevertheless, it has also been observed that signal decay rates may still be orientation-dependent even without this myelin-water component [13, 20, 23–25].

#### *1.2 Scope of This Work*

In myelinated white matter it has been reported that *T* <sup>∗</sup> <sup>2</sup> (where 1/*T* <sup>∗</sup> <sup>2</sup> = 1/*T*<sup>2</sup> + 1/*T* 2 and *T* <sup>2</sup> can be understood as capturing the reversible effects of field inhomogeneities) depends on the orientation of the fibre with respect to **B**<sup>0</sup> due to microscopic susceptibility effects [13, 16, 18, 19, 26]. Orientation-dependence of *T*<sup>2</sup> was also reported recently [20, 23, 24, 27] and could potentially characterise microscopicsusceptibility more reliably and reflect effects related to axon diameter [24, 28]. Experiments designed to probe relaxation-anisotropy commonly involve reorienting the head inside the scanner, and are thus challenged by unintended signal-to-noise ratio (SNR) variations across orientations caused by differences in proximities to the receiver coil, and by increased susceptibility to motion and artefacts due to patientdiscomfort. In this work, we re-purpose a tiltable RF coil (originally designed for patient comfort) to investigate *T*2-orientational dependence within the context of a diffusion-*T*<sup>2</sup> correlation experiment. The coil can be tilted around the left-right axis by 0◦, 9◦ and 18◦ to **B**0, which: (1) minimises patient-discomfort and thus improves reliability; (2) offers a new degree of freedom as tilting around the left-right axis is otherwise difficult to achieve; (3) fixes the coil-to-brain distance across orientations and thus reduces SNR variations; and (4) increases the reproducibility of the experiment.

Finally, instead of studying global variations across the whole brain volume, we adopt an along-tract profiling *tractometry* approach (i.e., which is the mapping of measures along pathways reconstructed with tractography [29, 30]) to assess spatial variations in more detail.

#### **2 Methods**

#### *2.1 Data Acquisition*

The study was approved by the Cardiff University School of Psychology Ethics Committee and written informed consent was obtained. Two healthy volunteers (female, 30 y.) were scanned on a 3 T 300 mT/m Connectom scanner equipped with a modified 20-channel head/neck tiltable coil (Siemens Healthcare, Erlangen, Germany). Each subject was in supine, head first position and the direction of the magnetic field **B**<sup>0</sup> was along the superior-inferior radiological axis. MRI data were acquired in the default (0◦) and tilted (18◦) orientations of the tiltable coil (Fig. 3a, b). One of the subjects underwent a second scan in the default head orientation to examine test-retest variability.

Diffusion-*T*<sup>2</sup> correlation data were acquired using a pulsed-gradient spin-echo echo-planar-imaging (PGSE-EPI) sequence [6] (Fig. 2), with different echo-times TE to probe *T*2, and diffusion-weighting strengths or *b*-values to probe diffusion, (Fig. 3c). The timings of the diffusion encoding gradients were fixed for all echo

**Fig. 3 a** The coil in default (0◦) and tilted (18◦) position. **<sup>b</sup>** *<sup>b</sup>* <sup>=</sup> 0 s/mm<sup>2</sup> images at different TE. **c** Acquisition parameters for the diffusion-correlation experiment

times. The signal in voxel **x** can then be denoted as *S***x**(*b*, TE). Additional *b*0-images were acquired in the halfway-tilted (9◦) position (not shown). Remaining parameters were repetition time TR 3.5 s and voxel size 3 × 3 × 3 mm3.

#### *2.2 MRI Signal Processing*

The diffusion-*T*<sup>2</sup> data were preprocessed to correct for subject motion, eddy current effects, Gibbs ringing and gradient non-linearities [31–33] for each subject and each head orientation. Spatial correspondence between the tilted and default head orientations was obtained in two ways: 1) by nonlinear registration [34] to the halfway-tilted (9◦) space, and 2) by a tractometry approach in native space.

The tractometry [29] approach in native space of each head orientation relied on the quantitative mapping of measures along reconstructed brain pathways. First, fibre orientation distribution functions (fODFs) [35, 36] were estimated at each voxel using multi-tissue multi-shell constrained spherical deconvolution (MSMT-CSD) [37]. For each coil position, peaks were extracted from the resulting fODFs and used as input to perform streamline tractography on the TE = 54 ms data. Bundles were automatically segmented [38], a representative core-streamline was computed [39], and the bundles were subsequently subdivided [40, 41] into n = 20 segments (*s*)

**Fig. 4** Extracted brain pathways of interest from subject 2. Bundles were segmented into 20 sections (0: ligth-blue, 20: dark-brown). CC: corpus callosum, UF: uncinate fasciculus, CST: cortiospinaltract, IFOF: inferior fronto-occipital fasciculus, ILF: inferior longitudinal fasciculus. Visualisation was performed using FiberNavigator [43]

(Fig. 4). Voxels with a single-fibre-population were identified [42] and assigned to *si* if their location was inside the segment and their orientation was within 30◦ of the orientation of the core-streamline in that segment. Note that only tract-segments with minimal fanning (assessed visually) were considered. *T*<sup>2</sup> values were then profiled for each bundle independently by taking the mean and standard error of the mean within each tract segment.

#### *2.3 Estimation*

SNR estimates were obtained from the background of the images acquired at TE = 54 ms and *b* = 0 s/mm<sup>2</sup> for both the (0◦) and (18◦) coil-orientations [44]. The voxelwise *T*<sup>2</sup> was estimated from the *b*0-signals as *S*(0, TE) = *S*(0, 0)*e*−TE/*T*<sup>2</sup> , using a nonlinear least-squares trust-region-reflective algorithm in Matlab.

Fibre orientation θ to the main magnetic field **B**<sup>0</sup> was estimated for the voxels with single fibre population. With prior knowledge of θ orientation-dependent (anisotropic) and -independent (isotropic) components of *R*<sup>2</sup> can be estimated as follows [20, 45]: *<sup>R</sup>*2(θ) <sup>=</sup> *<sup>R</sup>*<sup>2</sup>,isotropic <sup>+</sup> *<sup>R</sup>*<sup>2</sup>,anisotropic · sin<sup>4</sup> <sup>θ</sup>.

#### **3 Results**

**Signal-to-noise ratio**. The estimated noise standard deviation was similar for the two coil-orientations. Figure 5a compares the signals in the default and the tilted position after registration to the common space; the signal (and thus SNR) between coilorientations varied within a similar range as the range seen between test-retest scans in the same orientation. Globally, the images are aligned but some local misalignments can still be observed.

**Voxel-wise** *T*2-**estimates**. Figure 5b highlights differences in estimated *T*<sup>2</sup> for different coil-orientations. Globally, the difference in *T*2-values estimated from the data in default and tilted head orientations is larger than or equal to *T*2-values estimated from test-retest scans in default head orientation for subject 1 (Fig. 5b, left half). Local differences in *T*<sup>2</sup> can be observed between the default and tilted position, as indicated by red arrows in the *T*2-maps in Fig. 5b, for example. The inverse *T*2-values in white matter mostly range between 9 s−<sup>1</sup> and 21 s−<sup>1</sup> for both subjects and head orientation.

**Fig. 5 a** *<sup>b</sup>* <sup>=</sup> 0 s/mm2 image for the default (0◦) and tilted (18◦) orientation registered to the halfway-tilted (9◦) space, and their difference (tilted—default). σˆ is the estimated standard deviation in the background of the image. **b** Estimated *T*<sup>2</sup> for the default (0◦) and tilted (18◦) orientation registered to the halfway-tilted (9◦) space, and their difference. Red arrows indicate regions of visible difference

**Fig. 6** Relaxation rate 1/*T*<sup>2</sup> as a function of the fibre-orientation θ with the main magnetic field, in the default (left) and tilted (right) position. Each point represents a single-fibre-population voxel and is colour-coded according to its orientation (red, blue, and green correspond to the left-right, superior-inferior and anterior-posterior axis, respectively). The black line represents a model-fit of 1/*T*<sup>2</sup> as a function of θ as described in previous literature and references therein [13, 19, 20]

**GlobalWM isotropic and anisotropic** *T*2-**stimates**. The total isotropic (orientationindependent) component of relaxation rate of *R*<sup>2</sup> = 1/*T*<sup>2</sup> in WM was estimated at 13.7 ± 0.2 s−<sup>1</sup> and 13.6 ± 0.2 s−1, in default and tilted head orientations, respectively, for one of the subjects (Fig. 6). For the same subject the total WM anisotropic components of the inverse *T*<sup>2</sup> were 1.6 ± 0.25 s−<sup>1</sup> and 1.7 ± 0.25 s−1, for default and tilted head positions, respectively. The range of *T*2-values in white matter and their total isotropic and anisotropic components are consistent with previously reported values in [20, 45].

**Tractometry analysis**. Figures 7, 8 and 9 show the along-tract profiles of the estimated *T*<sup>2</sup> (top plot) and angle w.r.t. **B0** (bottom plot) for different tracts. Globally, the angular profiles show comparable characteristics between subjects in the default and tilted position. For Subject 1, the angular profiles remain similar in the default and default-retest acquisition, except for the inferior parts of the corticospinal tract (CST).

Profiling of *T*<sup>2</sup> in the CST, which runs along the *z*-axis (i.e., inferosuperior), reveals a significant increase in *T*2, for example in segment 16 which experiences a change in orientation w.r.t. **B0** from ∼35◦ in the default position to ∼55◦ in the tilted position (Fig. 7). This is in the regime where the derivative of the relaxation rate 1/*T*<sup>2</sup> as a function of angle is the largest (Fig. 6). The uncinate fasciculus (UF) shows a noisier pattern, likely because the number of single-fibre voxels per segment is generally lower (∼5−10 in the UF compared to ∼15−20 in the CST).

**Fig. 7** Tractometry results (mean and standard error) for the corticospinal tract (CST, top plots) and the uncinate fasciculus (UF, bottom plots). *T*2-values and fibre orientation to **B**<sup>0</sup> are plotted against segment numbers for each fibre tract and subject. Colour bar indicates tract-segment location, corresponding to the colour-encoding on the visualised tracts. Note that only tract-segments with minimal fanning (assessed visually) were considered

In the inferior longitudinal fasciculus (ILF) (Fig. 8) a global decrease in angle w.r.t. **B0** from default to tilted position leads to an overall, yet subtle, increase in *T*2. In the inferior fronto-occipital fasciculus (IFOF) the pattern is less clear.

In the callosal midbody the angle w.r.t. **B**<sup>0</sup> remains relatively unchanged and so does *T*2.

#### **4 Discussion**

In this work we have incorporated a tiltable coil in *T*2-diffusion-correlation experiment to modulate the white matter fibre orientation with respect to the main magnetic field **B**0.

**Fig. 8** Tractometry results (mean and standard error) for the inferior fronto-occipital fasciculus (IFOF, top plots) and the inferior longitudinal fasciculus (ILF, bottom plots). *T*2-values and fibre orientation to **B**<sup>0</sup> are plotted against segment numbers for each fibre tract and subject. Colour bar indicates tract-segment location, corresponding to the colour-encoding on the visualised tracts. Note that only tract-segments with minimal fanning (assessed visually) were considered

We observed changes in the *T*2-tract-profile after the participants' heads were re-oriented in the scanner, with up to ∼10 ms difference in *T*2-values between the default and the tilted head orientation. The test-retest *T*2-tract profiles in the default head position are more similar to each-other than to the tract profiles in the tilted head position, as for example evident from the results for the CST and IFOF. These initial results suggest variation of *T*<sup>2</sup> as a function of fibre orientation to **B**<sup>0</sup> and that tilting the participant's head by 18◦ can reveal those variations.

**Fig. 9** Tractometry results for the genu of the corpus callosum (CC2). *T*2-values and fibre orientation to **B**<sup>0</sup> are plotted against segment numbers for each subject. Colour bar corresponds to tract-segment location, and this is colour-encoded on the tracts themselves. Note that only tractsegments with minimal fanning (assessed visually) were considered

#### *4.1 Incorporating Tiltable Coil in T***2***-diffusion correlation experiments*

**Robustness of experimental setup when using a tiltable coil**. The tiltable coil allows us to control the pitch orientation of the participant's head. We have demonstrated an overall test-retest similarity of estimated fibre-tract orientations in the default position, whereas there were clear differences with the tilted head position. However, some differences in fibre orientation of the CST between the test and retest could be observed, particularly in the inferior segments, while differences in fibre orientation in the IFOF tract were less obvious. Given the apparent stability of the IFOF tract profiling and insensitivity of fibre orientation to the head rotation around **B**<sup>0</sup> in the default head position (yaw), the test-retest differences in the CST could be a result of an additional roll head orientation with respect to the coil between the two scans. Such additional differences in orientation could be mitigated by further restricting head position within the coil, e.g. additional padding or performing the experiments with differences in tilt immediately after each other without a break. Whereas this introduces a confound in assessing test-retest variability, the estimation of fibre-orientations is done in each coil orientation independently and as such we do not expect this to be detrimental in the overall assessment of *T*2-anisotropy.

**Range of orientations**. By re-purposing a coil that was designed to maximise patient comfort in clinical situations means that the range of coil orientations was limited. A larger range of orientations would allow the θ versus 1/*T*<sup>2</sup> relationship to be elucidate more robustly. Nevertheless, these initial experiments demonstrate the utility of this hardware design for uncovering orientational-anisotropy effects in vivo.

#### *4.2 Origin of T***2***-Contrast and -Anisotropy in WM*

The water signal from each WM voxel is a superposition of the signal from different water pools (e.g. intra- and extra-axonal, and at short TE also myelin water), therefore the macroscopic *T*2-values measured in this study could be approximated as the weighted average of the *T*2-values from each of the two compartments. This means that the macroscopic *T*<sup>2</sup> will depend on individual *T*2-values of the compartments, but also on the relative contribution to the signal. As such, tracts with the same apparent *T*<sup>2</sup> but differences in signal fractions and *T*2-values between the intra- and the extra-axonal compartments could exhibit very different apparent orientational anisotropy.

A clear separation of macroscopic *T*2-values in some tracts and segments between the default and the tilted head orientations suggests that macroscopic *T*<sup>2</sup> is also a function of orientation to**B**0. Fibre tract re-orientation in the magnetic field will cause local changes in the local magnetic field due to differences between the magnetic susceptibility of the myelin sheath and the surrounding tissue. Following the hollowcylinder model [26], the magnitude of these microscopic **B**0-field perturbations is expected to be larger in the extra-axonal compartment and, in combination with molecular Brownian motion, will cause an additional faster relaxation in the extrarelative to the intra-axonal compartment [46]. Future work will explore the effects of fibre orientation on individual compartments [47, 48].

#### *4.3 Considerations in Data Processing*

**Tractometry versus image registration** In this work we adopted tract profiling for comparing the *T*2-diffusion-correlation data in default and tilted head positions, instead of a voxel-based analysis; image registration was used for visualisation purposes only. Voxel-based analysis is known to suffer from confounds related to misregistration and data interpolation, an effect we visually observed in our study likely amplified by the large imaging voxels (3 mm isotropic) and imperfect correction for geometric distortions due to gradient nonlinearities. With the tractometry approach, spatial correspondence between coil orientations was established while keeping the data in native space, and each segment consitutes information from multiple voxels improving robustness to noise. We were able to reproduce *T*2-values from test-retest along the tracts.

**Correction for subject motion** The individual images have been corrected for subject motion, which could also involve head re-orientation with respect to the magnetic field. Subjects who participated in this study were experienced MR scanner participants, therefore the maximum rotation around any axis rarely exceeded 1.5◦. However, when considering the application of this method on less compliant subjects, subject motion could become a confounding factor.

#### **5 Conclusion**

Including a tiltable coil into the experimental set-up for diffusion-*T*<sup>2</sup> correlation measurements paves the way for a more reliable assessment of orientational *T*<sup>2</sup> dependence. Microstructural origins of the differences in *T*<sup>2</sup> could include differences in pathway properties (e.g. axon diameter) or susceptibility effects. *T*<sup>2</sup> orientationaldependence would furthermore impact analyses frameworks that assume constant *T*<sup>2</sup> along pathways [49]. Voxel-wise comparison of *T*<sup>2</sup> from different head-orientations remains challenging due to complications in experiment setup, imperfect correction for geometric distortions, and intrinsic scan-variability. Using a tractometry framework, we found indications of regional changes in *T*<sup>2</sup> upon tilting of the head. Studying this effect in a larger population is necessary to increase statistical power. In future work, the diffusion-*T*<sup>2</sup> correlation experiments can be used to study compartmental *T*<sup>2</sup> orientation-dependence.

**Acknowledgments** CMWT is supported by a Rubicon grant (680-50-1527) from the Netherlands Organisation for Scientific Research (NWO) and a Sir HenryWellcome Fellowship (215944/Z/19/Z). DKJ, CMWT and MC were all supported by a Wellcome Trust Investigator Award (096646/Z/11/Z) and DKJ was supported by a Wellcome Strategic Award (104943/Z/14/Z).

The data were acquired at the UK National Facility for In Vivo MR Imaging of Human Tissue Microstructure funded by the EPSRC (grant EP/M029778/1), and The Wolfson Foundation.

We are grateful to Fabrizio Fasano, Peter Gall, and Matschl Volker from Siemens Healthcare GmbH for their support.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Anisotropy in the Human Placenta in Pregnancies Complicated by Fetal Growth Restriction**

**Paddy J. Slator, Alison Ho, Spyros Bakalis, Laurence Jackson, Lucy C. Chappell, Daniel C. Alexander, Joseph V. Hajnal, Mary Rutherford, and Jana Hutter**

**Abstract** The placenta has a unique structure, which enables the transfer of oxygen and nutrients from the mother to the developing fetus. Abnormalities in placental structure are associated with major complications of pregnancy; for instance, changes in the complex branching structures of fetal villous trees are associated with fetal growth restriction. Diffusion MRI has the potential to measure such fine placental microstructural details. Here, we present in-vivo placental diffusion MRI scans from controls and pregnancies complicated by fetal growth restriction. We find that after 30 weeks' gestation fractional anisotropy is significantly higher in placentas associated with growth restricted pregnancies. This shows the potential of diffusion MRI derived measures of anisotropy for assessing placental function during pregnancy.

#### **1 Introduction**

Fetal growth restriction (FGR) is a condition where the developing fetus does not reach its full growth potential in-utero [13]. It constitutes a major pregnancy complication and is associated with a high degree of fetal mortality, morbidity and life-long complications [19]. Early onset FGR (defined as that diagnosed before 32 weeks' gestation) affects 0.5–1% of pregnancies and late onset FGR (diagnosed after 32

e-mail: p.slator@ucl.ac.uk

A. Ho · S. Bakalis · L. C. Chappell Women's Health Department, King's College London, London, UK

L. Jackson · J. V. Hajnal · M. Rutherford · J. Hutter Centre for the Developing Brain, School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK e-mail: jana.hutter@kcl.ac.uk

L. Jackson · J. V. Hajnal · J. Hutter Biomedical Engineering Department, School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK

P. J. Slator (B) · D. C. Alexander

Centre for Medical Image Computing, Department of Computer Science, University College London, London, UK

weeks') affects 5–10% of pregnancies [6, 9]. Early diagnosis and close monitoring are essential to optimize the outcome for pregnancies affected by FGR. Recently, pharmacological treatments have shown promise for severe early onset FGR cases [31]. However, while routine antenatal monitoring utilising symphysis-fundal height and ultrasound measurements can identify a significant proportion of FGR cases [10], detection remains a major problem—only 31% of cases were diagnosed during a previous study [11]. Detection is crucial as FGR is a leading cause of late unexpected stillbirth [11].

Post-delivery histopathological analysis shows a significant degree of characteristic pathologies in FGR placentas [15, 33], emphasising its importance and involvement in the cascade of events leading ultimately to sub-optimal growth. However, while post-delivery detection or confirmation plays an important role in increasing knowledge and possible causes of FGR, it comes too late to influence clinical care for these pregnancies. Therefore, recent novel developments, mainly using Magnetic Resonance Imaging (MRI) focus on studying the placenta during pregnancy to complement available antenatal screening.

#### *1.1 Placental Microstructure*

The placenta constitutes the key connection between mother and baby in utero and acts as the life support system for the growing fetus. Among its many functions are the exchange of oxygen and nutrients from the maternal blood circulation to the fetal blood, and the removal of waste products. The placenta comprises 10–40 individual lobules each constituting one key exchange unit. The transfer relies on a delicate and dynamically evolving microstructure within these units, focused around the fetal villi and depicted in Fig. 1A. These tree-like structures originate from the umbilical cord and contain fetal arteries and veins. They are bathed in maternal blood that enters the intervillous spaces from the spiral arteries at the level of the basal plate. A thin membrane called the syncytiotrophoblast separates maternal and fetal circulation, and allows the transport of oxygen and nutrients through it. Several histopathological features are associated with FGR, including elongated villous trees without the appropriate branching patterns [21] as illustrated schematically in Fig. 1B.

#### *1.2 Placental MRI*

Specific challenges of in-utero placental MRI include motion, such as maternal breathing and fetal movements, various air-tissue interfaces such as amniotic fluid, abdominal fat and bowel gas, and the suboptimal position of the imaging coil with respect to the organ of interest, especially for placentas located on the posterior wall of the uterus [5]. Another important limitation specifically for diffusion MRI arises

**Fig. 1** Schematic representation of single fetal villous trees in the placenta. Fetal blood flows through the convoluted branching structures, allowing nutrient exchange with the surrounding maternal blood. Panel **A**: complex branching structure of a healthy placenta. Panel **B**: pathological branching structure, associated with prengancy complications such as fetal growth restriction

from the safety requirement to reduce the acoustic sound level of the sequence to protect the fetal hearing. Therefore, typically the gradient slew rate is reduced, in consequence increasing the read-out length and thus the echo time resulting in lower Signal-to-Noise Ratios (SNR) [5, 16].

Diffusion MRI can reveal fine tissue microstructure details through sensitivity to the diffusion of water. By varying the strength and direction of diffusion gradients, the MRI sequence can be specifically tailored to microstructural features of interest. For example, sequences with a high number of distinct gradient directions can inform on tissue anisotropy and directionality. Such diffusion MRI sequences are commonly used to image white matter fibre tracts in the brain, however applications elsewhere in body exist, including in the placenta [3, 20, 27].

#### **2 Methods**

Pregnant women, recruited as part of a larger cohort for the Placental Imaging Project (PIP), underwent an MRI scan between 20 and 40 weeks of gestation. Informed consent was obtained (REC 14/LO/1169) and the scan was performed on a clinical 3T Philips Achieva (Best, The Netherlands) scanner using the 32-channel cardiac coil. All women were scanned in supine position under frequent monitoring of heart rate, saturation and blood pressure throughout the scan. Dedicated padding was provided to increase maternal comfort and verbal interaction was maintained.

After initial structural T2-weighted sequences of the entire uterus and the fetal brain in multiple orientations, a B0 map was acquired. An in-house developed tool for image based shimming was employed [12] to focus the shim on the placental parenchyma, avoiding air-tissue interfaces with bowel gas as much as possible. We next acquired a diffusion-prepared spin echo sequence. For some participants we used a modified sequence which acquires additional gradient-echos after the initial Spin Echo (e.g. [28]), although in this study we only utilised the first spin echo measurement. Conventional Stejskal-Tanner diffusion preparation was performed, with a total of 65 b-value/b-vector combinations, optimized specifically for the placenta as previously described [17, 26], with three diffusion gradient directions at b = [5, 10, 25, 50, 100, 200, 400, 600, 1200, 1600] s/mm2, eight directions at b = 18 s/mm2, seven at b = 36 s/mm2, fifteen at b = 800 s/mm2, and six b = 0 volumes. Further parameters include FOV = [300–340] × 320 × 84 mm, TR = 7 s, SENSE = 2.5, halfscan = 0.6, resolution = 3 mm3 for scans with additional gradient-echos and 2 mm3 otherwise. The total acquisition time was 8 min 30 s. Fat was suppressed with SPIR saturation pulses. The acquisition plane was coronal to the mother, chosen to assure that the in-plane direction coincides with the longest placental dimensions in mostly anterior and posterior placentas.

To correct for motion, the diffusion weighted volumes were registered non-rigidly to a common template using the ANTs multivariate template construction tool with the cross-correlation similarity metric [1]. Subsequently, a region of interest (ROI) comprising the whole placenta and adjacent basal placenta was manually segmented in all slices of the first b = 0 volume. We estimated diffusion tensor, mean diffusivity (MD), and fractional anisotropy (FA) maps—using all diffusion weightings—for the motion corrected scans using MRTrix [32].

#### *2.1 Recruitment*

We include a total of twenty-nine participants in this study, who were categorised as follows. Sixteen women were normal uncomplicated control pregnancies; their outcomes were obtained and checked to ensure that no new diagnosis of pre-eclampsia, gestational hypertension, fetal growth restriction or gestational diabetes had occurred, and their birthweight was greater than the 5th centile (by INTERGROWTH-21st). Seven women were diagnosed with fetal growth restriction, detected from antenatal ultrasound assessments. We also include six women recruited as uncomplicated control pregnancies who gave birth to a baby under the 5th centile, but did not have a formal antenatal FGR diagnosis. Although these could simply be constitutionally small babies, they could also be undiagnosed FGR cases, so we hence analysed them as a separate cohort. Five of the six below the 5th centile had co-morbidities, such as chronic hypertension in pregnancy (CHTN), or pre-eclampsia (PE). The patient population characteristics are given in Table 1.

**Table 1** Scan details for all participants. FGR denotes participants diagnosed with fetal growth restriction according to guidelines [13]. Under 5% denotes participants with no FGR diagnosis who delivered a baby weighting under the 5th percentile


#### **3 Results**

Figures 2 and 3 display FA maps, ordered by gestational age (GA) at scanning time, for control and growth restricted participants respectively. We next examine the evolution of FA values over gestation (Fig. 4). Finally, we compare FA values

#### 268 P. J. Slator et al.

**Fig. 2** Fractional anisotropy maps for all control participants. The boundary between the uterine wall (higher FA) and placenta (lower FA) is clear in most placentas with gestational age less than 30 weeks

**Fig. 3** Fractional anisotropy maps for FGR and low birthweight cohorts

**Fig. 4** Mean FA across placenta and uterine wall as a function of gestational age

**Fig. 5** Mean FA across placenta and uterine wall for the three cohorts, split at 30 weeks gestational age

between control and growth-restricted pregnancies (Fig. 5), assessing the potential of FA to inform on FGR-associated placental abnormalities.

The FA maps for all control participants (Fig. 2) show distinctive patterns in agreement with the literature [27]—the FA is lower in the placenta, and higher in the adjacent uterine wall. These observations likely represent isotropic nature of maternal blood pools and fetal villous tree branching within the placenta, and the prevalence of anisotropic fibrous muscle tissue in the uterine wall. Another factor in the high FA in the uterine wall may be the coherent orientation of vasculature in these areas. On the other hand, the distinction between the uterine wall and placenta is less clear in the growth restriction placentas (Fig. 3), and many of these maps show areas of high FA within the placenta.

Figure 4 plots the mean FA value within the ROI comprising the placenta and uterine wall. Control scans (Fig. 4, black dots) show an apparent decrease in mean FA over gestation, potentially reflecting microstructural changes during normal placenta maturation. On the other hand, growth restriction scans (Fig. 4, red and yellow dots), do not show a clear trend over gestation. At early gestational ages, control and compromised placentas have comparable mean FA values. Due to the downward FA trend in control placentas, from around 30 weeks gestation growth restriction placentas appear to have considerably lower FA values than control placentas. This difference was statisically significant, both when comparing controls with the combined FGR and low birth weight cohorts (*p* = 0*.*005, independent samples t-test), and when comparing controls to these two cohorts separately (Fig. 5). We tested if trends over GA differ between controls and growth restriction (i.e. FGR and under 5%) cohorts by calculating a linear regression to predict mean FA based on GA, cohort, and the interaction between GA and cohort. The coefficients and p-values are given in Table 2, and Fig. 6 visualises the fits. The coefficient of the interaction term between GA and low birth weight is statistically significant (p = 0.002), suggesting a different trend over gestation.


**Table 2** Coefficients and corresponding statistics from linear regression to predict mean FA based on GA, cohort, and GA × cohort (interaction term)

**Fig. 6** Linear regression to predict mean FA based on GA, cohort, and GA × cohort

#### **4 Discussion and Conclusion**

This study visualizes and quantifies fractional anisotropy in placentas of both control pregnancies and those affected by growth restriction. Encouragingly we find that the FA is significantly different across gestation in growth restricted pregnancies compared to controls. The fractional anisotropy is a coarse measurement that averages over a number of tissue properties. There are hence a number of microstructural or functional changes that could explain this observation. Our initial speculations of plausible factors behind the changes in FA are as follows.

We observed a clear pattern of decreasing FA with gestational age for control placentas, suggesting that normal placental development causes a reduction in coherently orientated tissue. A consistent hypothesis is that this reduction in FA reflects the normal onset of terminal villi formation along the surfaces of intermediate villi, which occurs predominately during the third trimester [2]. On the other hand, we observed higher FA for placentas associated with growth restricted pregnancies after 30 gestational weeks. This is consistent with histological findings that FGR placentas show a lack of side-branching terminal villi [2].

In this study, we utilised an MR protocol with multiple b-values and gradient directions. However, we only fit a relatively simple model to the data, the diffusion tensor. This data can support more complex models, such as anisotropic IVIM-type models, which separately consider the perfusion (i.e. low b-value) and diffusion (high b-value) signal components (e.g. [26]), and may help disentangle the tissue microstructure changes underlying the observed difference in FA. Another potential approach is to use orientation distribution functions, which might reveal differences in complexity of the orientation of villi (e.g. [23]).

There are a number of MRI-derived biomarkers that show promise for detecting placental dysfunction. These include: T2\* relaxometry [18, 24, 25], which relates to oxygenation levels; structural measures [7]; apparent diffusion coefficient (ADC) [4, 14, 30], relating to global tissue structure; and intravoxel incoherent motion (IVIM) MRI perfusion fraction [8, 22, 29], which relates to blood flow. Thus far, dMRI-derived potential biomarkers (ADC and perfusion fraction) have considered tissue to be isotropic. Our results strongly suggest that the FA is also sensitive to placental dysfunction, and hence that quantifying tissue anisotropy is an additional important avenue for assessing placental health. It may be the case that multiple biomarkers need to be combined in order to best assess the health of an individual placenta. The fact that we see higher FA values for placentas with birth weight under the 5th percentile, as well as those diagnosed with FGR is interesting and merits further investigation. It is likely that a significant proportion of cases under the 5th percentile are undiagnosed FGR. Our results suggest that quantifying tissue anisotropy in the placenta could have a role to play in the detection of FGR, We will investigate this by combining further scanning with post-delivery placental histology to test the ability to distinguish FGR cases from small but otherwise healthy babies.

**Acknowledgments** We thank all mothers, midwives, obstetricians, and radiographers who played a key role in obtaining the datasets. This work was supported by the NIH Human Placenta Project grant 1U01HD087202-01 (Placenta Imaging Project [PiP]); Wellcome Trust (201374/Z/16/Z); EPSRC (N018702, M020533, EP/N018702/1); NIHR (RP-2014-05-019); the Wellcome EPSRC Centre for Medical Engineering at Kings College London (WT 203148/Z/16/Z) and by the National Institute for Health Research (NIHR) Biomedical Research Centre based at Guy's and St Thomas' NHS Foundation Trust and Kings' College London. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

#### **References**


fetal growth restriction. Ultrasound Obstet. Gynecol. **48**(3), 340–348 (2016). https://doi.org/ 10.1002/uog.15879


Using IVIM Echo-Planar MRI. Placenta **21**(7), 726–732 (2000). https://doi.org/10.1053/plac. 2000.0567


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Index**

#### **A**

Angular modulation, 212, 213, 219, 220 Anisotropic fourth-order diffusion, 100, 104 Anisotropic nonlinear diffusion, 101, 103 Anisotropy, 45, 171, 198, 204, 212 Anisotropy field, 40, 55 Anisotropy iso-contours, 49 Anisotropy quadratic equation, 48, 49 Automatic annotation, 160 Average Absolute Error (AAE), 106 Axisymmetric, 211, 212, 216, 217 Axon, 173, 204, 218, 248–250

#### **B**

Barycentric interpolation, 45 Bessel function, 209, 211, 217 Bezier curve, 230 Bifurcations, 24 Biological tissue, 249 Biomarker, 270 Biophysical model, 180 Bivariate quadratic function, 46 Blocking effect, 104 Brain, 249, 250 Brownian molecular motion, 248, 249, 254 Bundle segmentation, 156 *b*-value, 249, 251

#### **C**

Coarse-to-fine tractography, 229, 230 Compartmental anisotropy, 212 Compartment anisotropy, 204 Confinement tensor, 206, 211 Connectivity, 230, 232

Connectom scanner, 251 Consistent anisotropy interpolation, 40 Continuous histogram, 39, 41, 51 Contour tree, 39, 44 Contrast parameter, 102, 111, 113 Control parameters, 228, 230 Convex Optimization Modeling for Microstructure Informed Tractography (COMMIT), 152 Coordinate transformation, 49 Critical point, 47 Crossing tracts, 232 Cumulant expansion, 177 Cumulative histogram, 41, 51 Curvature enhancement, 100, 105

#### **D**

Data tensor, 77 Deep learning, 158 Degenerate, 25 Degenerate curves, 23 Degenerate points, 26, 43 Density map, 137, 140 Diffusion, 204, 249 Diffusional anisotropy, 247 Diffusion-based inpainting, 101 Diffusion coefficient, 195 Diffusion encoding gradients, 251 Diffusion Kurtosis Imaging (DKI), 178 Diffusion MR, 3, 204 Diffusion MRI, 152, 171, 265 Diffusion propagator, 195 Diffusion-*T*<sup>2</sup> correlation, 247, 250, 251, 253–255

© The Editor(s) (if applicable) and The Author(s) 2021 E. Özarslan et al. (eds.), *Anisotropy Across Fields and Scales*, Mathematics and Visualization, https://doi.org/10.1007/978-3-030-56215-1 277

Diffusion tensor, 100, 105, 111, 266 Diffusion Tensor Imaging (DTI), 57, 126, 128, 178, 226 Diffusion time, 195, 219 Diffusion-weighted imaging, 126, 140 Diffusion-weighted magnetic resonance imaging, 194 Diffusion-weighted MRI, 248 Diffusivity, 182 Diffusivity function, 100, 109, 111 Diffusivity tensor, 206, 215 Dirichlet boundary condition, 101, 105, 110 Discretization, 104, 111 Discussion, 20 Divergence field, 88 Double diffusion encoding (DDE), 204, 205, 207 DTI visualization pipeline, 125–127, 129, 130, 132, 141

#### **E**

Echo-planar imaging, 251 Echo time, 251 Eddy currents, 251 Edge Enhancing Diffusion (EED), 100, 112, 113 Edge structures, 100 Effective medium theory, 198 Effective model, 205, 219 Elasticity tensor, 4 Encoding wave vector, 206 Ensemble anisotropy, 204, 205, 212 Ensembles, 136, 137 Equilibration rate, 206 Equivalence problem, 7 Equivalence, characterization of, 15 Equivalence, through invariants, 17 Exclusion ROI, 156

#### **F**

False negatives, 228, 233 False positives, 150, 227, 233 Fast Semi-Iterative Scheme (FSI), 105, 111 Fetal growth restriction, 263 Fiber process, 194 Fiber tracking, 129, 131, 132 Fibercup, 231 Fibre bundles, 249 Fibre orientation distribution, 251 Fibre tractography, 249 Field normalization, 49 Finite differences, 110

Fourth-order diffusion, 101, 109, 113 Fourth-order diffusion tensor, 100, 105, 109, 113 Fourth-Order Edge Enhancing Diffusion (FOEED), 106, 112 Fourth-order PDE, 100, 113 Fourth-order tensor, 104 Fractional anisotropy, 230, 234, 270 Free diffusion limit, 207, 215

#### **G**

Gaussian, 206, 213 Gaussian pre-smoothing, 110 Geodesic, 227, 229 Geodesic completeness, 227 Geodesic tractography, 226, 233 Gibbs ringing, 251 Glyphs, 127, 128, 137, 138 Gradient non-linearity, 251 Gray matter, 218 Grey matter, 172 Gyromagnetic ratio, 206

#### **H**

Hamilton-Jacobi equation, 227 Heat transport, 100 Hessian, 47 Hessian matrix, 104 High Angular Resolution Diffusion Imaging (HARDI), 126, 128, 131–133, 135, 139, 141, 179 Higher-order diffusion filters, 103 Histogram, 41, 44 Hollow cylinder fibre model, 250 Human brain, 171, 248 Hypergeometric function, 210, 212, 215, 217

#### **I**

Ill-posedness, 233 Illustrative, 136, 139 Image compression, 101, 102 Image compression techniques, 99 Image edges, 102 Image inpainting, 100, 101, 113 Image registration, 255 Image smoothing, 101, 102 Imaging artefacts, 250 Inclusion ROI, 156 Inner product, 78 Invariant, 8 Inverse problem, 172

Index 279

ISMRM 2015 Tractography challenge, 232 Iso-contour, 47 Isotropy, 172

#### **J**

Join tree, 40, 44

#### **K**

Kelvin notation, 7 Kissing tracts, 232 Kurtosis, 213

#### **L**

Laplace transform, 220 Larmor frequency, 248 Length functional, 227, 229 Linear diffusion, 100 Linear homogeneous diffusion, 102 Linearly Estimated Moments provide Orientations of Neurites And their Diffusivities Exactly (LEMONADE), 185 Localization regime, 199 Long-time limit, 197 Lossless compression, 99 Lossy compression, 99 Lossy image compression, 104

#### **M**

Magnetic field, 249, 251–253 Magnetic field gradients, 248, 249 Magnetic field gradient waveform, 206 Magnetic Resonance Imaging (MRI), 247, 248, 251, 264 Magnetic Resonance (MR), 204, 248 Magnetic susceptibility, 248–250 Magnetic susceptibility anisotropy, 250 Magnetisation, 248, 249 Magnetization density, 206 Magnitude, 25 Majority voting, 160 Mean Apparent Propagator (MAP)-MRI method, 179 Mean squared error (MSE), 106 Metric, 5 Microstructure, 194, 249, 255, 264 Microstucture informed tractography, 152 Mixing time, 207, 212 Monotonous triangles, 50 Multidimensional array, 77 Multidimensional data, 74

Multi-tissue multi-shell constrained spherical deconvolution, 251 Multivariate data, 76 Myelin, 249, 250 Myelin sheath, 249, 250 Myelin water, 250

#### **N**

Neumann boundary condition, 101 Neurite Orientation Dispersion and Density Imaging (NODDI), 183 Neurites, 182 Neuronal cell, 248 Neurons, 172 N-mode product, 78 Nonlinear diffusion, 100 Nonlinear least-squares, 252 Numerical stability, 104

#### **O**

Optimal tracts, 231 Orientational average, 207 Orientation distribution function, 199

#### **P**

Pancake, 205, 219 Parallelization, 53 Partial Differential Equations (PDEs), 100 PDE-based image compression, 103 PDE-based methods, 100 Perona-Malik equation, 103 Poincaré index, 40 Pore size, 219 Powder average, 204, 212 Power-law, 214, 215, 217 Pre-smoothing parameter, 112 Processes, 173 Progressive rank reduction, 86 Pulesed-gradient spin-echo, 251

#### **Q**

Q-shell, 176 Quadratic form, canonical, 12

#### **R**

Radius of gyration, 219 Rank reduction, 81 Receiver coil, 250 Relaxation-anisotropy, 250 Restricted diffusion, 205, 219 Riemannian metric, 227, 229 Rose of directions, 199 Rose of intersections, 199

#### **S**

Scalar product, 5 Second-order PDEs, 113 Short-time limit, 196 Signal fitting, 155 Signal representation, 173 Signal-to-noise, 250 Signal-to-noise ratio, 252 Single Diffusion Encoding (SDE), 204 Single fibre population, 251, 252 Singular value decomposition, 76 Somas, 173 Spaghetti plot, 136, 137, 139 Spatial selectivity, 85 Specific volume, 197 Spherical-deconvolution Informed Filtering of Tractograms (SIFT), 154 Spin-spin relaxation, 248, 252 Stability analysis, 110, 111 Stationarity, 198 Statistical homogeneity, 197 Steady state, 111 Stereology, 194 Stick, 205, 213, 215, 218, 219 Stochastic geometry, 194 Streamline, 129 Streamline clustering, 158 Streamline loop, 157 Streamline tractography, 227, 229 Stress field, 57 Structural exponent, 198 Structural universality class, 198 Structure tensor, 105, 114 Subdivision, 50 Sub-level set, 51 Subsampling, 85 Support function, 200 Surfa 'lo-ice-to-volume ratio, 197 Swelling artifact, 43 Synthetic data, 56

#### **T**

*T*2-anisotropy, 250, 253, 254 Tensor, 5, 26 Tensor approximation, 74 Tensor decomposition, 80 Tensor distribution, 220 Tensor field interpolation, 43 Tensor norm, 79 Tensor notation, 5 Tensor product, 78 Tensor rank, 79 Tensor reconstruction, 84 Tiltable RF coil, 247, 250, 251, 254 Tissue microstructure, 171 Tissue modelling, 173 Topological invariant, 43 Trace, 25 Tractogram filtering, 151, 154, 160 Tractography, 129–131, 133, 135, 139–141, 150, 155, 226, 250, 251 Tractometry, 250, 251, 253, 255 Triad, 226, 228 Trisectors, 26 Tucker decomposition, 80 Tucker model, 80 Two-point load, 57

#### **U**

Uncertainty visualization, 125–127, 135, 139–141

#### **V**

Vector field, 87 Voigt notation, 7 Von Mises stress, 57 Vorticity field, 88 Voxel, 204

#### **W**

Wedges, 26 White matter, 172, 204, 218, 247–250, 252– 254 White Matter Tract Integrity (WMTI), 184