**3D Ultrasound Image Segmentation: Interactive Texture-Based Approaches**

Julien Olivier2,1 and Ludovic Paulhac1

<sup>1</sup>*Université François Rabelais Tours, Laboratoire Informatique (EA2101)* <sup>2</sup>*École Nationale d'Ingénieurs du Val de Loire France*

#### **1. Introduction**

42 Medical Imaging

[20] Khan T, Jiang H. A new diffusion approximation to the radiative transfer equation for

[21] Shendeleva M L, Molly, J, A, Scaling property of the diffusion equation for light in a

[22] Trabelsi H, Gantri M, Sediki E, Ben Salah R. A Time-dependent Computational Model

[23] Lemonnier D, LeDez V. Discrete ordinate solution of radiative transfer across a slab

[24] Liu, L, H, Benchmark numerical solutions for radiative heat transfer in two-dimensional medium with graded index distribution, *JQSRT*; 102: 293 – 303, (2006) [25] Bal G. Radiative transfer equations with varying refractive index: a mathematical perspective, Journal of the Optical Society of America A; 23: 1639–1644 (2006) [26] Ferwerda, H, A, Radiative transfer equation for scattering media with a spatially varying refractive index, Journal of Optics A: Pure and Applied Optics; 1(3): L1–L2, (1999) [27] Tualle J M, Tinet E. Derivation of the radiative transfer equation for scattering media with a spatially varying refractive index, Optics communications; 228:33-38, (2003)

[28] Modest M, F. Radiative heat transfer; ISBN 0-12-503-163-7, Academic Press, (2003) [29] Trabelsi H , Sghaier T et Sifaoui M S. Theoretical Study of Radiation between two

[30] Cheong W F, Prahl S A, Welch A J. A Review of the Optical Properties of Biological

[31] Mobley J and Vo-Dinh T. optical properties of tissues, biomedical photonics handbook;

Legendre Transform, JQRST; 93:415-428, (2005)

ISBN 0-8493-1116-0, CRC Press LLC (2003)

Tissues. IEEE J. Quantum Electronics; 26:2166-2185, (1990)

Concentric Spheres Using a Modified Discrete Ordinates Method associated with

with variable refractive index. *JQSRT*; 72(2/5): 195–204, (2005)

5:137-141 (2003)

America A; 24 (9): 2902–2910, (2007)

Media III; Lisbon, Portugal, (2009)

scattering media with spatially varying refractive indices. J. Opt. A: Pure App. Opt;

turbid medium with varying refractive index, Journal of the Optical Society of

for Radiation in a Spatially Varying Refractive Index Biological Medium. Proceedings of Eurotherm 83, Computational Thermal Radiation in Participating

> The recent breakthroughs in 3D medical imaging technologies open new promising perspectives in the health domain. Significant efforts have being carried out and the precision of acquisition systems keeps on being improved. These devices are becoming widespread in the hospitals and the constantly increasing queues for this kind of exams prove the interest in these technologies. On the contrary, it seems that the evolution of 3D image analysis tools does not generate the same interest. It may be because, for a long time, people believed that considering 3D images as a succession of 2D frames was a sufficient and efficient way to make precise analysis. In our opinion, it is not the case and huge advancements can be obtained by considering 3D images with their entire complexity. This approach needs the development of specific models which can deal with multiple knowledge and are able to process huge information quantities. As a consequence, this chapter will present two original and efficient approaches of computer science for 3D image analysis and more particularly 3D ultrasound images.

> Ultrasound techniques present a certain number of advantages when compared to other acquisition techniques like magnetic resonance imaging (MRI), X-ray computed tomography (CT), etc. Ultrasounds are not ionizing, which means they are not invasive for the patients. Moreover this technique is inexpensive and allows real time acquisitions. Nevertheless, the interpretation of ultrasound images is very complex and specialists are usually needed throughout the examination. In the same way, the segmentation (extraction of specific regions in the image) of ultrasound images is a very difficult task as these medical images present characteristic artifacts like shadows, speckle, attenuations, missing boundaries etc.

> (Noble & Boukerroui (2006)) proposed a complete survey about native B-mode ultrasound image segmentation in which they tried to point out what makes a good ultrasound segmentation method. Among efficient techniques, the authors have identified methods dealing with image features such as gray level distribution, intensity gradient, phase, similarity measures and texture measures. As described in (Noble & Boukerroui (2006)), texture analysis is very efficient for ultrasound classification and segmentation. The classical method of Haralick's co-occurrence matrices (Haralick et al. (1973)) has often been used and has obtained good performances in several applications (Basset et al. (1993); Valckx &

Fig. 1. Example of a three-dimensional image of the skin obtained with 20 *MHz* ultrasound

3D Ultrasound Image Segmentation: Interactive Texture-Based Approaches 45

provided with high frequencies ultrasound allows to observe the skin perfectly and especially the dermis that has an average thickness ranging from 1 to 2 *mm*. It is also possible to explore a part of the hypodermis. On the other hand, the resolution is insufficient to explore the epidermis (Figure 1). Indeed, its thickness vary between 0.05 to 0.3 *mm* which need ultrasound frequencies higher than 80 *MHz*. Sonography of the skin allows tumor visualizations (cyst, nevus, melanoma, basal cell carcinoma (BCC) etc.), inflammatory pathologies, scars. The discrimination between the different lesions is not always obvious but cutaneous sonography is an important help for detection and diagnosis. The possibility to segment and characterize a lesion in three dimensions is very useful to establish therapeutic strategies. The three-dimensional sonography of the skin is rarely used because of the lack of three-dimensional image analysis tools but the recent evolution of three-dimensional probes should allow the emergence of new techniques. With a three-dimensional acquisition, it is possible to obtain features that are inaccessible with two dimensions. Moreover three-dimensional sonography is well adapted to supervise the evolution of a structure or

As it has been briefly presented in section 1, Noble & Boukerroui (2006) established that texture analysis was a very efficient way to process ultrasound images. Thus, the next section

When dealing with complex images (such as medical images), relying only on first order statistics such as gray level intensity means, standard deviations or histograms is not sufficient to carry out a precise analysis. Indeed, two objects with close intensity statistics and different visual aspects can generally be found in this kind of images (see figure 2). Such objects require

scanner (Atys Medical France)

a lesion notably using volume measures.

present the principle of this kind of analysis.

**3. Texture analysis**

**3.1 Definition**

Thijssen (1997)). Despite the great number of methods, results are still imprecise and the proposed solutions often focus on a given problematic, dedicated to one type of application. Nevertheless, it could be interesting to develop generic tools, automatic or semi-automatic, which would allow a more important usability and interactivity.

In this chapter, two original interactive systems for 3D skin ultrasound image segmentation are presented. Section 2 describes the interests of using 3D ultrasound images for contents analysis and presents classical methods to acquire this images. In section 3, we quickly explain the different families of texture analysis and we provide a discussion about the interest of using an interactive system in a man-aided application. In section 4, the first approach focuses on the development and the combination of efficient perceptual volumetric texture attributes easily understandable by humans. In section 5, the second one is based on a supervised segmentation model, interactively allowing the user to give useful information to the algorithm before the segmentation. To conclude, we discuss about our work and introduce the main prospects.

## **2. 3D ultrasound images**

This section presents how to construct an acquisition of a 3D ultrasound image. We quickly explain the interest of the three-dimensions and what it is possible to analyze in ultrasound images of the skin.

#### **2.1 Ultrasound image acquisition using a 3D probe**

A 3D ultrasound image is acquired using a 3D probe that allows the user to obtain a volume of echo (16 <sup>×</sup> <sup>16</sup> <sup>×</sup> <sup>8</sup> *mm*3) using the scanning of an acoustic beam. For the acquisition of 2D ultrasound images, three main methods have been used (Grégoire et al. (2006)):


By combining two 2D scanning modes, following two different axis, it is possible to obtain a 3D scanning. To construct a 3D acquisition, it is also possible to move manually a 2D probe according to its perpendicular axis. In this case, the quality of the acquisition depends on the operator dexterity. Then, it is preferable to use a real 3D probe, with small size and weight, to obtain the best acquisition quality. This kind of probes has been used to acquire the 3D ultrasound images in this paper.

#### **2.2 Ultrasound images of the skin**

Today, manufacturers propose echographic systems with a resolution ranging from 100 *μm* down to 30 *μm*. This requires ultrasonic frequencies ranging from 20 *MHz* to 60 *MHz*. The increase of ultrasonic frequencies allows a resolution improvement but the wave in the media is attenuated, which limits the applications to superficial exploration. Resolution

Fig. 1. Example of a three-dimensional image of the skin obtained with 20 *MHz* ultrasound scanner (Atys Medical France)

provided with high frequencies ultrasound allows to observe the skin perfectly and especially the dermis that has an average thickness ranging from 1 to 2 *mm*. It is also possible to explore a part of the hypodermis. On the other hand, the resolution is insufficient to explore the epidermis (Figure 1). Indeed, its thickness vary between 0.05 to 0.3 *mm* which need ultrasound frequencies higher than 80 *MHz*. Sonography of the skin allows tumor visualizations (cyst, nevus, melanoma, basal cell carcinoma (BCC) etc.), inflammatory pathologies, scars. The discrimination between the different lesions is not always obvious but cutaneous sonography is an important help for detection and diagnosis. The possibility to segment and characterize a lesion in three dimensions is very useful to establish therapeutic strategies. The three-dimensional sonography of the skin is rarely used because of the lack of three-dimensional image analysis tools but the recent evolution of three-dimensional probes should allow the emergence of new techniques. With a three-dimensional acquisition, it is possible to obtain features that are inaccessible with two dimensions. Moreover three-dimensional sonography is well adapted to supervise the evolution of a structure or a lesion notably using volume measures.

As it has been briefly presented in section 1, Noble & Boukerroui (2006) established that texture analysis was a very efficient way to process ultrasound images. Thus, the next section present the principle of this kind of analysis.

#### **3. Texture analysis**

#### **3.1 Definition**

2 Will-be-set-by-IN-TECH

Thijssen (1997)). Despite the great number of methods, results are still imprecise and the proposed solutions often focus on a given problematic, dedicated to one type of application. Nevertheless, it could be interesting to develop generic tools, automatic or semi-automatic,

In this chapter, two original interactive systems for 3D skin ultrasound image segmentation are presented. Section 2 describes the interests of using 3D ultrasound images for contents analysis and presents classical methods to acquire this images. In section 3, we quickly explain the different families of texture analysis and we provide a discussion about the interest of using an interactive system in a man-aided application. In section 4, the first approach focuses on the development and the combination of efficient perceptual volumetric texture attributes easily understandable by humans. In section 5, the second one is based on a supervised segmentation model, interactively allowing the user to give useful information to the algorithm before the segmentation. To conclude, we discuss about our work and introduce

This section presents how to construct an acquisition of a 3D ultrasound image. We quickly explain the interest of the three-dimensions and what it is possible to analyze in ultrasound

A 3D ultrasound image is acquired using a 3D probe that allows the user to obtain a volume of echo (16 <sup>×</sup> <sup>16</sup> <sup>×</sup> <sup>8</sup> *mm*3) using the scanning of an acoustic beam. For the acquisition of 2D

• The linear scanning: gives rectangular images well adapted to superficial exploration but

• The sectorial scanning: faster method but the obtained image is not rectangular and

By combining two 2D scanning modes, following two different axis, it is possible to obtain a 3D scanning. To construct a 3D acquisition, it is also possible to move manually a 2D probe according to its perpendicular axis. In this case, the quality of the acquisition depends on the operator dexterity. Then, it is preferable to use a real 3D probe, with small size and weight, to obtain the best acquisition quality. This kind of probes has been used to acquire the 3D

Today, manufacturers propose echographic systems with a resolution ranging from 100 *μm* down to 30 *μm*. This requires ultrasonic frequencies ranging from 20 *MHz* to 60 *MHz*. The increase of ultrasonic frequencies allows a resolution improvement but the wave in the media is attenuated, which limits the applications to superficial exploration. Resolution

ultrasound images, three main methods have been used (Grégoire et al. (2006)):

• The circular scanning: very simple method but the image shape is a disk.

which would allow a more important usability and interactivity.

the main prospects.

images of the skin.

**2. 3D ultrasound images**

the acquisition is slow.

presents an angulation.

ultrasound images in this paper.

**2.2 Ultrasound images of the skin**

**2.1 Ultrasound image acquisition using a 3D probe**

When dealing with complex images (such as medical images), relying only on first order statistics such as gray level intensity means, standard deviations or histograms is not sufficient to carry out a precise analysis. Indeed, two objects with close intensity statistics and different visual aspects can generally be found in this kind of images (see figure 2). Such objects require

• Stochastic textures: they are composed of non-regular pattern. These textures can be

3D Ultrasound Image Segmentation: Interactive Texture-Based Approaches 47

• Quasi-deterministic textures: they are composed of several patterns that are very close to each other, but rarely identical. For this reason it remains difficult to isolate one unique

(a) Deterministic textures

(b) Stochastic textures

(c) Quasi-deterministic textures

The purpose of a texture feature is to describe a textured zone with, at least, one numerical value. In an ideal scheme, two different textures will have two different values for one given

The next part of this chapter presents the most used texture features.

pattern. The natural textures often belong to this type of textures.

considered as bi-dimensional random fields.

These three texture types are illustrated in figure (3).

Fig. 3. The three main texture types

**3.2 Texture features**

to use analysis of higher order such as texture analysis, which allow us to study not only the intensity statistics of the pixels but also their spatial distribution in the image.

(a) Grey level intensity mean : 133.9 ; Standard deviation : 42.7

(b) Grey level intensity mean : 126.5 ; Standard deviation : 40.7

Fig. 2. First order analysis limitations: images a) and b) possess the same first order statistics (mean, standard deviation and gray level histogram) but their visual aspects are clearly different. Thus a high order analysis becomes necessary

Before presenting the main texture features, let us try to present a definition for the notion of texture.

Even if several definitions appear in the literature (see Tuceryan & Jain (1993)), if we stay in a general point of view we can define a textured zone in an image as a gray level distribution respecting an ordered scheme and for which the determination of unique features is possible.

However, three main texture types are identified in the literature (Richards & Polit (1974)).

• Deterministic textures: they are composed of one unique element, called a texton (see Julesz (1975)), which is regularly repeated in the space according to specific orientation and period. Even if this kind of textures can easily be characterized, it remains very rare in natural images. That is why, most of the time, deterministic textures are artificial ones.


These three texture types are illustrated in figure (3).

4 Will-be-set-by-IN-TECH

to use analysis of higher order such as texture analysis, which allow us to study not only the

(a) Grey level intensity mean : 133.9 ; Standard deviation : 42.7

(b) Grey level intensity mean : 126.5 ; Standard deviation : 40.7

Fig. 2. First order analysis limitations: images a) and b) possess the same first order statistics (mean, standard deviation and gray level histogram) but their visual aspects are clearly

Before presenting the main texture features, let us try to present a definition for the notion of

Even if several definitions appear in the literature (see Tuceryan & Jain (1993)), if we stay in a general point of view we can define a textured zone in an image as a gray level distribution respecting an ordered scheme and for which the determination of unique features is possible. However, three main texture types are identified in the literature (Richards & Polit (1974)).

• Deterministic textures: they are composed of one unique element, called a texton (see Julesz (1975)), which is regularly repeated in the space according to specific orientation and period. Even if this kind of textures can easily be characterized, it remains very rare in natural images. That is why, most of the time, deterministic textures are artificial ones.

different. Thus a high order analysis becomes necessary

texture.

intensity statistics of the pixels but also their spatial distribution in the image.

(b) Stochastic textures

(c) Quasi-deterministic textures

Fig. 3. The three main texture types

The next part of this chapter presents the most used texture features.

#### **3.2 Texture features**

The purpose of a texture feature is to describe a textured zone with, at least, one numerical value. In an ideal scheme, two different textures will have two different values for one given

When dealing with ultrasound images, the various number of applications, each using different texture features, confirm this observation. As shown in (Noble & Boukerroui (2006)), the classical method of Haralick's co-occurrence matrices (Haralick et al. (1973)) has been widely used and has obtained great performances in several applications (Basset et al. (1993); Valckx & Thijssen (1997)). However, the increasing use of three-dimensional acquisition technologies in clinical practices requires three-dimensional segmentation or analysis methods. Boukerroui et al. (2001) propose a multi-resolution segmentation of three-dimensional ultrasound data (2D+T, 3D) using gray scale intensity, three-dimensional Haralick texture features and three-dimensional tissue characterizing information obtained from the local frequency spectra of the radio-frequency signals. The authors conclude that the use of complementary and/or redundant texture features allows a more robust segmentation. Sahiner et al. (2004) characterize breast masses on three-dimensional ultrasound images. They developed 2D and 3D active contour models for an automated segmentation. Then, they extracted three-dimensional texture and morphological features from the segmented mass. In their study, classification results of malignant and benign breast masses are similar to those experienced by breast radiologists. Zhan & Shen (2003; 2006) present a deformable model to segment three-dimensional ultrasound images. They compute texture features using two banks of two-dimensional Gabor filters located in the two orthogonal planes. Indeed, the use of a three-dimensional Gabor filter bank should have increased the number of filters and computation time. Nevertheless with two banks of two-dimensional Gabor filters,

3D Ultrasound Image Segmentation: Interactive Texture-Based Approaches 49

information is lost in comparison with a bank of three-dimensional Gabor filters.

texture features rarely refer to something easily understandable.

**4. Human Understandable Features (HUF)**

regularity), contrast and, finally, roughness of the image.

Thus, as no unique efficient texture feature can be identified, it remains very important to choose the right texture features but this task is very difficult for a non-expert user because no single mathematical model describing all the features exists. Moreover, the names of the

This aspect of the texture analysis motivated us to develop interactive approaches which allow a non expert user to carry out texture-based segmentations without having to choose some precise texture features. This two approaches are presented in the following two sections.

In this first approach, a set of 3D texture features, inspired by the human way to describe a texture, is proposed. By using human understandable texture features, it is easier and possible for an operator (specialist or not in image analysis) to select the relevant features to process an

The set of characteristics has been chosen using analysis results from some previous works (Amadasun & King (1989); Tamura et al. (1978)). The chosen characteristics are the following: Granularity, which can be represented by the number of three-dimensional patterns constituting the texture, shape information about these patterns (volume, compacity,

In sub-section 4.1, we present a multiresolution scheme for segmentation and we detail the HUF computation. Sub-section 4.2 presents psychological experiments proving the correspondence between our texture features and a human description of textures. Finally,

image. Moreover in a given application it allows a better contents interpretation.

feature. Unfortunately, this rarely happens because, usually, the discriminating power of each texture feature is strongly dependent to the kind of treated textures. In Richards & Polit (1974), four principal texture feature families are identified: statistical methods, geometric methods, model-based methods and, finally, frequency-based methods. Here, we will voluntary not give details about the feature calculations but will only present their principles. If interested, the reader will find further details by following the reference of each approach.

The statistical methods represent the oldest approaches in the texture analysis field. Their principle is to study the pixel's gray level spatial distribution in the image. They usually use statistics of at least second order (*i.e.* the pixels are studied by pair instead of independently). The features from this family represent the best choice for non-expert users because of their implementation easiness. Among the statistical features, the following ones can be considered as the most used: the Haralick coefficients of the coocurrence matrix (Haralick et al. (1973)), the autocorrelation measure (Otsu & Kurita (1988)) and the Local Binary Pattern (LBP) (Ojala et al. (1996)).

The approaches from the geometric method family are based on the principle of repetitiveness present in most of the textures. Indeed, they try to identify and characterize the texture primitive of a texture zone (the texton). As a consequence, this methods are very efficient on deterministic textures but encounter more difficulties when dealing with natural textures such as stochastic or quasi-deterministic ones. The Voronoï tessellation (Shamos & Hoey (1975)) and the structural methods (Voorhees & Poggio (1987); Zucker (1976)) can be considered as the two principal approaches of this family.

The principle of the model-based methods is to build an artificial texture model which must be as similar to the studied texture as possible. The evolution of this artificial model characteristics is observed during its construction and once the model is close enough to the studied texture, these characteristics are kept as the texture features. The Markov random fields (Li (1995)) and the fractal approaches (Mandelbrot (1977)) represent the two principal model-based methods.

The frequency-based methods consider that a texture can be characterized by analyzing the repetition of one or several pattern according to various spatial frequencies. Thus, several methods, initialy developed for 1D signals (like sound), have been adjusted to 2D signals such as images. The most popular frequency-based methods are the Fourier analysis (Azencott et al. (1997)), the Gabor filters (Turner (1986)) and the wavelet transform (Mallat (1989)).

#### **3.3 Choosing the right texture feature**

As it has been presented in the previous section, several texture features have been developed over the past thirty years. Unfortunately, most of these methods do not have a general applicability and cannot identify some classes of texture. For example, some of these approaches are not able to describe the directionality properties. In comparison, the human visual system can adapt to all types of textures even in the case of an unfavorable context. As a consequence, when developing an texture-based image analysis application, it would seems natural to think of including the maximum of texture features in order to obtain the best accuracy. Unfortunately, this strategy does not represent a good choice as some of the included features will alter the segmentation if they are not discriminating enough.

6 Will-be-set-by-IN-TECH

feature. Unfortunately, this rarely happens because, usually, the discriminating power of each texture feature is strongly dependent to the kind of treated textures. In Richards & Polit (1974), four principal texture feature families are identified: statistical methods, geometric methods, model-based methods and, finally, frequency-based methods. Here, we will voluntary not give details about the feature calculations but will only present their principles. If interested,

The statistical methods represent the oldest approaches in the texture analysis field. Their principle is to study the pixel's gray level spatial distribution in the image. They usually use statistics of at least second order (*i.e.* the pixels are studied by pair instead of independently). The features from this family represent the best choice for non-expert users because of their implementation easiness. Among the statistical features, the following ones can be considered as the most used: the Haralick coefficients of the coocurrence matrix (Haralick et al. (1973)), the autocorrelation measure (Otsu & Kurita (1988)) and the Local Binary Pattern (LBP) (Ojala

The approaches from the geometric method family are based on the principle of repetitiveness present in most of the textures. Indeed, they try to identify and characterize the texture primitive of a texture zone (the texton). As a consequence, this methods are very efficient on deterministic textures but encounter more difficulties when dealing with natural textures such as stochastic or quasi-deterministic ones. The Voronoï tessellation (Shamos & Hoey (1975)) and the structural methods (Voorhees & Poggio (1987); Zucker (1976)) can be considered as

The principle of the model-based methods is to build an artificial texture model which must be as similar to the studied texture as possible. The evolution of this artificial model characteristics is observed during its construction and once the model is close enough to the studied texture, these characteristics are kept as the texture features. The Markov random fields (Li (1995)) and the fractal approaches (Mandelbrot (1977)) represent the two principal

The frequency-based methods consider that a texture can be characterized by analyzing the repetition of one or several pattern according to various spatial frequencies. Thus, several methods, initialy developed for 1D signals (like sound), have been adjusted to 2D signals such as images. The most popular frequency-based methods are the Fourier analysis (Azencott et al. (1997)), the Gabor filters (Turner (1986)) and the wavelet transform (Mallat (1989)).

As it has been presented in the previous section, several texture features have been developed over the past thirty years. Unfortunately, most of these methods do not have a general applicability and cannot identify some classes of texture. For example, some of these approaches are not able to describe the directionality properties. In comparison, the human visual system can adapt to all types of textures even in the case of an unfavorable context. As a consequence, when developing an texture-based image analysis application, it would seems natural to think of including the maximum of texture features in order to obtain the best accuracy. Unfortunately, this strategy does not represent a good choice as some of the

included features will alter the segmentation if they are not discriminating enough.

the reader will find further details by following the reference of each approach.

et al. (1996)).

model-based methods.

the two principal approaches of this family.

**3.3 Choosing the right texture feature**

When dealing with ultrasound images, the various number of applications, each using different texture features, confirm this observation. As shown in (Noble & Boukerroui (2006)), the classical method of Haralick's co-occurrence matrices (Haralick et al. (1973)) has been widely used and has obtained great performances in several applications (Basset et al. (1993); Valckx & Thijssen (1997)). However, the increasing use of three-dimensional acquisition technologies in clinical practices requires three-dimensional segmentation or analysis methods. Boukerroui et al. (2001) propose a multi-resolution segmentation of three-dimensional ultrasound data (2D+T, 3D) using gray scale intensity, three-dimensional Haralick texture features and three-dimensional tissue characterizing information obtained from the local frequency spectra of the radio-frequency signals. The authors conclude that the use of complementary and/or redundant texture features allows a more robust segmentation. Sahiner et al. (2004) characterize breast masses on three-dimensional ultrasound images. They developed 2D and 3D active contour models for an automated segmentation. Then, they extracted three-dimensional texture and morphological features from the segmented mass. In their study, classification results of malignant and benign breast masses are similar to those experienced by breast radiologists. Zhan & Shen (2003; 2006) present a deformable model to segment three-dimensional ultrasound images. They compute texture features using two banks of two-dimensional Gabor filters located in the two orthogonal planes. Indeed, the use of a three-dimensional Gabor filter bank should have increased the number of filters and computation time. Nevertheless with two banks of two-dimensional Gabor filters, information is lost in comparison with a bank of three-dimensional Gabor filters.

Thus, as no unique efficient texture feature can be identified, it remains very important to choose the right texture features but this task is very difficult for a non-expert user because no single mathematical model describing all the features exists. Moreover, the names of the texture features rarely refer to something easily understandable.

This aspect of the texture analysis motivated us to develop interactive approaches which allow a non expert user to carry out texture-based segmentations without having to choose some precise texture features. This two approaches are presented in the following two sections.

#### **4. Human Understandable Features (HUF)**

In this first approach, a set of 3D texture features, inspired by the human way to describe a texture, is proposed. By using human understandable texture features, it is easier and possible for an operator (specialist or not in image analysis) to select the relevant features to process an image. Moreover in a given application it allows a better contents interpretation.

The set of characteristics has been chosen using analysis results from some previous works (Amadasun & King (1989); Tamura et al. (1978)). The chosen characteristics are the following: Granularity, which can be represented by the number of three-dimensional patterns constituting the texture, shape information about these patterns (volume, compacity, regularity), contrast and, finally, roughness of the image.

In sub-section 4.1, we present a multiresolution scheme for segmentation and we detail the HUF computation. Sub-section 4.2 presents psychological experiments proving the correspondence between our texture features and a human description of textures. Finally,

(a) (b) (c) (d)

3D Ultrasound Image Segmentation: Interactive Texture-Based Approaches 51

components are computed for each binary textures of the produced sequence. Figure 5

Connected components represent the basic objects inside binary textures. Their analysis can provide important geometrical and volume information and it allows computation of features like granularity which corresponds to the number of patterns per volume unit, the volume and compacity of each connected components providing information about the shape of texture patches. The regularity can also be estimated using the variance of these patterns. In our case, the number of patterns corresponds to the number of connected components (*nbCC*) per volume unit. For a given texture, if the number of connected component is important for the resolution *β* then, there is an important number of patterns and the granularity (*fgran<sup>β</sup>* ) of the

Besides the number of connected components, we compute shape characteristics with the average volume and the average compacity of connected components. Like the number of connected components, the volume is an additional information to identify the fineness of a

> *nbCC<sup>β</sup>* ∑ *i*=1

connected components are located in a cube of size *N*<sup>3</sup> centered at the coordinates (*x*, *y*, *z*).

*Ci*,*<sup>β</sup>* <sup>=</sup> *<sup>S</sup>*

3 2 *i*,*β Vi*,*<sup>β</sup>*

Compacity of connected components gives information about pattern shape. A texture with an important compacity is a texture with compact patterns. Otherwise this is a texture with elongate shapes. This characteristic is invariant by translation, rotation but also to scale changes (Zhang & Tan (2002)). It has been used to texture characterization in (Goyal et al.

*Vi*,*β*)/*nbCC<sup>β</sup>* (1)

(2)

*th* connected component for the resolution *β*. The considered

texture. The average volume of connected components is computed as follows:

*fvol<sup>β</sup>* (*x*, *y*, *z*)=(

(1995)). The compacity of a connected component can be computed as follows:

Fig. 5. a) One ultrasound image, b) The same ultrasound image after a binarization, [c-d]

Connected components extracted from different binary images

texture is significant.

where *Vi*,*<sup>β</sup>* is the volume of the *i*

presents a binarized image with examples of connected components.

#### Fig. 4. Multiresolution segmentation

sub-section 4.3, shows segmentation results of 3D ultrasound images of the skin. More details about this work are available in Paulhac, Makris, Gregoire & Ramel (2009).

#### **4.1 A multiresolution scheme for texture segmentation**

The proposed approach allows us to give an image description using texture features for several resolutions by using the 3D discrete wavelet transform (Figure 4). Mallat (1989) proposes a decomposition scheme using filters: a highpass filter, which allows to obtain detail coefficients and a lowpass filter which gives approximation coefficients. Roughness is computed using detail coefficients whereas the other proposed features are computed using approximation coefficients. Before the segmentation process, the proposed features are computed for each voxel and for several resolutions. In order to obtain a segmentation of the start image, a step of feature upsampling is necessary for each resolution. Thus, a voxel of the initial image is described by a vector containing 6*n* different features with *n* the number of resolutions and 6 the number of proposed features. The user is then asked to choose the features that seem the most relevant from his point of view. And it is because they are human understandable that this choice will be easy. Finally, the K-means algorithm (Coleman & Andrews (1979)) allows to generate a segmentation using the set of computed vectors.

#### **4.1.1 A geometric approach for 3D texture analysis**

We have chosen to describe the geometric structure of the textures with the help of the three-dimensional connected components which can be viewed as the patch patterns in the texture. To compute connected components, we propose a similar method to the one presented in (Shoshany (2008)): a gray-level textured image is decomposed into a progressive sequence of binary textures in order to study patterns and their evolution. In our approach, a clustering algorithm applied on the voxels of the initial 3D image allows us to identify a set of thresholds used to compute the sequence of binary versions of the image. Then, connected 8 Will-be-set-by-IN-TECH

sub-section 4.3, shows segmentation results of 3D ultrasound images of the skin. More details

The proposed approach allows us to give an image description using texture features for several resolutions by using the 3D discrete wavelet transform (Figure 4). Mallat (1989) proposes a decomposition scheme using filters: a highpass filter, which allows to obtain detail coefficients and a lowpass filter which gives approximation coefficients. Roughness is computed using detail coefficients whereas the other proposed features are computed using approximation coefficients. Before the segmentation process, the proposed features are computed for each voxel and for several resolutions. In order to obtain a segmentation of the start image, a step of feature upsampling is necessary for each resolution. Thus, a voxel of the initial image is described by a vector containing 6*n* different features with *n* the number of resolutions and 6 the number of proposed features. The user is then asked to choose the features that seem the most relevant from his point of view. And it is because they are human understandable that this choice will be easy. Finally, the K-means algorithm (Coleman & Andrews (1979)) allows to generate a segmentation using the set of computed vectors.

We have chosen to describe the geometric structure of the textures with the help of the three-dimensional connected components which can be viewed as the patch patterns in the texture. To compute connected components, we propose a similar method to the one presented in (Shoshany (2008)): a gray-level textured image is decomposed into a progressive sequence of binary textures in order to study patterns and their evolution. In our approach, a clustering algorithm applied on the voxels of the initial 3D image allows us to identify a set of thresholds used to compute the sequence of binary versions of the image. Then, connected

about this work are available in Paulhac, Makris, Gregoire & Ramel (2009).

**4.1 A multiresolution scheme for texture segmentation**

**4.1.1 A geometric approach for 3D texture analysis**

Fig. 4. Multiresolution segmentation

Fig. 5. a) One ultrasound image, b) The same ultrasound image after a binarization, [c-d] Connected components extracted from different binary images

components are computed for each binary textures of the produced sequence. Figure 5 presents a binarized image with examples of connected components.

Connected components represent the basic objects inside binary textures. Their analysis can provide important geometrical and volume information and it allows computation of features like granularity which corresponds to the number of patterns per volume unit, the volume and compacity of each connected components providing information about the shape of texture patches. The regularity can also be estimated using the variance of these patterns. In our case, the number of patterns corresponds to the number of connected components (*nbCC*) per volume unit. For a given texture, if the number of connected component is important for the resolution *β* then, there is an important number of patterns and the granularity (*fgran<sup>β</sup>* ) of the texture is significant.

Besides the number of connected components, we compute shape characteristics with the average volume and the average compacity of connected components. Like the number of connected components, the volume is an additional information to identify the fineness of a texture. The average volume of connected components is computed as follows:

$$f\_{\text{pol}\_{\beta}}(\mathbf{x}, y, z) = (\sum\_{i=1}^{nb \text{CC}\_{\beta}} V\_{i, \emptyset}) / nb \text{CC}\_{\beta} \tag{1}$$

where *Vi*,*<sup>β</sup>* is the volume of the *i th* connected component for the resolution *β*. The considered connected components are located in a cube of size *N*<sup>3</sup> centered at the coordinates (*x*, *y*, *z*).

Compacity of connected components gives information about pattern shape. A texture with an important compacity is a texture with compact patterns. Otherwise this is a texture with elongate shapes. This characteristic is invariant by translation, rotation but also to scale changes (Zhang & Tan (2002)). It has been used to texture characterization in (Goyal et al. (1995)). The compacity of a connected component can be computed as follows:

$$\mathbf{C}\_{i,\beta} = \frac{\mathbf{S}\_{i,\beta}^{\frac{3}{2}}}{V\_{i,\beta}} \tag{2}$$

*<sup>α</sup>*4,*<sup>β</sup>* <sup>=</sup> *<sup>μ</sup>*4,*<sup>β</sup> σ*4 *β*

3D Ultrasound Image Segmentation: Interactive Texture-Based Approaches 53

where *μ*<sup>4</sup> is the mean fourth moment and *σ*<sup>2</sup> the variance of gray-levels for the resolution *β*. In order to take into account the dynamic range of gray-levels, they combine the kurtosis with

> *σβ αn* 4,*β*

*fcont<sup>β</sup>* (*x*, *y*, *z*) =

where *n* is a positive value. In their paper, Tamura et al make comparisons between psychological experiments and their operators before concluding that the value *n* = 1/4 gives

These experiments have been carried out to prove that a strong correspondence exists between the proposed features and the human vision. Thus, we propose to construct psychometric

The set of textures presented in Figure 6 has been used in our experiments. These textures have been constructed using methods presented in (Kopf et al. (2007); Paulhac, Makris & Ramel (2009)) except for textures (j) and (l) that are subsets of ultrasound images. Each of them is a volumetric texture (texture in the 3D domain) of size 128<sup>3</sup> with 256 gray-levels. They have been printed using a printer HP Color LaserJet 3700 and presented to human subjects. The group of human subjects was composed of 15 men and 11 women, and the majority of them had no knowledge in image and texture analysis. We distributed to each one of them a questionnaire containing the set of textures (Figure 6) and an explanation of the texture features used in our model. For each feature, textures had to be classified in descending order, *i.e.* from the most rough to the most smooth, from the most regular to the most irregular, etc. Using these questionnaires, we constructed a ranking of these textures for each texture attribute. For a given characteristic, a score has been assigned to a texture according to its ranking. For example, The most rough takes the value +12 (for the roughness feature), the second one +11, the last one +1, and this for all the questionnaires. The addition of the questionnaire scores for each texture allowed us to obtain a final ranking for a given texture

Using the proposed texture attributes, we also generate a feature ranking. A vector of 6 features has been computed for each texture in the questionnaire. Here only the first resolution has been considered (*β* = 1), because this is the one which corresponds the best to the

To compare human and feature ranking, the degree of correspondences between them has been determined. In this respect, we chose to use the well-known Spearman's coefficient of

observation of textures by human subjects through questionnaires.

**4.2.1 Comparison between human and computational ranking**

the standard deviation as follows:

centered at the coordinates (*x*, *y*, *z*).

**4.2 Psychological experiments**

feature.

the best approximation. At last, the values of *σβ* and *α<sup>n</sup>*

prototypes and to compare them to our texture measures.

(6)

(7)

4,*<sup>β</sup>* are computed in a cube of size *<sup>N</sup>*<sup>3</sup>

where *Si*,*<sup>β</sup>* is the surface and *Vi*,*<sup>β</sup>* is the volume of the *i th* connected component for the resolution *β*.

It is then possible to compute the average compacity *fcomp<sup>β</sup>* :

$$f\_{\text{comp}\,p\_{\beta}}(x,y,z) = \frac{1}{nb\mathbb{C}\mathbb{C}\_{\beta}} \sum\_{i=1}^{nb\mathbb{C}\mathbb{C}\_{\beta}} \mathbb{C}\_{i,\beta} \tag{3}$$

We can also obtain information about the regularity of a texture from the study of the connected components. Therefore, we decided to use the compacity variance. We have seen that this characteristic is invariant by any transformation (Zhang & Tan (2002)). The shape of patterns is the only element that affects the variance feature. A low variance of the compacity indicates an important regularity of the connected components whatever their spatial organization is.

$$f\_{\text{reg}\_{\mathcal{B}}}(\mathbf{x}, y, z) = E(\mathbf{C}\_{\mathcal{B}}^2) - \left(E(\mathbf{C}\_{\mathcal{B}})\right)^2 \tag{4}$$

where *E* is the expected value.

#### **4.1.2 Statistical and frequency based method for contrast and roughness measure**

The surface of a rough texture presents a high number of asperities. In an image, roughness can be described as a set of quick spatial transitions with varying amplitude. From a frequential point of view, the image asperities in the spatial domain correspond to the presence of high frequencies. Detail coefficients from the wavelet transform give a description of high frequencies in an image among several directions. It is then possible to have an estimation of the texture roughness for a specific resolution.

Finally, we propose to compute the roughness attribute as follows:

$$f\_{r\mathcal{G}^{h\_{\mathcal{G}}}}(\mathbf{x}, \mathbf{y}, \mathbf{z}) = \sum\_{a=1}^{M} (\sum\_{i=1}^{N} \sum\_{j=1}^{N} \sum\_{k=1}^{N} \left| w\_{a, \mathcal{G}}(i, j, k) \right|) / M \tag{5}$$

where *frgh<sup>β</sup>* is the roughness at the resolution *β*. *wα*,*β*(*i*, *j*, *k*) corresponds to the set of detail coefficients in a cube of size *N*<sup>3</sup> centered at a voxel of a sub-band *α* at the coordinates (*x*, *y*, *z*) and *M* represents the number of detail coefficient sub-bands for a resolution.

Haralick et al. (1973) propose a measure to estimate contrast by computing the moment of inertia from the main diagonal of the co-occurrence matrix. Nevertheless, the construction of a co-occurrence matrix, only to obtain an estimation of the contrast, can be expensive in computing time. Tamura et al. (1978) claim that four factors are supposed to influence the contrast difference between two textures. They are the dynamic range of gray-levels, the polarization of the distribution of black and white on the gray-level histogram or ratio of black and white areas, the sharpness of edges, and the period of repeating patterns. They propose to approximate the contrast with a measure incorporating the two first factors. We use this approximation in our work. To obtain a measure of polarization the kurtosis *α*<sup>4</sup> is used. It allows a measurement of the disposition of probability mass around their center.

$$
\alpha\_{4,\beta} = \frac{\mu\_{4,\beta}}{\sigma\_{\beta}^4} \tag{6}
$$

where *μ*<sup>4</sup> is the mean fourth moment and *σ*<sup>2</sup> the variance of gray-levels for the resolution *β*. In order to take into account the dynamic range of gray-levels, they combine the kurtosis with the standard deviation as follows:

$$f\_{\rm cont\_{\beta}}(\mathbf{x}, y, z) = \frac{\sigma\_{\beta}}{\mathfrak{a}\_{\mathbf{4}, \beta}^{\mathfrak{n}}} \tag{7}$$

where *n* is a positive value. In their paper, Tamura et al make comparisons between psychological experiments and their operators before concluding that the value *n* = 1/4 gives the best approximation. At last, the values of *σβ* and *α<sup>n</sup>* 4,*<sup>β</sup>* are computed in a cube of size *<sup>N</sup>*<sup>3</sup> centered at the coordinates (*x*, *y*, *z*).

#### **4.2 Psychological experiments**

10 Will-be-set-by-IN-TECH

*nbCC<sup>β</sup>*

We can also obtain information about the regularity of a texture from the study of the connected components. Therefore, we decided to use the compacity variance. We have seen that this characteristic is invariant by any transformation (Zhang & Tan (2002)). The shape of patterns is the only element that affects the variance feature. A low variance of the compacity indicates an important regularity of the connected components whatever their

*nbCC<sup>β</sup>* ∑ *i*=1

*fcomp<sup>β</sup>* (*x*, *<sup>y</sup>*, *<sup>z</sup>*) = <sup>1</sup>

*freg<sup>β</sup>* (*x*, *<sup>y</sup>*, *<sup>z</sup>*) = *<sup>E</sup>*(*C*<sup>2</sup>

**4.1.2 Statistical and frequency based method for contrast and roughness measure**

*M* ∑ *α*=1 ( *N* ∑ *i*=1

and *M* represents the number of detail coefficient sub-bands for a resolution.

The surface of a rough texture presents a high number of asperities. In an image, roughness can be described as a set of quick spatial transitions with varying amplitude. From a frequential point of view, the image asperities in the spatial domain correspond to the presence of high frequencies. Detail coefficients from the wavelet transform give a description of high frequencies in an image among several directions. It is then possible to have an estimation of

> *N* ∑ *j*=1

where *frgh<sup>β</sup>* is the roughness at the resolution *β*. *wα*,*β*(*i*, *j*, *k*) corresponds to the set of detail coefficients in a cube of size *N*<sup>3</sup> centered at a voxel of a sub-band *α* at the coordinates (*x*, *y*, *z*)

Haralick et al. (1973) propose a measure to estimate contrast by computing the moment of inertia from the main diagonal of the co-occurrence matrix. Nevertheless, the construction of a co-occurrence matrix, only to obtain an estimation of the contrast, can be expensive in computing time. Tamura et al. (1978) claim that four factors are supposed to influence the contrast difference between two textures. They are the dynamic range of gray-levels, the polarization of the distribution of black and white on the gray-level histogram or ratio of black and white areas, the sharpness of edges, and the period of repeating patterns. They propose to approximate the contrast with a measure incorporating the two first factors. We use this approximation in our work. To obtain a measure of polarization the kurtosis *α*<sup>4</sup> is used. It allows a measurement of the disposition of probability mass around their center.

*N* ∑ *k*=1 

*wα*,*β*(*i*, *j*, *k*)

 

)/*<sup>M</sup>* (5)

*th* connected component for the

*Ci*,*<sup>β</sup>* (3)

*<sup>β</sup>*) <sup>−</sup> (*E*(*Cβ*))<sup>2</sup> (4)

where *Si*,*<sup>β</sup>* is the surface and *Vi*,*<sup>β</sup>* is the volume of the *i*

It is then possible to compute the average compacity *fcomp<sup>β</sup>* :

resolution *β*.

spatial organization is.

where *E* is the expected value.

the texture roughness for a specific resolution.

Finally, we propose to compute the roughness attribute as follows:

*frgh<sup>β</sup>* (*x*, *y*, *z*) =

These experiments have been carried out to prove that a strong correspondence exists between the proposed features and the human vision. Thus, we propose to construct psychometric prototypes and to compare them to our texture measures.

The set of textures presented in Figure 6 has been used in our experiments. These textures have been constructed using methods presented in (Kopf et al. (2007); Paulhac, Makris & Ramel (2009)) except for textures (j) and (l) that are subsets of ultrasound images. Each of them is a volumetric texture (texture in the 3D domain) of size 128<sup>3</sup> with 256 gray-levels. They have been printed using a printer HP Color LaserJet 3700 and presented to human subjects.

The group of human subjects was composed of 15 men and 11 women, and the majority of them had no knowledge in image and texture analysis. We distributed to each one of them a questionnaire containing the set of textures (Figure 6) and an explanation of the texture features used in our model. For each feature, textures had to be classified in descending order, *i.e.* from the most rough to the most smooth, from the most regular to the most irregular, etc. Using these questionnaires, we constructed a ranking of these textures for each texture attribute. For a given characteristic, a score has been assigned to a texture according to its ranking. For example, The most rough takes the value +12 (for the roughness feature), the second one +11, the last one +1, and this for all the questionnaires. The addition of the questionnaire scores for each texture allowed us to obtain a final ranking for a given texture feature.

Using the proposed texture attributes, we also generate a feature ranking. A vector of 6 features has been computed for each texture in the questionnaire. Here only the first resolution has been considered (*β* = 1), because this is the one which corresponds the best to the observation of textures by human subjects through questionnaires.

#### **4.2.1 Comparison between human and computational ranking**

To compare human and feature ranking, the degree of correspondences between them has been determined. In this respect, we chose to use the well-known Spearman's coefficient of

Region Description Texture attributes scoring

3D Ultrasound Image Segmentation: Interactive Texture-Based Approaches 55

These lesions are present in the dermis and have an average or

can contain more or less echogenicity according to

epidermis. Moreover it is similar to the plastic membrane in ultrasound images with a

These results show a huge correlation between human and feature ranking. For the volume, the Spearman's coefficient indicates that there is a link between the two measurements with a confidence rate included between 95 and 98 percent. The compacity feature gives the best result with a confidence rate which tends toward 100 percent. The volume feature has the smallest correlation results. It can be supposed that it is sometimes difficult for our human

As explained in Noble & Boukerroui (2006), the scatterer distribution and their relative volumes to the wavelength of the incident ultrasound pulse produces different three-dimensional texture patterns. During an ultrasound examination, echogenicity represents the ability of a cellular tissue to create an echo. In an ultrasound image, echogenic zones contain a large number of white three-dimensional patterns and this is an important characteristic used by specialists to identify pathologies. From a texture analysis point of view, echogenicity of a zone can be described mainly by granularity measures carried out on these three-dimensional patterns. The interest of each of the proposed HUF features has been validated by specialists in dermatology. Table 2 presents a synthesis of the correspondence between the HUF features and the characteristics according to the regions in the skin. It was not possible to predict the exact value of the HUF features and a score (low:+, medium:++, high:+++) has been proposed for each texture attribute according to the regions in the skin.

The architecture of the proposed system is composed of three main modules. The first one computes texture features according to user-defined parameters. Then, HUF features are used in the second module of segmentation. Finally, the segmented image can be exploited by the 3D visualization and manipulation module. This module allows to visualize the initial segmentation and to improve, in an interactive way, this first result. It is also possible to

granularity:+, compacity:+++, contrast:+, roughness:+,

granularity:+++, compacity:+, contrast:++, roughness:++,

granularity:++, compacity:++, contrast:++, roughness:++,

granularity:+, compacity:+, contrast:++, roughness:+,

regularity:++

regularity:+

regularity:+

regularity:++

low echogenicity.

high echogenicity.

echogenicity.

Normal Dermis In this zone, there is a regular

Hypodermis This region of the skin

zones.

Epidermis Resolution is to low to analyze

subjects to visualize the volume of patterns because of the 3D.

**4.3 Segmentation of 3D ultrasound images with the HUF system**

Table 2. 3D interesting textures in skin images

**4.3.1 Presentation of the segmentation software**

Nevus, Histiocytofibroma, Cyst, Melanoma, BCC

(e) (f) (g) (h)

Fig. 6. Set of Solid textures used for psychological experiments

rank correlation:

$$r\_s = 1 - \frac{6}{n^3 - n} \sum\_{i=1}^n d\_i^2 \tag{8}$$

where *n* is the number of individuals, and *di* is the difference between the ranks assigned to the *i th* object in the two measurements. The value of this coefficient varies between <sup>−</sup><sup>1</sup> and 1. Value 1 corresponds to the complete agreement of the two rankings whereas value −1 indicates complete disagreement. Table 1 presents coefficients of rank correlations between human and feature ranking.


Table 1. Coefficients of rank correlations between human and feature ranking


Table 2. 3D interesting textures in skin images

12 Will-be-set-by-IN-TECH

(a) (b) (c) (d)

(e) (f) (g) (h)

(i) (j) (k) (l)

*<sup>n</sup>*<sup>3</sup> − *<sup>n</sup>*

*th* object in the two measurements. The value of this coefficient varies between <sup>−</sup><sup>1</sup>

where *n* is the number of individuals, and *di* is the difference between the ranks assigned

and 1. Value 1 corresponds to the complete agreement of the two rankings whereas value −1 indicates complete disagreement. Table 1 presents coefficients of rank correlations between

> *fgran fcomp fvol freg frgh fcont* 0.83 0.9 0.61 0.82 0.75 0.65

Table 1. Coefficients of rank correlations between human and feature ranking

*n* ∑ *i*=1 *d*2

*<sup>i</sup>* (8)

*rs* <sup>=</sup> <sup>1</sup> <sup>−</sup> <sup>6</sup>

Fig. 6. Set of Solid textures used for psychological experiments

rank correlation:

human and feature ranking.

to the *i*

These results show a huge correlation between human and feature ranking. For the volume, the Spearman's coefficient indicates that there is a link between the two measurements with a confidence rate included between 95 and 98 percent. The compacity feature gives the best result with a confidence rate which tends toward 100 percent. The volume feature has the smallest correlation results. It can be supposed that it is sometimes difficult for our human subjects to visualize the volume of patterns because of the 3D.

## **4.3 Segmentation of 3D ultrasound images with the HUF system**

As explained in Noble & Boukerroui (2006), the scatterer distribution and their relative volumes to the wavelength of the incident ultrasound pulse produces different three-dimensional texture patterns. During an ultrasound examination, echogenicity represents the ability of a cellular tissue to create an echo. In an ultrasound image, echogenic zones contain a large number of white three-dimensional patterns and this is an important characteristic used by specialists to identify pathologies. From a texture analysis point of view, echogenicity of a zone can be described mainly by granularity measures carried out on these three-dimensional patterns. The interest of each of the proposed HUF features has been validated by specialists in dermatology. Table 2 presents a synthesis of the correspondence between the HUF features and the characteristics according to the regions in the skin. It was not possible to predict the exact value of the HUF features and a score (low:+, medium:++, high:+++) has been proposed for each texture attribute according to the regions in the skin.

## **4.3.1 Presentation of the segmentation software**

The architecture of the proposed system is composed of three main modules. The first one computes texture features according to user-defined parameters. Then, HUF features are used in the second module of segmentation. Finally, the segmented image can be exploited by the 3D visualization and manipulation module. This module allows to visualize the initial segmentation and to improve, in an interactive way, this first result. It is also possible to

(a) Nevus, K=2 (b) Nevus, K=2

3D Ultrasound Image Segmentation: Interactive Texture-Based Approaches 57

(c) Histiocytofibroma, K=3 (d) Histiocytofibroma, K=4

(e) Layers of the skin

in order to help specialists in their diagnostic, to track pathologies evolution or to carry out more precise extractions etc. With a clustering, all the voxels of an image are classified and it is possible to make a visualization of the different skin layers(figure 8(e)). For a specialist, this visualization is interesting because skin layers evolve according to the patient age and it is interesting to supervise the healing of a burned skin. This visualization is also of high interest for the evaluation of cosmetic product effects. To evaluate our results, the segmented images and their corresponding meshes have been presented to specialists in ultrasound images. The evaluation is only a qualitative one for the moment because producing ground truth for 3D images is a very hard task. Indeed, it is necessary to produce an expert segmentation for each two-dimensional cut (for example in z-axis direction) of a three-dimensional ultrasound image. With these set of two-dimensional ground truths it could be possible to construct a three-dimensional one but the gathering of the two-dimensional images could generate holes as well as precision problems. Thus, it stays difficult or even impossible to obtain a

**5. Texture-based supervised active contours driven by a classifier for 3D**

In this section, we present our second interactive texture-based model. The heart of the system is a 3D active contour, but for the user to easily deal with texture features, it has been chosen to guide the active contour segmentation with a supervised binary classifier. First, the principle of active contour segmentation will be presented. In a second part, we will describe the

Fig. 8. Segmentation of three-dimensional ultrasound images of the skin

fine resolution for the evaluation.

**ultrasound image segmentation**

represent regions using a mesh or to compute volume information to help specialists in their diagnostic.

Before running the segmentation, the user needs to select the features that seem the most relevant to process a 3D image. A graphic interface allows the user to define his choice (Figure 7). The selected textural features, the parameters and the processed volumetric image are then exploited by the module of feature computation. For each voxel, the selected textural features are computed. Then, the segmentation module uses the set of computed feature vectors to generate a first segmentation with the K-means algorithm (Coleman & Andrews (1979)). Finally, using the graphical interface, the user can improve the segmentation using merging and splitting operations. Then, it becomes possible to merge two regions or to focus on a particular region by running, once more, a segmentation (a Kmeans clustering) of this region.

Fig. 7. Interface of the proposed software for segmentation of 3D ultrasound images.

#### **4.3.2 Segmentation results**

Our software has been provided to several dermatologists in order to help them to collect 3D information about pathologies. Figure 8 presents segmentation results for different skin ultrasound images. Using a segmentation, many extractions are possible: lesions, tendons, skin layers etc. Figure 8[a-b] shows two naevi, figure 8[c-d] shows two Histiocytofibroma, figures 8[a-d] contain at the left the original three-dimensional ultrasound image, at the center an image of classified voxels and on the right a mesh of the lesion built using the segmented image. With these results, it becomes possible to perform measures like volume and depth, 14 Will-be-set-by-IN-TECH

represent regions using a mesh or to compute volume information to help specialists in their

Before running the segmentation, the user needs to select the features that seem the most relevant to process a 3D image. A graphic interface allows the user to define his choice (Figure 7). The selected textural features, the parameters and the processed volumetric image are then exploited by the module of feature computation. For each voxel, the selected textural features are computed. Then, the segmentation module uses the set of computed feature vectors to generate a first segmentation with the K-means algorithm (Coleman & Andrews (1979)). Finally, using the graphical interface, the user can improve the segmentation using merging and splitting operations. Then, it becomes possible to merge two regions or to focus on a particular region by running, once more, a segmentation (a Kmeans clustering) of this

Fig. 7. Interface of the proposed software for segmentation of 3D ultrasound images.

Our software has been provided to several dermatologists in order to help them to collect 3D information about pathologies. Figure 8 presents segmentation results for different skin ultrasound images. Using a segmentation, many extractions are possible: lesions, tendons, skin layers etc. Figure 8[a-b] shows two naevi, figure 8[c-d] shows two Histiocytofibroma, figures 8[a-d] contain at the left the original three-dimensional ultrasound image, at the center an image of classified voxels and on the right a mesh of the lesion built using the segmented image. With these results, it becomes possible to perform measures like volume and depth,

diagnostic.

region.

**4.3.2 Segmentation results**

(e) Layers of the skin

Fig. 8. Segmentation of three-dimensional ultrasound images of the skin

in order to help specialists in their diagnostic, to track pathologies evolution or to carry out more precise extractions etc. With a clustering, all the voxels of an image are classified and it is possible to make a visualization of the different skin layers(figure 8(e)). For a specialist, this visualization is interesting because skin layers evolve according to the patient age and it is interesting to supervise the healing of a burned skin. This visualization is also of high interest for the evaluation of cosmetic product effects. To evaluate our results, the segmented images and their corresponding meshes have been presented to specialists in ultrasound images. The evaluation is only a qualitative one for the moment because producing ground truth for 3D images is a very hard task. Indeed, it is necessary to produce an expert segmentation for each two-dimensional cut (for example in z-axis direction) of a three-dimensional ultrasound image. With these set of two-dimensional ground truths it could be possible to construct a three-dimensional one but the gathering of the two-dimensional images could generate holes as well as precision problems. Thus, it stays difficult or even impossible to obtain a fine resolution for the evaluation.

#### **5. Texture-based supervised active contours driven by a classifier for 3D ultrasound image segmentation**

In this section, we present our second interactive texture-based model. The heart of the system is a 3D active contour, but for the user to easily deal with texture features, it has been chosen to guide the active contour segmentation with a supervised binary classifier. First, the principle of active contour segmentation will be presented. In a second part, we will describe the

(PDE or motion equation) such as:

(1993); Malladi et al. (1995)).

**5.2 Supervised binary classification**

network (Rosenblatt (1958)).

following section.

**5.3 Interactive segmentation process**

(2008)).

**<sup>x</sup>** <sup>∈</sup>/ **<sup>R</sup>***m*.

*∂***C**(*s*, *t*)

PDE of models implicitly represented with level sets are usually deduced from the energy of the curve using Euler-Lagrange equations (Caselles et al. (1997); Zhu & Yuille (1996)), even though the first models were geometric ones, directly defined by their PDE (Caselles et al.

3D Ultrasound Image Segmentation: Interactive Texture-Based Approaches 59

As the goal of the presented application was to ensure the most precise segmentation on 3D ultrasound images, the developed model is implicitly represented and uses Haralick texture features (Haralick et al. (1973)). Moreover, in order to let our model choose the best texture features, we introduce a supervised binary classifier in the motion equation of the active contour. The next section presents the principle of supervised binary classification. Details about the introduction of the classifier in the motion equation can be found in (Olivier et al.

The principle of supervised binary classification is to observe a learning dataset **X** composed of several samples such as **<sup>X</sup>** <sup>=</sup> {**x1**, ..., **xn**}. For each sample **xi** <sup>∈</sup> **<sup>R</sup>***m*, *<sup>i</sup>* ∈ {1, ..., *<sup>n</sup>*}, its *<sup>m</sup>* features as well as the class to which its belongs are known and used to define a classification rule *l* = *f*(**x**) allowing the classifier to automatically determine the class *l* of each sample

In other words, given a set of already classified samples, a supervised classifier is able to

In the presented model, it has been chosen to give the classification task to a artificial neuron

As it as been presented in the previous section, using a supervised classifier requires the definition of a learning dataset composed of classified samples. As a consequence, our model evolves in two step. First, during an interactive step, an expert segmentation is carried out on one single slice of the 3D ultrasound image. The local Haralick texture features of some pixels from this segmentation are used to create the samples of a learning dataset, allowing the classifier to achieve its learning task. In a second step, a 3D active contour driven by this classifier is launched on the 3D image to carry out the desired segmentation. Figure 10 depicts the complete segmentation process. The extraction of the learning dataset is detailed in the

determine the class of every unknown sample according to its features, automatically.

The next section present the complete segmentation process.

with *F* the speed function of the model and **N** its inward normal vector.

*<sup>∂</sup><sup>t</sup>* <sup>=</sup> *<sup>F</sup>***<sup>N</sup>** , (10)

principle of supervised binary segmentation. Then, the complete segmentation process will be presented and finally, the segmentation software will be described.

#### **5.1 Active contour segmentation**

Initially developed by Kass et al. (1988), active contours are powerful segmentation approaches widely used in the segmentation field. An active contour is defined as a parametrized curve **C** mapping a parameter *s* to a pixel **x**(*s*) in the image Ω. The curve is initialized in Ω and evolves, under some constraints, according to its normal and tangential directions until it stops on the boundary of the object to be segmented. (see figure 9)

Fig. 9. Example of active contour segmentation

As the curve evolves in time, **C**(*s*, *t*) represents the family of curves obtained during the evolution. An energy is attached to the model and defined to control its evolution as:

$$E(\mathbf{C}(\mathbf{s},t)) = \int f(\mathbf{s},t)d\mathbf{s}\,,\tag{9}$$

*f* being a force composed of geometric criteria and external terms computed from the image (data terms). By minimizing (9), the model evolves until the final curve is placed on a position corresponding to a local minimum of *E*. Initially, the energy attached to the model was obtained by integrating *f* only along the curve (Caselles et al. (1997); Xu & Prince (1998)). These models are called boundary-based but their application is restricted to objects whose boundaries can be defined by gradients and are not very efficient on complex images like textures. Later, region-based models emerged by integrating *f* inside the curve or over the entire domain of Ω (Paragios & Deriche (2002); Zhu & Yuille (1996)).

Originally, Kass *et al* chose to represent the curve explicitly (the curve is sampled and its evolution is guided by several control points) but this prevents the model from handling topology changes without additional implementation. Yet, an implicit representation called the level set implementation emerged from the work of Osher and Sethian (Osher & Sethian (1988)). The main advantage of level sets is to allow the curve to handle topology changes automatically, as it can actually split or merge with others naturally. In order to implicitly represent a curve **C**, it is necessary to define its evolution with a partial differential equation (PDE or motion equation) such as:

16 Will-be-set-by-IN-TECH

principle of supervised binary segmentation. Then, the complete segmentation process will

Initially developed by Kass et al. (1988), active contours are powerful segmentation approaches widely used in the segmentation field. An active contour is defined as a parametrized curve **C** mapping a parameter *s* to a pixel **x**(*s*) in the image Ω. The curve is initialized in Ω and evolves, under some constraints, according to its normal and tangential

As the curve evolves in time, **C**(*s*, *t*) represents the family of curves obtained during the

*f* being a force composed of geometric criteria and external terms computed from the image (data terms). By minimizing (9), the model evolves until the final curve is placed on a position corresponding to a local minimum of *E*. Initially, the energy attached to the model was obtained by integrating *f* only along the curve (Caselles et al. (1997); Xu & Prince (1998)). These models are called boundary-based but their application is restricted to objects whose boundaries can be defined by gradients and are not very efficient on complex images like textures. Later, region-based models emerged by integrating *f* inside the curve or over the

Originally, Kass *et al* chose to represent the curve explicitly (the curve is sampled and its evolution is guided by several control points) but this prevents the model from handling topology changes without additional implementation. Yet, an implicit representation called the level set implementation emerged from the work of Osher and Sethian (Osher & Sethian (1988)). The main advantage of level sets is to allow the curve to handle topology changes automatically, as it can actually split or merge with others naturally. In order to implicitly represent a curve **C**, it is necessary to define its evolution with a partial differential equation

*f*(*s*, *t*)*ds* , (9)

evolution. An energy is attached to the model and defined to control its evolution as:

*E*(**C**(*s*, *t*)) =

entire domain of Ω (Paragios & Deriche (2002); Zhu & Yuille (1996)).

directions until it stops on the boundary of the object to be segmented. (see figure 9)

be presented and finally, the segmentation software will be described.

**5.1 Active contour segmentation**

Fig. 9. Example of active contour segmentation

*∂***C**(*s*, *t*) *<sup>∂</sup><sup>t</sup>* <sup>=</sup> *<sup>F</sup>***<sup>N</sup>** , (10)

with *F* the speed function of the model and **N** its inward normal vector.

PDE of models implicitly represented with level sets are usually deduced from the energy of the curve using Euler-Lagrange equations (Caselles et al. (1997); Zhu & Yuille (1996)), even though the first models were geometric ones, directly defined by their PDE (Caselles et al. (1993); Malladi et al. (1995)).

As the goal of the presented application was to ensure the most precise segmentation on 3D ultrasound images, the developed model is implicitly represented and uses Haralick texture features (Haralick et al. (1973)). Moreover, in order to let our model choose the best texture features, we introduce a supervised binary classifier in the motion equation of the active contour. The next section presents the principle of supervised binary classification. Details about the introduction of the classifier in the motion equation can be found in (Olivier et al. (2008)).

#### **5.2 Supervised binary classification**

The principle of supervised binary classification is to observe a learning dataset **X** composed of several samples such as **<sup>X</sup>** <sup>=</sup> {**x1**, ..., **xn**}. For each sample **xi** <sup>∈</sup> **<sup>R</sup>***m*, *<sup>i</sup>* ∈ {1, ..., *<sup>n</sup>*}, its *<sup>m</sup>* features as well as the class to which its belongs are known and used to define a classification rule *l* = *f*(**x**) allowing the classifier to automatically determine the class *l* of each sample **<sup>x</sup>** <sup>∈</sup>/ **<sup>R</sup>***m*.

In other words, given a set of already classified samples, a supervised classifier is able to determine the class of every unknown sample according to its features, automatically.

In the presented model, it has been chosen to give the classification task to a artificial neuron network (Rosenblatt (1958)).

The next section present the complete segmentation process.

#### **5.3 Interactive segmentation process**

As it as been presented in the previous section, using a supervised classifier requires the definition of a learning dataset composed of classified samples. As a consequence, our model evolves in two step. First, during an interactive step, an expert segmentation is carried out on one single slice of the 3D ultrasound image. The local Haralick texture features of some pixels from this segmentation are used to create the samples of a learning dataset, allowing the classifier to achieve its learning task. In a second step, a 3D active contour driven by this classifier is launched on the 3D image to carry out the desired segmentation. Figure 10 depicts the complete segmentation process. The extraction of the learning dataset is detailed in the following section.

with *k*1(*i*), ..., *km*(*i*) the *m* Haralick coefficients of the *i*

segmentation carried out during the interactive step.

band, c) definition of *Cin*, d) definition of *Cout*.

Fig. 12. The segmentation software interface.

**5.3.2 3D segmentation**

**5.4 Segmentation software**

object.

**5.4.1 Interface**

representing the region to which it belongs. Figure 11 shows an example of a manual

3D Ultrasound Image Segmentation: Interactive Texture-Based Approaches 61

(a) (b) (c) (d) Fig. 11. Example of expert segmentation. a): definition of *C*∗, b) definition of the narrow

Once the learning dataset is extracted, the automatic segmentation of the image is carried out. The learned classifier is used to guide the 3D active contour in the 3D image. The segmentation is stopped as the model reaches the boundaries of the object to be segmented. Once the segmentation is carried out, the final result is obtained by transforming the segmentation into a 3D mesh. Such a representation allows the user to easily understand the geometry of the

*th* extracted pixel and *l*(*i*) a label

Fig. 10. The complete segmentation process: the expert segmentation is used to determine the learning dataset and to carry out the classifier's learning task. The segmentation is then launched on the 3D image.

#### **5.3.1 Extraction of the learning dataset**

During this step, one single slice Ω∗ is extracted from the complete 3D utrasound image. Then, the user is asked to carry out an expert segmentation *C*∗. This segmentation must be composed of two ideal regions *Cin* and *Cout*. *Cin* will contain pixels from the object to be segmented while *Cout* will be composed of pixels belonging to a narrow band surrounding *Cin*. Intuitively, *Cin* should be taken as the interior of *C*<sup>∗</sup> and *Cout* as all the remaining image, giving <sup>Ω</sup><sup>∗</sup> <sup>=</sup> *<sup>C</sup>in* <sup>∪</sup> *<sup>C</sup>out*. But in real images the region outside the segmented object is rarely homogeneous and sometimes even contains regions with textures very close to those of the inside region. Moreover, even if this phenomenon does not appear in Ω∗, it can be present in other slices of the 3D image. This may involve a high variability in the textures from *Cout* in the 3D image, making it hard for the classifier to correctly carry out its task. Thus, in order to maximize the distance between textures from *Cin* and *Cout* (giving a higher discriminant power to the Haralick texture features and to keep texture properties as constant as possible, we chose to define *Cin* and *Cout* as respectively the interior of *C*<sup>∗</sup> and a narrow band giving homogeneous information about the boundary region of *C*∗.

Once the two regions determined, *m* Haralick features are computed for each of their pixels. Thus, a learning dataset **X** composed of *n* = *n*<sup>1</sup> + *n*<sup>2</sup> samples, *n*<sup>1</sup> belonging to *Cin* and *n*<sup>2</sup> belonging to *<sup>C</sup>out*, is determined. <sup>∀</sup>*<sup>i</sup>* <sup>∈</sup> [1, *<sup>n</sup>*], a sample *<sup>s</sup>*(*i*) will be described as:

$$\begin{aligned} s(i) &= \{ \, \, k\_1(i), k\_2(i)...k\_m(i), l(i) \}, \\ l(i) &= \begin{cases} -1 \text{ if } s(i) \text{ belongs to } \mathbb{C}^m \\ 1 \text{ if } s(i) \text{ belongs to } \mathbb{C}^{out} \end{cases} \end{aligned} \tag{11}$$

with *k*1(*i*), ..., *km*(*i*) the *m* Haralick coefficients of the *i th* extracted pixel and *l*(*i*) a label representing the region to which it belongs. Figure 11 shows an example of a manual segmentation carried out during the interactive step.

Fig. 11. Example of expert segmentation. a): definition of *C*∗, b) definition of the narrow band, c) definition of *Cin*, d) definition of *Cout*.

#### **5.3.2 3D segmentation**

18 Will-be-set-by-IN-TECH



> -

 ! -

 - -

Fig. 10. The complete segmentation process: the expert segmentation is used to determine the learning dataset and to carry out the classifier's learning task. The segmentation is then

During this step, one single slice Ω∗ is extracted from the complete 3D utrasound image. Then, the user is asked to carry out an expert segmentation *C*∗. This segmentation must be composed of two ideal regions *Cin* and *Cout*. *Cin* will contain pixels from the object to be segmented while *Cout* will be composed of pixels belonging to a narrow band surrounding *Cin*. Intuitively, *Cin* should be taken as the interior of *C*<sup>∗</sup> and *Cout* as all the remaining image, giving <sup>Ω</sup><sup>∗</sup> <sup>=</sup> *<sup>C</sup>in* <sup>∪</sup> *<sup>C</sup>out*. But in real images the region outside the segmented object is rarely homogeneous and sometimes even contains regions with textures very close to those of the inside region. Moreover, even if this phenomenon does not appear in Ω∗, it can be present in other slices of the 3D image. This may involve a high variability in the textures from *Cout* in the 3D image, making it hard for the classifier to correctly carry out its task. Thus, in order to maximize the distance between textures from *Cin* and *Cout* (giving a higher discriminant power to the Haralick texture features and to keep texture properties as constant as possible, we chose to define *Cin* and *Cout* as respectively the interior of *C*<sup>∗</sup> and a narrow band giving

Once the two regions determined, *m* Haralick features are computed for each of their pixels. Thus, a learning dataset **X** composed of *n* = *n*<sup>1</sup> + *n*<sup>2</sup> samples, *n*<sup>1</sup> belonging to *Cin* and *n*<sup>2</sup>

*s*(*i*) = { *k*1(*i*), *k*2(*i*)...*km*(*i*), *l*(*i*)} ,

<sup>−</sup>1 if *<sup>s</sup>*(*i*) belongs to *<sup>C</sup>in*

1 if *<sup>s</sup>*(*i*) belongs to *<sup>C</sup>out*, (11)

belonging to *<sup>C</sup>out*, is determined. <sup>∀</sup>*<sup>i</sup>* <sup>∈</sup> [1, *<sup>n</sup>*], a sample *<sup>s</sup>*(*i*) will be described as:


 -



 - 

homogeneous information about the boundary region of *C*∗.

*l*(*i*) =


- -

 -

 -


> --

> > -



 


launched on the 3D image.

**5.3.1 Extraction of the learning dataset**

Once the learning dataset is extracted, the automatic segmentation of the image is carried out. The learned classifier is used to guide the 3D active contour in the 3D image. The segmentation is stopped as the model reaches the boundaries of the object to be segmented. Once the segmentation is carried out, the final result is obtained by transforming the segmentation into a 3D mesh. Such a representation allows the user to easily understand the geometry of the object.

## **5.4 Segmentation software**

#### **5.4.1 Interface**


Fig. 12. The segmentation software interface.

The segmentation can be turned into a 3D mesh by pressing the "Create mesh" button (see figure 13.c). Figures 14.a and 14.b shows other segmentation results. As it has been presented in section 4.3.2, numerically validating a 3D segmentation method is not easy as 3D ground truths are hard to obtain. As a consequence, all segmentation results have been presented to

3D Ultrasound Image Segmentation: Interactive Texture-Based Approaches 63

(a)

(b)

In this chapter, two texture-based approaches for 3D ultrasound image segmentation have been presented. These methods have been developed for the user comfort and propose an

In the first method a multiresolution scheme for volumetric texture segmentation have been defined. It includes perceptual features inspired by a human way to describe textures. The main interests of this approach are an easier way for an operator (specialist or not in image analysis) to select the relevant texture features to process an image and a better contents interpretation in a given application. We also presented psychological experiments that prove the strong correspondence between the proposed features and human perception of textures. In the second approach, an interactive step is used where the user is asked to carry out a manual 2D segmentation on one single slice from the 3D image. Then, this segmentation is used to carry out the learning task of a supervised binary classifier. This last will determine the best texture features and will guide the segmentation of a 3D active contour among the 3D image, automatically. Thus, the major innovation of this work is to allow any non-expert user (in the image analysis domain) to achieve a texture-based segmentation. In some future work, this method will include more texture features in order to increase its adaptability.

Even if the applications presented in this chapter were focuses on ultrasound images, the two presented systems has also been successfully applied for the segmentation of MRI images as

dermatologists and visually validated.

Fig. 14. Other segmentation results using the software.

important usability and interactivity.

well as images obtained by confocal microscopy.

**6. Conclusion**

The figure 12 presents the complete segmentation interface. The left part (blue square) is the visualization area. It allows the user to see the complete 3D ultrasound image. The red square in the right shows the segmentation command buttons while the green square surrounds the interactive part dealing with the manual segmentation and the supervised classification.

#### **5.4.2 Interactive step**

Once the 3D image is open, the user select one slice from it and carry out the manual segmentation (see fig 13.a). Then, the learning step of the artificial neural network is launched by pressing the "Learn" button. This step will allow the software to extract Haralick texture features and to determine the best ones.

#### **5.4.3 Segmentation step**

Now that the texture features have been determined, the 3D segmentation can be launched. When pressing the "Launch segmentation" button, the user will see the 3D active contour grow in the image until it stops on the boundaries of the object to be segmented (see figure 13.b). It can be noticed that the average segmentation duration is about half a minute on a common computer. The final step of the segmentation is the 3D reconstruction of the model.

(a) (b)

(c)

Fig. 13. a) Interactive part of the software. b) 3D segmentation. c) 3D mesh

The segmentation can be turned into a 3D mesh by pressing the "Create mesh" button (see figure 13.c). Figures 14.a and 14.b shows other segmentation results. As it has been presented in section 4.3.2, numerically validating a 3D segmentation method is not easy as 3D ground truths are hard to obtain. As a consequence, all segmentation results have been presented to dermatologists and visually validated.

Fig. 14. Other segmentation results using the software.

## **6. Conclusion**

20 Will-be-set-by-IN-TECH

The figure 12 presents the complete segmentation interface. The left part (blue square) is the visualization area. It allows the user to see the complete 3D ultrasound image. The red square in the right shows the segmentation command buttons while the green square surrounds the interactive part dealing with the manual segmentation and the supervised classification.

Once the 3D image is open, the user select one slice from it and carry out the manual segmentation (see fig 13.a). Then, the learning step of the artificial neural network is launched by pressing the "Learn" button. This step will allow the software to extract Haralick texture

Now that the texture features have been determined, the 3D segmentation can be launched. When pressing the "Launch segmentation" button, the user will see the 3D active contour grow in the image until it stops on the boundaries of the object to be segmented (see figure 13.b). It can be noticed that the average segmentation duration is about half a minute on a common computer. The final step of the segmentation is the 3D reconstruction of the model.

(a) (b)

(c)

Fig. 13. a) Interactive part of the software. b) 3D segmentation. c) 3D mesh

**5.4.2 Interactive step**

**5.4.3 Segmentation step**

features and to determine the best ones.

In this chapter, two texture-based approaches for 3D ultrasound image segmentation have been presented. These methods have been developed for the user comfort and propose an important usability and interactivity.

In the first method a multiresolution scheme for volumetric texture segmentation have been defined. It includes perceptual features inspired by a human way to describe textures. The main interests of this approach are an easier way for an operator (specialist or not in image analysis) to select the relevant texture features to process an image and a better contents interpretation in a given application. We also presented psychological experiments that prove the strong correspondence between the proposed features and human perception of textures.

In the second approach, an interactive step is used where the user is asked to carry out a manual 2D segmentation on one single slice from the 3D image. Then, this segmentation is used to carry out the learning task of a supervised binary classifier. This last will determine the best texture features and will guide the segmentation of a 3D active contour among the 3D image, automatically. Thus, the major innovation of this work is to allow any non-expert user (in the image analysis domain) to achieve a texture-based segmentation. In some future work, this method will include more texture features in order to increase its adaptability.

Even if the applications presented in this chapter were focuses on ultrasound images, the two presented systems has also been successfully applied for the segmentation of MRI images as well as images obtained by confocal microscopy.

Olivier, J., Rousselle, J.-J., Boné, R. & Cardot, H. (2008). Active contours driven by supervised

3D Ultrasound Image Segmentation: Interactive Texture-Based Approaches 65

Osher, S. & Sethian, J. A. (1988). Fronts propagation with curvature-dependent speed:

Otsu, N. & Kurita, T. (1988). A new scheme for practical flexible and intelligent vision systems,

Paragios, N. & Deriche, R. (2002). Geodesic active regions and level set methods for supervised texture segmentation, *International Journal of Computer Vision* 46(3): 223–247. Paulhac, L., Makris, P., Gregoire, J.-M. & Ramel, J.-Y. (2009). Human understandable

Paulhac, L., Makris, P. & Ramel, J.-Y. (2009). A solid texture database for segmentation

Rosenblatt, F. (1958). The perceptron: A theory of statistical separability in cognitive systems,

Sahiner, B., Chan, H.-P., Roubidoux, M. A., Helvie, M. A., Hadjiiski, L. M., Ramachandran,

Shamos, M. I. & Hoey, D. (1975). Closest-point problems, *the* 16*th Annual Symposium on*

Shoshany, M. (2008). An evolutionary patch pattern approach for texture discrimination,

Tamura, H., Mori, S. & Yamawaki, T. (1978). Texture features corresponding to visual perception, *IEEE Transactions on Systems, Man, and Cybernetics* 8(6): 460–473. Tuceryan, M. & Jain, A. K. (1993). *Texture analysis*, World Scientific Publishing Co., Inc., River

Turner, M. R. (1986). Texture discrimination by Gabor functions, *Biological Cybernetics*

Valckx, M. J. F. & Thijssen, J. M. (1997). Characterization of echographic image texture by cooccurrence matrix parameters, *Ultrasound in Medicine and Biology* 23(4): 559–571. Voorhees, H. & Poggio, T. (1987). Detecting textons and texture boundaries in natural images,

Xu, C. & Prince, J. (1998). Snakes, shapes, and gradient vector flow, *IEEE Transactions on Image*

Zhan, Y. & Shen, D. (2003). Automated segmentation of 3d us prostate images using

*Conference on Medical Image Computing and Computer-Assisted Intervention*.

*proceedings of the International Conference on Computer Vision (ICCV87)*, London,

statistical texture-based matching method, *MICCAI '03: Proceedings of the International*

*Technical report VG-1196-G-1* M. A. Cornell Aeronautical Laboratory.

*IAPR Workshop on Computer Vision*, Tokyo, Japan, pp. 431–435.

*Computing : ISVC 08*, Las Vegas, Nevada.

*Symposium on vision computing*, pp. 379–390.

*Foundations of Computer Science* pp. 131–162.

*Pattern Recognition* 41: 2327–2336.

Edge, NJ, USA, pp. 235–276.

England, pp. 250–258.

*Processing* 7(3): 359–369.

55: 443–457.

*Conference on Computer Vision Theory and Applications*. Richards, W. & Polit, A. (1974). Texture matching, *Kibernetic* 16: 155–162.

79: 12–49.

*Physics* 31: 744–754.

binary classifiers for texture segmentation, *the 4th International Symposium on Visual*

algorithms based on Hamilton-Jacobi formulations, *Journal of Computational Physics*

features for segmentation of solid texture, *ISVC '09: Proceedings of the 5th International*

and classification experiments, *VISSAPP '09: Proceedings of the Fourth International*

A., Paramagul, C., LeCarpentier, G. L., Nees, A. & Blane, C. (2004). Computerized characterization of breast masses on three-dimensional ultrasound volumes, *Medical*

#### **7. References**


22 Will-be-set-by-IN-TECH

Amadasun, M. & King, R. (1989). Texture features corresponding to textural properties, *IEEE*

Azencott, R., Wang, J.-P. & Younes, L. (1997). Texture classification using windowed Fourier filters, *IEEE Transactions on Pattern Analysis and Machine Intelligence* 19(2): 148–153. Basset, O., Sun, Z., Mestas, J. & Gimenez, G. (1993). Texture analysis of ultrasonic images of the prostate by means of co-occurrence matrices, *Ultrasonic Imaging* 15: 218–237. Boukerroui, D., Basset, O., Baskurt, A. & Gimenez, G. (2001). A multiparametric

Caselles, V., Kimmel, R. & Sapiro, G. (1997). Geodesic active contours, *International Journal of*

Coleman, G. & Andrews, H. (1979). Image segmentation by clustering, *Proceedings of the IEEE*,

Goyal, R., Goh, W., Mital, D. & Chan, K. (1995). Scale and rotation invariant texture analysis

Grégoire, J.-M., Serrière, S., Georgesco, G., Jamet, F., Bleuzen, A., Ossant, F., Levassort, F.,

Haralick, R. M., Shanmugam, K. & Dinstein, I. (1973). Texture features for image classification,

Kopf, J., Fu, C.-W., Cohen-Or, D., Deussen, O., Lischinski, D. & Wong, T.-T. (2007).

Malladi, R., Sethian, J. A. & Vemuri, B. C. (1995). Shape modeling with front propagation:

Mallat, S. G. (1989). A theory for multiresolution signal decomposition: the wavelet

Ojala, T., Pietikäinen, M. & Harwood, D. (1996). A comparative study of texture measures with classification based on feature distributions, *Pattern Recognition* 29(1): 51–59.

Mandelbrot, B. B. (1977). *Fractals, Forms, Chance and Dimension*, Freeman, San Fransisco. Noble, J. A. & Boukerroui, D. (2006). Ultrasound image segmentation: A survey, *IEEE*

*International Conference on Computer Graphics and Interactive Techniques*. Li, S. Z. (1995). *Markov Random Field Modeling in Computer Vision*, Springer-Verlag New York,

Julesz, B. (1975). *Experiments in the visual perception of texture*, Vol. 232, Scientific American. Kass, M., Witkin, A. & Terzopoulos, D. (1988). Snakes: active contour models, *International*

based on structural property, *IECON '95: Proceedings of the International Conference on*

Tranquart, F. & Patat, F. (2006). Techniques et applications de l'échographie haute

Solid texture synthesis from 2d exemplars, *SIGGRAPH '07: Proceedings of the 34th*

a level set approach, *IEEE Transactions on Pattern Analysis and Machine Intelligence*

representation, *IEEE Transactions on Pattern Analysis and Machine Intelligence*

and multiresolution segmentation algorithm of 3-d ultrasonic data, *TUFFC '01: Proceedings of the IEEE transactions on ultrasonics, ferroelectrics, and frequency control*. Caselles, V., Catte, F., Coll, T. & Dibos, F. (1993). A geometric model for active contours,

*Transactions on Systems, Man, and Cybernetics* 19(5): 1264–1274.

*Numerishe Mathematik* 66: 1–31.

*Computer Vision* 22(1): 61–79.

*Industrial Electronics, Control, and Instrumentation*.

*Journal of Computer Vision* 1(4): 321–331.

*Transactions on Medical Imaging* 25(8): 987–1010.

résolution non invasive, *Journal de Radiologie* 87: 1920–1936.

*IEEE Transactions on Systems, Man, and Cybernetics* 3(6): 610–621.

pp. 773–785.

Inc.

17(2): 158–175.

11: 674–693.

**7. References**


**4** 

*1,3Romania 2USA* 

**Radiation Protection in Medical Imaging** 

*1Iuliu Haţieganu University of Medicine and Pharmacy, Cluj-Napoca* 

*3Technical University, Cluj-Napoca* 

Horaţiu Colosi1, Dan Colosi2, Vlad Mureşan3 and Marius Roman3

*2State University of New York, School of Dental Medicine, Stony Brook, New York* 

Life on Earth has developed continuously in the presence of natural radiation. Radiation is a form of energy transfer through space and matter. The ability of radiation to transfer energy at the molecular level to living organisms is at the root of current efforts to understand and

The radiation spectrum includes forms of radiation that carry various energy levels. Radiation forms, such as x-radiation, which carry sufficient energy to overcome binding energies of electrons and to eject them from their atomic orbits, can cause the formation of ions in the matter through which they travel, and are termed ionizing radiation. Ionizing radiation may also dissociate molecular bonds, leading to the formation of free radicals.

A great proportion of the effects of ionizing radiation on living matter are a result of the interaction of these forms of radiation with water, a ubiquitous molecule that is present in most tissues and organs. This interaction leads to the generation of highly reactive radical species, mainly from the hydroxyl and hydroperoxyl categories, which, in turn, can interact with cellular DNA, leading to the formation of strand breaks and base damage (Hall & Giaccia 2005). To a lesser extent, radiation can also directly ionize or damage DNA and other biologic molecules: RNA, proteins, lipids. Many occurrences of damage limited to one DNA strand are rapidly detected by the surveillance mechanisms of the cell and the DNA integrity is restored using the intact complementary DNA strand as template. However, double-strand breaks, in which an intact DNA template is not available, are more difficult to repair accurately (Mladenov & Iliakis 2011). As a result, radiation-induced DNA double strand breaks may often lead to misrepaired DNA, resulting in point mutations, deletions or chromosomal translocations. Radiation-induced DNA damage that escapes detection by cell cycle checkpoints during cellular mitosis can result in an uncontrolled, accelerated rate of mitotic division of the affected cell (Hall & Giaccia 2005), a cellular hallmark of cancer. The association between exposure to radiation and cancer induction is well documented through

**1. Introduction** 

limit the effects of radiation on humans.

**1.1 Biologic effects of ionizing radiation** 

epidemiological studies (Ron 2003).

