**Identification of Structures in Medical Images<sup>1</sup>**

Marina de Sá Rebelo1, Sérgio Shiguemi Furuie2, Lincoln de Assis Moura Jr2, Eduardo Tavares Costa3,4 and Marco Antonio Gutierrez1 *1Heart Institute (InCor) - University of São Paulo Medical School 2Polytechnic School – University of São Paulo 3School of Electrical and Computer Engineering- University of Campinas 4Center for Biomedical Engineering - University of Campinas Brazil* 

## **1. Introduction**

The development of automatic systems for medical image processing, which can effectively act as an agent to aid medical diagnosis, is a goal that has been pursued by researchers since the first works on the field of medical image processing in the 80´s. The automation of image analysis tasks can produce very interesting results such as less time spent by specialists, decrease of intra-and inter-observer differences, second opinions to non-specialists and in educational systems. Any automatic system deployed for analysis and visualization of images involves the identification of objects and often the relationships among them. The automatic identification of structures is a research area of image processing that still has great challenges. In tasks that involve primarily activities of calculation, the computer's processing power is incomparably higher than the humans. However, in recognition and analysis tasks, the human brain possesses a strong ability that is not obtained by any computational system. For a given scene, humans possess a unique ability to distinguish the objects that are significant and, among them, those that represent the focus of interest for a particular situation.

In the routine of medical image analysis, doctors often deal with images in which the visual perception is not sufficient for identification of some particular structure under study. They need, therefore, conceptual hypotheses so as to perform this task. At each new image analyzed a doctor uses a huge amount of knowledge accumulated over the years of clinical practice. This is a typical feedback and a case base reasoning system, as each new image analyzed gives the doctor more knowledge about a specific problem. A system of medical aid for identification and analysis of structures should also be able to incorporate and use knowledge about the problem domain.

## **2. Identification of structures in digital images**

## **2.1 Problem statement**

Automatic recognition and identification of structures are fields of Computer Vision. The recognition of structures is a difficult task because many factors influence its computation.

<sup>1</sup> Portions adapted, with permission, from *Computers in Biology and Medicine* 37 (2007) 1183 – 1193

Identification of Structures in Medical Images 5

goals and strategies on how to achieve those goals more efficiently. Using knowledge about the image structure, image data is systematically evaluated. Hypotheses are confirmed or rejected by the data. Routines for processing high-level information should be organized to produce rapid searches. The difference between high-level and intermediate level processing is the explicit use of knowledge about the context of the image under analysis.

In medical applications, the ultimate goal of automatic image analysis is to obtain not only qualitative results, but also quantitative data for assessing objectively the changes in relation to normal standards. To obtain such results it is necessary first to identify the object of interest and then carefully analyze it. Among the applications for which identification of structures is an important step are: visualization and surgical planning, planning for radiotherapy procedures, identification of tumors, malformations and dysfunctions, classification of organs,

In the case of image volumes, or 3D image data, automatic identification involves a large number of operations starting from the original acquired data: they must be processed so that object slices are formed by applying filters and reconstruction algorithms. Once the slices have been obtained, further transformations in the image are needed in order to make the data more suitable for identification. Such transformations may include processes as edge detection algorithms and structure recognition. Then high level procedures involving the use of *a priori* knowledge to determine the searched structure are used and, specific

The results of methods for identification of structures in medical images can be affected by image artifacts. Images contain noise that can alter the intensity of a pixel such that its classification becomes uncertain. They also have finite pixel size and, for this reason, are subject to partial volume averaging effect where individual pixel volumes contain a mixture of tissue classes so that the intensity of a pixel in the image may not be consistent with any one structure. The intensity level of a single tissue class can vary over the extent of the image, and internal materials (for instance surgical clips) may cause distortion in the imaged organ (Rani et al, 2011; Withey & Koles, 2007). Some of these effects are depicted in Figure 1.

Fig. 1. Examples of effects that can affect structure identification can be seen in the three images: (A) kidney in Ultrasound (US), (B) lung in Nuclear Medicine (NM) and (C) brain in Computed Tomography (CT). The noise associated with each image modality has to be dealt

with in different ways. In image (C) the presence of dental metal pieces degraded the reconstructed slice of the brain (images acquired at the Imaging Department - Heart

tissues and cells, and retrieval of similar cases in image content based databases.

**3. Automatic identification in medical images** 

analysis of the found structures can be performed

Institute – HCFMUSP- with patient anonymization).

These can include restrictions on permissible forms, the semantics of the context of the scene and the information present in the image itself (Suetens et al, 1992). After recognizing the set of objects in the scene, the next step in the interpretation process is the identification of one or more objects of particular interest in the recognized set. There is a distinction between the concepts of recognition and identification of structures. There are many definitions in the literature for recognition (or discovery) of structures. We use here a simple definition, given by Suetens et al (1992):

*"The recognition (or detection) of structures is the task of finding and labeling parts of a digital image that correspond to objects in the scene."* 

Once the recognition step has been performed, the next step is to identify one or all instances of a particular structure. The definition of object identification is not precise and, depending on the application, may be mistaken for recognition. In this text, identification is defined by the authors as:

*The identification of a particular set of structures is to set apart from the recognized structures the one or ones that correspond to the object under study.* 

The term segmentation is sometimes used by researchers as a synonym to identification as defined above. The canonical definition of segmentation is the subdivision of the image into its components (Gonzales & Woods, 2000; Jain, 1989). Strictly speaking, segmentation corresponds only to the first step of the recognition process.

#### **2.2 Automatic identification**

The steps involved in the automatic identification of structures are usually divided into different levels. Some authors have proposed a division into two levels: low and high (Ballard & Brown, 1982; Sonka et al, 1998) while others use three processing levels: low, medium and high (Gonzales & Woods, 2000; Niessen, 1997).

The low processing level deals with functions similar to automated human visual reactions that normally require little or no knowledge about the content of the image (Gonzalez & Woods, 2000; Sonka, 1998). It is mainly or almost entirely based on intrinsic image data, without inclusion of *a priori* information. It includes activities such as the basic process of image formation, filtering for noise reduction and correction of acquisition artifacts. In addition, there are processes to decrease the huge amount of data present in the original image to obtain parametric images describing relevant information. Examples of relevant parameters are local contrast, texture, curvature and movement. At the intermediate processing level the tasks are performed by extraction and characterization of components in the images resulting from the low level process. This stage includes activities such as identification, representation and description of the image information through models. For authors who do not use this classification, these functions are usually defined as low-level processing (Ballard & Brown, 1982; Sonka, 1998). The authors that include the intermediate level point out that the difference to the low level processing is that, based on some restrictions, it is possible to select - or highlight - relevant information to a particular application and formalize this information using models that include some information *a priori* about the application domain, thus making it easier the subsequent high-level tasks (Niessen, 1997). Moreover, the result of this processing step generates no more pictures, but representation of objects in the picture. The high processing level is based on knowledge,

These can include restrictions on permissible forms, the semantics of the context of the scene and the information present in the image itself (Suetens et al, 1992). After recognizing the set of objects in the scene, the next step in the interpretation process is the identification of one or more objects of particular interest in the recognized set. There is a distinction between the concepts of recognition and identification of structures. There are many definitions in the literature for recognition (or discovery) of structures. We use here a simple definition, given

Once the recognition step has been performed, the next step is to identify one or all instances of a particular structure. The definition of object identification is not precise and, depending on the application, may be mistaken for recognition. In this text, identification is

The term segmentation is sometimes used by researchers as a synonym to identification as defined above. The canonical definition of segmentation is the subdivision of the image into its components (Gonzales & Woods, 2000; Jain, 1989). Strictly speaking, segmentation

The steps involved in the automatic identification of structures are usually divided into different levels. Some authors have proposed a division into two levels: low and high (Ballard & Brown, 1982; Sonka et al, 1998) while others use three processing levels: low,

The low processing level deals with functions similar to automated human visual reactions that normally require little or no knowledge about the content of the image (Gonzalez & Woods, 2000; Sonka, 1998). It is mainly or almost entirely based on intrinsic image data, without inclusion of *a priori* information. It includes activities such as the basic process of image formation, filtering for noise reduction and correction of acquisition artifacts. In addition, there are processes to decrease the huge amount of data present in the original image to obtain parametric images describing relevant information. Examples of relevant parameters are local contrast, texture, curvature and movement. At the intermediate processing level the tasks are performed by extraction and characterization of components in the images resulting from the low level process. This stage includes activities such as identification, representation and description of the image information through models. For authors who do not use this classification, these functions are usually defined as low-level processing (Ballard & Brown, 1982; Sonka, 1998). The authors that include the intermediate level point out that the difference to the low level processing is that, based on some restrictions, it is possible to select - or highlight - relevant information to a particular application and formalize this information using models that include some information *a priori* about the application domain, thus making it easier the subsequent high-level tasks (Niessen, 1997). Moreover, the result of this processing step generates no more pictures, but representation of objects in the picture. The high processing level is based on knowledge,

*structures the one or ones that correspond to the object under study.* 

*digital image that correspond to objects in the scene."* 

corresponds only to the first step of the recognition process.

medium and high (Gonzales & Woods, 2000; Niessen, 1997).

*"The recognition (or detection) of structures is the task of finding and labeling parts of a* 

*The identification of a particular set of structures is to set apart from the recognized* 

by Suetens et al (1992):

defined by the authors as:

**2.2 Automatic identification** 

goals and strategies on how to achieve those goals more efficiently. Using knowledge about the image structure, image data is systematically evaluated. Hypotheses are confirmed or rejected by the data. Routines for processing high-level information should be organized to produce rapid searches. The difference between high-level and intermediate level processing is the explicit use of knowledge about the context of the image under analysis.

## **3. Automatic identification in medical images**

In medical applications, the ultimate goal of automatic image analysis is to obtain not only qualitative results, but also quantitative data for assessing objectively the changes in relation to normal standards. To obtain such results it is necessary first to identify the object of interest and then carefully analyze it. Among the applications for which identification of structures is an important step are: visualization and surgical planning, planning for radiotherapy procedures, identification of tumors, malformations and dysfunctions, classification of organs, tissues and cells, and retrieval of similar cases in image content based databases.

In the case of image volumes, or 3D image data, automatic identification involves a large number of operations starting from the original acquired data: they must be processed so that object slices are formed by applying filters and reconstruction algorithms. Once the slices have been obtained, further transformations in the image are needed in order to make the data more suitable for identification. Such transformations may include processes as edge detection algorithms and structure recognition. Then high level procedures involving the use of *a priori* knowledge to determine the searched structure are used and, specific analysis of the found structures can be performed

The results of methods for identification of structures in medical images can be affected by image artifacts. Images contain noise that can alter the intensity of a pixel such that its classification becomes uncertain. They also have finite pixel size and, for this reason, are subject to partial volume averaging effect where individual pixel volumes contain a mixture of tissue classes so that the intensity of a pixel in the image may not be consistent with any one structure. The intensity level of a single tissue class can vary over the extent of the image, and internal materials (for instance surgical clips) may cause distortion in the imaged organ (Rani et al, 2011; Withey & Koles, 2007). Some of these effects are depicted in Figure 1.

Fig. 1. Examples of effects that can affect structure identification can be seen in the three images: (A) kidney in Ultrasound (US), (B) lung in Nuclear Medicine (NM) and (C) brain in Computed Tomography (CT). The noise associated with each image modality has to be dealt with in different ways. In image (C) the presence of dental metal pieces degraded the reconstructed slice of the brain (images acquired at the Imaging Department - Heart Institute – HCFMUSP- with patient anonymization).

Identification of Structures in Medical Images 7

Techniques specifically designed for computerized detection of abnormalities in chest radiographs were among the first works in the area of identification of structures in medical images (Duncan & Ayache, 2000). The field has evolved continuously since then and the amount of knowledge has grown constantly with a huge number of methods described in the literature, as pointed out in many survey papers (Bandyopadhyay, 2011; Duncan & Ayache, 2000; Kirbas & Quek, 2002; Lesage et al, 2009; Ma et al, 2008; Oliver et al, 2010;

The assumptions and requirements taken by identification methods vary substantially for different body organs and imaging modalities. Therefore, the choice of the method will be dependent on the particular application, regarding the organ or system under study, as well as the imaging modality (Bandyopadhyay, 2011). There have been described general methods that intended to be more comprehensive, and therefore can be applied to a number of applications. However, their performance is worse than that of methods designed for specific applications, as the latter generally use *a priori* information available about the

In the next paragraphs we will present some methods proposed in recent years. A general overview of the methods described for medical image identification in the last decades is presented in the first section. The next sections are dedicated to clinical applications with emphasis on the organ under study, presenting the well-known specific challenges for identifying the particular organ and exploring some recently published methods. This text avoids performance comparisons due to fact that existing applications are so different in their purposes and practical applications, that direct comparisons do not make sense.

There is a number of papers discussing the evolution of the algorithms for extraction or identification of medical structures. It is out of the scope of this chapter to present and compare all studies and methodologies for classifying the existing methods. Instead, we present a description of the field evolution based on two surveys, presented by Ma et al (2008) and Witney & Koles (2007), which we believe presents the evolution of the field in a simple and informative way. For the other reviews, readers are referred to the literature (Duncan & Ayache, 2002; Petitjean et al, 2011). Withey and Koles (2007) classified the medical image identification approaches in literature into three generations, while Ma et al (2008) classified the methods in three categories that can be fitted to the three generations

The first generation techniques are those almost based on intrinsic image information,

Threshold: algorithms belonging to this type assume that the searched structures are

 Region growing: from an initial seed located in the image, adjacent pixels are checked against a set of predefined homogeneity criteria. Pixels that meet the criteria are

 Edge tracing: after the application of an edge detection algorithm, the edge pixels with adjacent neighbor connectivity are followed sequentially and collected into a list to

related to clearly quantifiable features as image intensity and gradient.

**3.2 A review of applications in medical imaging** 

PetitJean & Dacher, 2011; Witney & Koles,2007).

problem under study (Bandyopadhyay, 2011).

model of Witney and Koles.

included in the region.

represent an object boundary.

without any *a priori* information. They include:

**3.2.1 Evolution of the methods along the last decades** 

All these factors may affect the identification of organs due to poorly defined boundaries, blur or weak edges homogeneity. Moreover, the identification of organs in medical imaging presents some unique challenges as the creation of efficient models for representation of biological structures that are not easily described by mathematical models. All these factors, added to the variability in tissue distribution among individuals in the human population, means that some degree of uncertainty must be attached to all segmentation results. Figure 2 exemplifies the difficulty in identifying brain structures in Computerized Tomography (CT) and Magnetic Resonance (MR) images.

Fig. 2. Example challenging identification tasks in brain images. The areas inside the red circles are the areas to be identified. Due to the aforementioned problems, the identification of some tissues in medical images can be difficult even in modalities with high resolution as CT (left image) and MR (middle and right) (images acquired at the Imaging Department - Heart Institute – HCFMUSP- with patient anonymization).

## **3.1 Use of a priori information for identification in medical images**

In the context of medical images, the concept of identification, as defined above, involves the discrimination against a particular organ or the discrimination between normality and abnormality for a particular organ, based on the attributes of the image. Although trained clinicians are able to find and classify body structures precisely, there is not a technique that can reproduce precisely their ability in computational environments. The use of simple methods that rely only on information contained in the image data, in general, does not produce satisfactory results. In other words, the intrinsic information of the image is not enough to characterize the structures present in real objects (Suetens et al, 1992). Effective procedures for identifying structures should incorporate knowledge of the model or the context.

There are several studies in the literature aiming the characterization of the knowledge about a particular organ in a medical image modality to aid the automatic processing of the image. Typically, the methods use information about shape, size, texture and position of a given structure. A major difficulty in using *a priori* information about the forms of the organs is to characterize them geometrically. Unlike the rigid objects from other applications of computer vision, most human organs have few calculable and stable invariants. Apart from that, the features to be detected in images may be small and subtle, sometimes characterized only by tiny changes in gray level. In many medical applications there is a significant amount of known information about the human anatomy. However, there is a major difficulty involving the creation of models to represent biological structures, comprising very complex forms that are not described easily. Furthermore, for a given population these structures usually show variations both in form and size.

All these factors may affect the identification of organs due to poorly defined boundaries, blur or weak edges homogeneity. Moreover, the identification of organs in medical imaging presents some unique challenges as the creation of efficient models for representation of biological structures that are not easily described by mathematical models. All these factors, added to the variability in tissue distribution among individuals in the human population, means that some degree of uncertainty must be attached to all segmentation results. Figure 2 exemplifies the difficulty in identifying brain structures in Computerized Tomography (CT)

Fig. 2. Example challenging identification tasks in brain images. The areas inside the red circles are the areas to be identified. Due to the aforementioned problems, the identification of some tissues in medical images can be difficult even in modalities with high resolution as CT (left image) and MR (middle and right) (images acquired at the Imaging Department -

In the context of medical images, the concept of identification, as defined above, involves the discrimination against a particular organ or the discrimination between normality and abnormality for a particular organ, based on the attributes of the image. Although trained clinicians are able to find and classify body structures precisely, there is not a technique that can reproduce precisely their ability in computational environments. The use of simple methods that rely only on information contained in the image data, in general, does not produce satisfactory results. In other words, the intrinsic information of the image is not enough to characterize the structures present in real objects (Suetens et al, 1992). Effective procedures for identifying structures should incorporate knowledge of the

There are several studies in the literature aiming the characterization of the knowledge about a particular organ in a medical image modality to aid the automatic processing of the image. Typically, the methods use information about shape, size, texture and position of a given structure. A major difficulty in using *a priori* information about the forms of the organs is to characterize them geometrically. Unlike the rigid objects from other applications of computer vision, most human organs have few calculable and stable invariants. Apart from that, the features to be detected in images may be small and subtle, sometimes characterized only by tiny changes in gray level. In many medical applications there is a significant amount of known information about the human anatomy. However, there is a major difficulty involving the creation of models to represent biological structures, comprising very complex forms that are not described easily. Furthermore, for a given

Heart Institute – HCFMUSP- with patient anonymization).

**3.1 Use of a priori information for identification in medical images** 

population these structures usually show variations both in form and size.

and Magnetic Resonance (MR) images.

model or the context.

#### **3.2 A review of applications in medical imaging**

Techniques specifically designed for computerized detection of abnormalities in chest radiographs were among the first works in the area of identification of structures in medical images (Duncan & Ayache, 2000). The field has evolved continuously since then and the amount of knowledge has grown constantly with a huge number of methods described in the literature, as pointed out in many survey papers (Bandyopadhyay, 2011; Duncan & Ayache, 2000; Kirbas & Quek, 2002; Lesage et al, 2009; Ma et al, 2008; Oliver et al, 2010; PetitJean & Dacher, 2011; Witney & Koles,2007).

The assumptions and requirements taken by identification methods vary substantially for different body organs and imaging modalities. Therefore, the choice of the method will be dependent on the particular application, regarding the organ or system under study, as well as the imaging modality (Bandyopadhyay, 2011). There have been described general methods that intended to be more comprehensive, and therefore can be applied to a number of applications. However, their performance is worse than that of methods designed for specific applications, as the latter generally use *a priori* information available about the problem under study (Bandyopadhyay, 2011).

In the next paragraphs we will present some methods proposed in recent years. A general overview of the methods described for medical image identification in the last decades is presented in the first section. The next sections are dedicated to clinical applications with emphasis on the organ under study, presenting the well-known specific challenges for identifying the particular organ and exploring some recently published methods. This text avoids performance comparisons due to fact that existing applications are so different in their purposes and practical applications, that direct comparisons do not make sense.

#### **3.2.1 Evolution of the methods along the last decades**

There is a number of papers discussing the evolution of the algorithms for extraction or identification of medical structures. It is out of the scope of this chapter to present and compare all studies and methodologies for classifying the existing methods. Instead, we present a description of the field evolution based on two surveys, presented by Ma et al (2008) and Witney & Koles (2007), which we believe presents the evolution of the field in a simple and informative way. For the other reviews, readers are referred to the literature (Duncan & Ayache, 2002; Petitjean et al, 2011). Withey and Koles (2007) classified the medical image identification approaches in literature into three generations, while Ma et al (2008) classified the methods in three categories that can be fitted to the three generations model of Witney and Koles.

The first generation techniques are those almost based on intrinsic image information, without any *a priori* information. They include:


Identification of Structures in Medical Images 9

Most automatic systems for analysis of brain images have three processing steps: (i) brain tissue identification; (ii) registration: the voxels of interest are matched to a template or to an earlier scan from the same individual; (iii) statistical comparison of different groups of

In their review of the methods to assess brain structures, Ashburner et al (2003) discuss the strengths and limitations of algorithms for identification and registration of brain images, together with evidence of their usefulness at the clinical and research level. A more recent review of current methods used for computer automated identification in brain anatomical images is presented by Bandyopadhyay (2011). Most methods for extraction of brain structures and tissue segmentation use some kind of *a priori* information. Some authors, however, try to minimize the need of *a priori* knowledge. As an example, Scherrer et al (2009) extracted tissue and sub-cortical structures in MR images by considering a local approach to cope with intensity non-uniformity and a multi-agent based implementation.

Cardiovascular diseases are the leading cause of death in developed countries (Allender et al, 2008). Imaging techniques are nowadays among the most valuable tools in helping diagnosis, treatment and follow-up of cardiovascular pathologies, as they provide qualitative and quantitative information of the heart, which would be impossible to obtain otherwise (Cordero-Grande et al, 2011). Diagnosis and treatment follow-up of these pathologies can rely on numerous cardiac imaging modalities, which include echography, CT (computerized tomography), digital subtractive angiography, coronary angiography

**Heart.** Quantitative analysis of the heart structure and function can be performed by the extraction of cardiac descriptors from medical images. These descriptors can provide information about global and local function. This includes estimating left ventricle volume, ejection fraction, cardiac output, myocardial mass, myocardial morphology, myocardial uptake rates, tissue viability, blood flow, local motion and deformation (wall thickening). The determination of most of these descriptors involves an appropriate 3D identification of cardiac structures – being the cardiac chambers, cardiac muscle or cardiac valves - as a prerequisite (Alattar et al, 2010; Cordero-Grande et al, 2011). The imaging modality of choice will depend on the problem under study and the complexity of the identification of the

Automatic isolation of the heart structures in MR images is a challenging task because of the inherent noise associated with cine MRI, caused by factors such as patient movement, cardiac dynamics, and complex intensity characteristics (Lynch, 2008). Even in noiseless cases, MR images may present blurred edges due to the partial volume effect and can be affected by motion artifacts. In addition, low contrast between the myocardium and surrounding tissues complicates the identification of the cardiac structures. As an example, the papillary muscles, the trabeculae, or the liver have pixel intensity values which are virtually indistinguishable from those of the myocardium (Cordero-Grande et al, 2011). In the literature there is a huge number of recent works on heart identification in MR images (Alattar et al, 2010; Cordero-Grande et al, 2011; Lekadir et al, 2011; O´Brien et al, 2011; O'Donnell et al, 2006; Rouchdy et al, 2007; Schaerer et al, 2010; Zhao et al, 2009). A

patients or same patient in different times.

**3.2.3 Cardiovascular applications** 

cardiac structures will depend on the modality chosen.

and cardiac MRI.

The second generation is composed of algorithms using image models, optimization methods, and uncertainty models, and includes:


The third generation is characterized by algorithms that are capable of incorporating knowledge, such as:


#### **3.2.2 Brain applications**

Environmental factors, age and disease can affect neuroanatomical structures during the process of ageing. These structures are also affected by genetic factors in patients with degenerative diseases (Ashburner et al, 2003). Besides providing anatomical information, medical imaging procedures are able to provide extremely relevant information about brain physiology, which can be used to understand physical and psychological clinical conditions.

In order to capture the extraordinary morphological variability of the human brain, a number of automatic methods for identification and analysis of structures have been developed for different imaging modalities (Ashburner et al, 2003; Bandyopadhyay, 2011; Bresser et al, 2011; Ishii et al, 2009; Lopes et al, 2008; Ribbens et al, 2010; Rousset et al, 2007; Shiee et al, 2010; Tu & Bai, 2010; Yi et al, 2009; Zhang et al, 2011).

The second generation is composed of algorithms using image models, optimization

 Pattern recognition and clustering: as certain structures in medical images can be treated as patterns, segmentation algorithms that combine pattern recognition

 Deformable models: deformable models use curve evolution to perform segmentation. A moving equation should be defined to drive the initial curves to the right structure

 Graph search: image pixels are used to form nodes in a graph and the nodes are interconnected to neighbors, mapping the corresponding pixel associations in the image. Costs are assigned for each interconnection. Algorithms from combinatorial

 Multiresolution methods: multiresolution or multiscale analysis refers to the use of scale reduction to group pixels into image objects. A stack of images is formed by recursively reducing the scale of the original image by blurring followed by down sampling. The central idea in this category is that there may be scales that are more

The third generation is characterized by algorithms that are capable of incorporating

 Shape and appearance models: the active shape model (ASM) was inspired by deformable models with the added intention of limiting the extent of the model deformation. A statistical representation of an object is formed by identifying a set of landmark points on an object boundary and analyzing the variation of each across a set

 Atlas-based: it is a generic technique for automatic delineation of structures in volumetric images, which starts by registering an anatomical image from an atlas with a target image to be segmented. A critical underlying assumption is that it is possible to find a deformation that aligns the atlas with the target image in such a way that label

 Rule-based: image primitives are usually derived from first-generation and secondgeneration algorithms and then interpreted using anatomical and image knowledge

Environmental factors, age and disease can affect neuroanatomical structures during the process of ageing. These structures are also affected by genetic factors in patients with degenerative diseases (Ashburner et al, 2003). Besides providing anatomical information, medical imaging procedures are able to provide extremely relevant information about brain physiology, which can be used to understand physical and psychological clinical conditions. In order to capture the extraordinary morphological variability of the human brain, a number of automatic methods for identification and analysis of structures have been developed for different imaging modalities (Ashburner et al, 2003; Bandyopadhyay, 2011; Bresser et al, 2011; Ishii et al, 2009; Lopes et al, 2008; Ribbens et al, 2010; Rousset et al, 2007;

techniques have been proposed to extract the searched structures.

optimization are used to obtain minimum-cost solutions.

suited for processing some image features.

propagation lines up the objects of interest.

Shiee et al, 2010; Tu & Bai, 2010; Yi et al, 2009; Zhang et al, 2011).

methods, and uncertainty models, and includes:

boundaries.

knowledge, such as:

of training images.

applied as a set of rules.

**3.2.2 Brain applications** 

Most automatic systems for analysis of brain images have three processing steps: (i) brain tissue identification; (ii) registration: the voxels of interest are matched to a template or to an earlier scan from the same individual; (iii) statistical comparison of different groups of patients or same patient in different times.

In their review of the methods to assess brain structures, Ashburner et al (2003) discuss the strengths and limitations of algorithms for identification and registration of brain images, together with evidence of their usefulness at the clinical and research level. A more recent review of current methods used for computer automated identification in brain anatomical images is presented by Bandyopadhyay (2011). Most methods for extraction of brain structures and tissue segmentation use some kind of *a priori* information. Some authors, however, try to minimize the need of *a priori* knowledge. As an example, Scherrer et al (2009) extracted tissue and sub-cortical structures in MR images by considering a local approach to cope with intensity non-uniformity and a multi-agent based implementation.

#### **3.2.3 Cardiovascular applications**

Cardiovascular diseases are the leading cause of death in developed countries (Allender et al, 2008). Imaging techniques are nowadays among the most valuable tools in helping diagnosis, treatment and follow-up of cardiovascular pathologies, as they provide qualitative and quantitative information of the heart, which would be impossible to obtain otherwise (Cordero-Grande et al, 2011). Diagnosis and treatment follow-up of these pathologies can rely on numerous cardiac imaging modalities, which include echography, CT (computerized tomography), digital subtractive angiography, coronary angiography and cardiac MRI.

**Heart.** Quantitative analysis of the heart structure and function can be performed by the extraction of cardiac descriptors from medical images. These descriptors can provide information about global and local function. This includes estimating left ventricle volume, ejection fraction, cardiac output, myocardial mass, myocardial morphology, myocardial uptake rates, tissue viability, blood flow, local motion and deformation (wall thickening). The determination of most of these descriptors involves an appropriate 3D identification of cardiac structures – being the cardiac chambers, cardiac muscle or cardiac valves - as a prerequisite (Alattar et al, 2010; Cordero-Grande et al, 2011). The imaging modality of choice will depend on the problem under study and the complexity of the identification of the cardiac structures will depend on the modality chosen.

Automatic isolation of the heart structures in MR images is a challenging task because of the inherent noise associated with cine MRI, caused by factors such as patient movement, cardiac dynamics, and complex intensity characteristics (Lynch, 2008). Even in noiseless cases, MR images may present blurred edges due to the partial volume effect and can be affected by motion artifacts. In addition, low contrast between the myocardium and surrounding tissues complicates the identification of the cardiac structures. As an example, the papillary muscles, the trabeculae, or the liver have pixel intensity values which are virtually indistinguishable from those of the myocardium (Cordero-Grande et al, 2011). In the literature there is a huge number of recent works on heart identification in MR images (Alattar et al, 2010; Cordero-Grande et al, 2011; Lekadir et al, 2011; O´Brien et al, 2011; O'Donnell et al, 2006; Rouchdy et al, 2007; Schaerer et al, 2010; Zhao et al, 2009). A

Identification of Structures in Medical Images 11

(2009), Masuda et al (2010) and Pu et al (2009). Linguraru et al (2009) proposed a method to isolate both the spleen and liver in contrast-enhanced CT images by first aligning the images with models from an atlas and then improving the results by an active contour technique.

Pulmonary nodules are potential manifestations of lung cancer, and their detection and inspection are essential for screening and diagnosis of the disease (Kubota et al, 2011). Once the nodule has been detected, monitoring of its size, density, edge-smoothness and growth rate provide information about what treatment, if any, is appropriate (Murphy et al, 2009). An extensive review of computer aided diagnosis in chest radiography was published by van Ginneken et al (2001). Automated detection of nodules has been described by Murphy et al (2009) and Kubota et al (2011). Bouma et al (2009) proposed a system for the automatic detection of pulmonary embolism in contrast-enhanced CT images and Pu et al (2011)

**Mammography**. In mammography, the use of computer-aided diagnosis (CAD) systems is intended to assist radiologists in the automatic detection and classification of mammographic abnormalities. There is a large number of different types of mammographic abnormalities. In the majority of cases the abnormalities are either micro-calcifications or masses (Oliver et al, 2010). Mammography CAD systems usually start by a preprocessing step to extract the breast tissue, background tissue and pectoral muscle (Cheng et al, 2003; Samulski & Karssemeijer, 2010; Timp et al, 2007). Oliver et al (2011) published an extensive review on the theme in which they divide the automatic methods for detection of abnormalities in three groups: (i) Region-based methods: region growing and related methods, watershed methods, split and merge methods; (ii) Contour-based methods; (iii)

**Thyroid**. Diseases of the thyroid gland are among the most frequent endocrine disorders. Ultrasound has become the most important technique for thyroid gland imaging and computerized systems have been described to aid doctors in the task of thyroid image analysis. Hegedüs (2004) and Kollorz et al (2008) presented methods to automatically isolate the thyroid gland in US images, using geodesic active contour level set based approach.

**Kidney**. In renal applications Oguro et al (2011) described a system to aid CT-guided percutaneous cryoablation of renal tumors. Xie et al (2005) used *a priori* information about texture and size to segment kidney in ultrasound images. The described method provides the ability to deal with objects with incomplete boundaries by taking into account their texture.

**Meniscal tears**. The meniscus is an important part of the knee supporting mechanism. The knee joint can be severely damaged by a variety of causes, such as arthritis or knee injury. This can cause pain and inability to walk. In some cases, replacing parts of the joint is the appropriate course of action (Andra et al, 2008). Meniscal tears are a knee injury that is common in both young athletes and the aging population. It requires accurate diagnosis and, if necessary, surgical intervention. Ramakrishna et al (2009) described a novel CAD diagnostic system for automatic detection of meniscal tears in the knee in MR images. Andra et al (2008) described a system for pre-operative surgical planning based on structural

proposed an automated scheme to extract the airway tree depicted on CT images.

**3.2.5 Lung applications** 

**3.2.6 Other applications** 

Clustering methods.

comprehensive survey focusing on extracting the heart cavities in short axis view from cardiac MR images has been published recently by Petitjean & Dacher (2011).

The analysis of cardiac conditions using CT images needs a highly consistent identification of the involved surfaces. To measure blood volume, segmentation should consistently follow the cavity border. Wall thickness measurements should be based on a convex hull around the cavity that excludes papillary muscles from the myocardium. For the epicardium, it is important to avoid local confusions with the nearby pericardium or lung transition (Peters et al, 2010). A number of authors have described methods to isolate the cardiac walls and chambers in CT images (Ecabert et al, 2008; O'Donnell et al, 2006; Peters et al, 2010; Zheng et al, 2010).

Ultrasound imaging is arguably the hardest medical imaging modality upon which to perform organ identification (Noble, 2010). The main problem in detecting, for example, heart boundaries in ultrasound images is related to the high level of multiplicative noise (mainly speckle noise), low contrast among structures, artifacts such as shadowing from the lungs and, attenuation. All these factors can hinder the automatic analysis of the images (Antunes et al, 2010). A recent work by Mora et al (2009) combined supervised neural networks with different image processing techniques to identify heart cavities. Antunes et al (2010) applied a geometric deformable model to identify the four heart chambers of newborns.

**Vascular system.** Vascular identification is a particular challenging problem. The network trees of vessels are complex structures, with high variability in size and curvature. The general tubular organ geometry can be perturbed by stents, calcifications, aneurysms, and stenosis, besides being embedded in complex anatomical scenes, surrounded by other organs. Lesage et al (2009) published a review on vessel lumen segmentation techniques focused on 3D contrastenhanced imaging modalities (Magnetic Resonance Angiography - MRA and Computed Tomography Angiography - CTA). A former survey on vessel extraction techniques was presented by Kirbas and Kerk (2002). Destrempes et al (2009) describe a method for extracting carotid artery in ultrasound images. A segmentation and visualization of the left coronary heart arteries in CT images was presented by Rahman et al (2010). The arteries extraction methods tend to use a strong cylinder assumption. Qian et al (2009) presented a method in which they try to avoid this common strong prior knowledge. The analysis of temporal change in vessel images was addressed by Zhao et al (2009) who described a method to identify subjects with connective tissue disorder in 4D cardiovascular MRI.

#### **3.2.4 Liver applications**

Precise measurement of shape and composition of liver is the basis for diagnosis, surgery planning and therapy control of liver pathologies such as cirrhosis, liver cancer, and fulminant hepatic failure (Campadelli et al, 2009). CT images are preferred by the doctors for performing these tasks. Liver identification in CT scans poses problems due to the low contrast and blurred edges that characterize CT images. In their survey of liver extraction in CT scans, Campadelli et al (2009) identified the main applications of the technique as: (1) automatic detection of liver cancer from other liver diseases; (ii) measurement of liver volume - an important index in cases of living donor liver transplantation; (iii) 3D volume rendering of abdominal organs for surgical planning and radiation treatment programs. Another review of the methods for liver identification in CT images was published by Heimann et al (2009). Recent works in this area have also been described by Ruskó et al (2009), Masuda et al (2010) and Pu et al (2009). Linguraru et al (2009) proposed a method to isolate both the spleen and liver in contrast-enhanced CT images by first aligning the images with models from an atlas and then improving the results by an active contour technique.

#### **3.2.5 Lung applications**

10 Medical Imaging

comprehensive survey focusing on extracting the heart cavities in short axis view from

The analysis of cardiac conditions using CT images needs a highly consistent identification of the involved surfaces. To measure blood volume, segmentation should consistently follow the cavity border. Wall thickness measurements should be based on a convex hull around the cavity that excludes papillary muscles from the myocardium. For the epicardium, it is important to avoid local confusions with the nearby pericardium or lung transition (Peters et al, 2010). A number of authors have described methods to isolate the cardiac walls and chambers in CT images (Ecabert et al, 2008; O'Donnell et al, 2006; Peters et

Ultrasound imaging is arguably the hardest medical imaging modality upon which to perform organ identification (Noble, 2010). The main problem in detecting, for example, heart boundaries in ultrasound images is related to the high level of multiplicative noise (mainly speckle noise), low contrast among structures, artifacts such as shadowing from the lungs and, attenuation. All these factors can hinder the automatic analysis of the images (Antunes et al, 2010). A recent work by Mora et al (2009) combined supervised neural networks with different image processing techniques to identify heart cavities. Antunes et al (2010) applied a geometric

**Vascular system.** Vascular identification is a particular challenging problem. The network trees of vessels are complex structures, with high variability in size and curvature. The general tubular organ geometry can be perturbed by stents, calcifications, aneurysms, and stenosis, besides being embedded in complex anatomical scenes, surrounded by other organs. Lesage et al (2009) published a review on vessel lumen segmentation techniques focused on 3D contrastenhanced imaging modalities (Magnetic Resonance Angiography - MRA and Computed Tomography Angiography - CTA). A former survey on vessel extraction techniques was presented by Kirbas and Kerk (2002). Destrempes et al (2009) describe a method for extracting carotid artery in ultrasound images. A segmentation and visualization of the left coronary heart arteries in CT images was presented by Rahman et al (2010). The arteries extraction methods tend to use a strong cylinder assumption. Qian et al (2009) presented a method in which they try to avoid this common strong prior knowledge. The analysis of temporal change in vessel images was addressed by Zhao et al (2009) who described a method to identify

Precise measurement of shape and composition of liver is the basis for diagnosis, surgery planning and therapy control of liver pathologies such as cirrhosis, liver cancer, and fulminant hepatic failure (Campadelli et al, 2009). CT images are preferred by the doctors for performing these tasks. Liver identification in CT scans poses problems due to the low contrast and blurred edges that characterize CT images. In their survey of liver extraction in CT scans, Campadelli et al (2009) identified the main applications of the technique as: (1) automatic detection of liver cancer from other liver diseases; (ii) measurement of liver volume - an important index in cases of living donor liver transplantation; (iii) 3D volume rendering of abdominal organs for surgical planning and radiation treatment programs. Another review of the methods for liver identification in CT images was published by Heimann et al (2009). Recent works in this area have also been described by Ruskó et al

cardiac MR images has been published recently by Petitjean & Dacher (2011).

deformable model to identify the four heart chambers of newborns.

subjects with connective tissue disorder in 4D cardiovascular MRI.

al, 2010; Zheng et al, 2010).

**3.2.4 Liver applications** 

Pulmonary nodules are potential manifestations of lung cancer, and their detection and inspection are essential for screening and diagnosis of the disease (Kubota et al, 2011). Once the nodule has been detected, monitoring of its size, density, edge-smoothness and growth rate provide information about what treatment, if any, is appropriate (Murphy et al, 2009). An extensive review of computer aided diagnosis in chest radiography was published by van Ginneken et al (2001). Automated detection of nodules has been described by Murphy et al (2009) and Kubota et al (2011). Bouma et al (2009) proposed a system for the automatic detection of pulmonary embolism in contrast-enhanced CT images and Pu et al (2011) proposed an automated scheme to extract the airway tree depicted on CT images.

#### **3.2.6 Other applications**

**Mammography**. In mammography, the use of computer-aided diagnosis (CAD) systems is intended to assist radiologists in the automatic detection and classification of mammographic abnormalities. There is a large number of different types of mammographic abnormalities. In the majority of cases the abnormalities are either micro-calcifications or masses (Oliver et al, 2010). Mammography CAD systems usually start by a preprocessing step to extract the breast tissue, background tissue and pectoral muscle (Cheng et al, 2003; Samulski & Karssemeijer, 2010; Timp et al, 2007). Oliver et al (2011) published an extensive review on the theme in which they divide the automatic methods for detection of abnormalities in three groups: (i) Region-based methods: region growing and related methods, watershed methods, split and merge methods; (ii) Contour-based methods; (iii) Clustering methods.

**Thyroid**. Diseases of the thyroid gland are among the most frequent endocrine disorders. Ultrasound has become the most important technique for thyroid gland imaging and computerized systems have been described to aid doctors in the task of thyroid image analysis. Hegedüs (2004) and Kollorz et al (2008) presented methods to automatically isolate the thyroid gland in US images, using geodesic active contour level set based approach.

**Kidney**. In renal applications Oguro et al (2011) described a system to aid CT-guided percutaneous cryoablation of renal tumors. Xie et al (2005) used *a priori* information about texture and size to segment kidney in ultrasound images. The described method provides the ability to deal with objects with incomplete boundaries by taking into account their texture.

**Meniscal tears**. The meniscus is an important part of the knee supporting mechanism. The knee joint can be severely damaged by a variety of causes, such as arthritis or knee injury. This can cause pain and inability to walk. In some cases, replacing parts of the joint is the appropriate course of action (Andra et al, 2008). Meniscal tears are a knee injury that is common in both young athletes and the aging population. It requires accurate diagnosis and, if necessary, surgical intervention. Ramakrishna et al (2009) described a novel CAD diagnostic system for automatic detection of meniscal tears in the knee in MR images. Andra et al (2008) described a system for pre-operative surgical planning based on structural

Identification of Structures in Medical Images 13

**Linear scale-space**. The linear scale-space concept was introduced in the Western literature by Witkin (1983) and Koenderink (1984). Weickert (1997) showed that the concept had already been introduced in Japan approximately 20 years before. One of the main requirements of the scale-space formalism is that structures at coarser scales in the multiscale representation should constitute simplifications of corresponding structures at finer scales. This requirement has been stated in different forms by several authors (Babaud et al 1986; Koenderink, 1984; Lindeberg, 1994; Yuille & Poggio, 1987). Adding the requisites of linearity and invariance to

that leads to monotonic destruction of detail under consecutive blurring. A blurred version of

determines the scale of the output image in each image dimension. In an isotropic condition, assumes the same value for all image dimensions. The stack of images, as a function of

*Hx Ex gx*

(,) () (,)

2

In order to keep the property of scale invariance, the sampling of the scale parameter, , must be logarithmic (Keonderink, 1984). In the implementation of the Gaussian filter bank, a new parameter is usually introduced and it varies linearly throughout scales and relates to

**Multiscale representation - scale space primal sketch**. Primal sketch is an image structure first described by Marr (2010) based on the concept of gray level blobs (GLBlob), which are regions of the image that are lighter or darker than the surrounding background. Relating these regions to image structures results from the fact they have an intensity, or range of intensities, that may single them out among the other structures. Qualitatively, the concept of gray level blob for bi-dimensional images can be explained using the watershed operation from Mathematical Morphology (Lindeberg, 1994): the image function is seen as a flooded geographic area. As the water level decreases maxima points appear. At a certain point – or water level - two different peaks become connected. This point is called saddle. The difference between the image intensity at the maximum and at the saddle defines the contrast of each blob. The gray level of the saddle point is called blob base level. The support region of the blob is defined as the region containing the points with intensity values higher than the blob base

*g x e*

<sup>1</sup> (;)

2

 *<sup>n</sup>*. *<sup>n</sup> <sup>e</sup>* , where n is a natural number (*N*) consisting of zero and all positive integers, that represents the level in the scale space; is the parameter related to the resolution, or inner scale, of each level and determines the degree of spacing between the levels of the scale space. Linear scale space can be seen as a generic representation of image data that is common to different types of processing tasks. The main issues at this point are: how to describe properly the hierarchy throughout scales and how to extract important structures in each level of scale space and create a new data representation that can be suitable for high

is obtained by its convolution with the Gaussian filter, for a specific

*x*

 

2 <sup>2</sup> (2 )

) is the intensity of element after convolution. The width

(1)

) ) is the only filter

spatial displacement, those authors have shown that the Gaussian ( *g*(*x*;

 

increasing scale parameter , is known as linear scale-space.

**4.1.1 Methodology** 

the original image *E*(*x*)

level processing routines.

as:

(Equation 1). In Equation 1, *H*(*x*;

simulation of the bone-prosthesis system. The model of the bone was constructed based on information from CT datasets.

**Radiotherapy planning.** In the process of radiotherapy planning the most critical issue is the identification of the gross tumor volume and its relationship with surrounding tissues in order to decide the appropriate irradiation angles to minimize non-tumor tissue damage. Zaidi & El Naqa (2010) published a survey on methods for tumor identification in PET images.

#### **3.2.7 General methods**

Kohlberger et al (2009) described a method for general organ segmentation with level sets that incorporates local intensity statistics and local curvature by means of a point-based tracking mechanism. Campadelli et al (2010) described a method for extracting abdominal organs in CT images, using a gray-level based segmentation framework and an *a priori* anatomical knowledge that can be adapted to segment different abdominal organs. Noble (2010) made a review on segmentation techniques in ultrasound images as well as ultrasound tissue characterization for general applications. Crum et al (2004) reviewed non-rigid image registration, with emphasis in cardiac and brain applications. Van Rixoort et al (2010) described a framework for multi-atlas segmentation, in which the most appropriate atlases for a target image are automatically selected. The framework was applied to segment heart from non ECG gated volumetric chest CT scans and caudate nucleus from MR brain images.

## **4. Multiscale methods for identification, some research and examples**

The fact that real objects appear different depending on the scale of observation has important consequences to their description. The concept of scale is very important when analyzing image data by means of automatic methods (Koenderink, 1984). An object in a scene can be described in several forms, such as: its edges, its geometrical properties, its density or color, or its surface area. Ideally, each object has a set of features that set it aside from the other structures in the scene. Any method for the interpretation of image content must involve a phase for the detection of those features. Such detection is performed by the application of some operator to the image data. The application raises some questions about the type and size of the operators: the result of the operation is strongly dependent on these choices. The use of multiple scales is a way to overcome the problem of determining the size of the operator (Yuille & Poggio, 1986; Witkin, 1983). For optimal detection in a single scale, the size of the structure to be found should be known *a priori*. Several multiscale methods have been described in the literature for detection and classification of tissues and organs (Amira et al, 2008; Bernard & Friboulet, 2009; Gooya et al, 2008; Ugarriza et al, 2009; Yoon et al, 2008; Yu et al, 2008). In the following section we present an example of a representation for identification of organ structures that uses prior information in a multiscale technique.

#### **4.1 Example application**

In this example we propose a representation for the identification of medical structures by using a multiresolution approach, the scale-space. We evaluate the use of a data representation that allows the inclusion of *a priori* knowledge about the structures in several scales and we also develop the idea of an optimal scale to perform the processing.

#### **4.1.1 Methodology**

12 Medical Imaging

simulation of the bone-prosthesis system. The model of the bone was constructed based on

**Radiotherapy planning.** In the process of radiotherapy planning the most critical issue is the identification of the gross tumor volume and its relationship with surrounding tissues in order to decide the appropriate irradiation angles to minimize non-tumor tissue damage. Zaidi & El

Kohlberger et al (2009) described a method for general organ segmentation with level sets that incorporates local intensity statistics and local curvature by means of a point-based tracking mechanism. Campadelli et al (2010) described a method for extracting abdominal organs in CT images, using a gray-level based segmentation framework and an *a priori* anatomical knowledge that can be adapted to segment different abdominal organs. Noble (2010) made a review on segmentation techniques in ultrasound images as well as ultrasound tissue characterization for general applications. Crum et al (2004) reviewed non-rigid image registration, with emphasis in cardiac and brain applications. Van Rixoort et al (2010) described a framework for multi-atlas segmentation, in which the most appropriate atlases for a target image are automatically selected. The framework was applied to segment heart from

Naqa (2010) published a survey on methods for tumor identification in PET images.

non ECG gated volumetric chest CT scans and caudate nucleus from MR brain images.

**4. Multiscale methods for identification, some research and examples** 

The fact that real objects appear different depending on the scale of observation has important consequences to their description. The concept of scale is very important when analyzing image data by means of automatic methods (Koenderink, 1984). An object in a scene can be described in several forms, such as: its edges, its geometrical properties, its density or color, or its surface area. Ideally, each object has a set of features that set it aside from the other structures in the scene. Any method for the interpretation of image content must involve a phase for the detection of those features. Such detection is performed by the application of some operator to the image data. The application raises some questions about the type and size of the operators: the result of the operation is strongly dependent on these choices. The use of multiple scales is a way to overcome the problem of determining the size of the operator (Yuille & Poggio, 1986; Witkin, 1983). For optimal detection in a single scale, the size of the structure to be found should be known *a priori*. Several multiscale methods have been described in the literature for detection and classification of tissues and organs (Amira et al, 2008; Bernard & Friboulet, 2009; Gooya et al, 2008; Ugarriza et al, 2009; Yoon et al, 2008; Yu et al, 2008). In the following section we present an example of a representation for identification of organ structures that uses

In this example we propose a representation for the identification of medical structures by using a multiresolution approach, the scale-space. We evaluate the use of a data representation that allows the inclusion of *a priori* knowledge about the structures in several

scales and we also develop the idea of an optimal scale to perform the processing.

information from CT datasets.

**3.2.7 General methods** 

prior information in a multiscale technique.

**4.1 Example application** 

**Linear scale-space**. The linear scale-space concept was introduced in the Western literature by Witkin (1983) and Koenderink (1984). Weickert (1997) showed that the concept had already been introduced in Japan approximately 20 years before. One of the main requirements of the scale-space formalism is that structures at coarser scales in the multiscale representation should constitute simplifications of corresponding structures at finer scales. This requirement has been stated in different forms by several authors (Babaud et al 1986; Koenderink, 1984; Lindeberg, 1994; Yuille & Poggio, 1987). Adding the requisites of linearity and invariance to spatial displacement, those authors have shown that the Gaussian ( *g*(*x*; ) ) is the only filter that leads to monotonic destruction of detail under consecutive blurring. A blurred version of the original image *E*(*x*) is obtained by its convolution with the Gaussian filter, for a specific (Equation 1). In Equation 1, *H*(*x*;) is the intensity of element after convolution. The width determines the scale of the output image in each image dimension. In an isotropic condition, assumes the same value for all image dimensions. The stack of images, as a function of increasing scale parameter , is known as linear scale-space.

$$\begin{cases} H(\vec{x}, \sigma) = E(\vec{x}) \* g(\vec{x}, \sigma) \\\\ g(\vec{x}; \sigma) = \frac{1}{\sqrt{2\pi\sigma^2}} \int e^{-\left(\frac{\|\vec{x}\|^2}{\left(2\sigma^2\right)}\right)} \end{cases} \tag{1}$$

In order to keep the property of scale invariance, the sampling of the scale parameter, , must be logarithmic (Keonderink, 1984). In the implementation of the Gaussian filter bank, a new parameter is usually introduced and it varies linearly throughout scales and relates to as: *<sup>n</sup>*. *<sup>n</sup> <sup>e</sup>* , where n is a natural number (*N*) consisting of zero and all positive integers, that represents the level in the scale space; is the parameter related to the resolution, or inner scale, of each level and determines the degree of spacing between the levels of the scale space. Linear scale space can be seen as a generic representation of image data that is common to different types of processing tasks. The main issues at this point are: how to describe properly the hierarchy throughout scales and how to extract important structures in each level of scale space and create a new data representation that can be suitable for high level processing routines.

**Multiscale representation - scale space primal sketch**. Primal sketch is an image structure first described by Marr (2010) based on the concept of gray level blobs (GLBlob), which are regions of the image that are lighter or darker than the surrounding background. Relating these regions to image structures results from the fact they have an intensity, or range of intensities, that may single them out among the other structures. Qualitatively, the concept of gray level blob for bi-dimensional images can be explained using the watershed operation from Mathematical Morphology (Lindeberg, 1994): the image function is seen as a flooded geographic area. As the water level decreases maxima points appear. At a certain point – or water level - two different peaks become connected. This point is called saddle. The difference between the image intensity at the maximum and at the saddle defines the contrast of each blob. The gray level of the saddle point is called blob base level. The support region of the blob is defined as the region containing the points with intensity values higher than the blob base

Identification of Structures in Medical Images 15

life time

appearing scale

disappearing scale

original

level 1

level 3

level 2

level 5

level 4

3

1

2 3

<sup>4</sup> <sup>3</sup> <sup>5</sup>

<sup>4</sup> <sup>3</sup> <sup>5</sup>

<sup>4</sup> <sup>3</sup> <sup>5</sup>

**(c)**

1

2 3

merging

n n n n n n n

**(b)**

Fig. 3. (a) an image that comprises three regions with higher counts than their background. The lower image of (a) shows the image represented as isosurface; (b) SSPS built for the scale space shown in (c); (c) Six levels of the scale space of image (a) and their corresponding

As an example, the general significance measure can be applied to extract significant structures in thoracic magnetic resonance images, such as cardiac cavities, arteries and

Fig. 4. (left): slice of a T1 spin echo MRI thoracic sequence, the arrows point to the five target structures. (right) Dark blobs from 9 levels of the scale space of the image shown in (a). Values of are: 1 = 1.0 (original image), 2= 1.19; 3 = 1.41; 4 = 1.68; 5 = 2.00; 6 = 2.37; <sup>7</sup> = 2.82; 8 = 3.35; 9 = 3.97; 10 = 5.61. (Original image acquired at the Imaging Department -

Heart Institute – HCFMUSP- with patient anonymization).

merging destruction

4 5

**(a)**

veins, shown in Figure 4 left.

GLBlobs.

merging

level and can be reached from a local maximum without crossing points with intensity values lower than the blob base level. For 2D images, the gray level blob is defined as the tridimensional volume defined by the surface of the gray level and the base level.

A more formal definition of the blob concept is: the elliptical region associated to an extreme that is delimited by the above mentioned extreme and a saddle point. Differential geometry theory states that any image characteristic that has a geometrical meaning can be represented by a suitable differential operator. Florack et al (1992) presented an operator for the detection of elliptic regions – or blobs - in the images, the umbilicity operator. In 3D, it is given by Equation 2:

$$\text{LI} = \frac{\hat{\mathcal{O}}^2}{\hat{\mathcal{O}}\mathbf{x}^2} \cdot \frac{\hat{\mathcal{O}}^2}{\hat{\mathcal{O}}y^2} + \frac{\hat{\mathcal{O}}^2}{\hat{\mathcal{O}}\mathbf{x}^2} \cdot \frac{\hat{\mathcal{O}}^2}{\hat{\mathcal{O}}z^2} + \frac{\hat{\mathcal{O}}^2}{\hat{\mathcal{O}}y^2} \cdot \frac{\hat{\mathcal{O}}^2}{\hat{\mathcal{O}}z^2} - \left(\frac{\hat{\mathcal{O}}^2}{\hat{\mathcal{O}}\mathbf{x}\hat{\mathcal{O}}y}\right)^2 - \left(\frac{\hat{\mathcal{O}}^2}{\hat{\mathcal{O}}\mathbf{x}\hat{\mathcal{O}}z}\right)^2 - \left(\frac{\hat{\mathcal{O}}^2}{\hat{\mathcal{O}}y\hat{\mathcal{O}}z}\right)^2 \tag{2}$$

Regions with positive values of the umbilicity define the gray level blob regions. To discriminate light from dark blobs the laplacian operator is used (Equation 3):

$$L = \frac{\stackrel{\circ}{\partial}^2}{\stackrel{\circ}{\partial}x^2} + \frac{\stackrel{\circ}{\partial}^2}{\stackrel{\circ}{\partial}y^2} + \frac{\stackrel{\circ}{\partial}^2}{\stackrel{\circ}{\partial}z^2}, \qquad \begin{cases} \mathcal{U} > 0 \quad \text{and} \quad L > 0 \quad \Rightarrow \; \text{dark} \; \text{block} \\ \mathcal{U} > 0 \quad \text{and} \; \text{L} < 0 \quad \Rightarrow \; \text{light} \; \text{block} \end{cases} \tag{3}$$

Dark and light blobs are not only prominent in the original image, but also tend to maintain this prominence throughout the scales (Koenderink, 1984). That characteristic makes them a suitable choice for the elements of an image representation containing scale information. The Scale Space Primal Sketch (SSPS), first described by Lindeberg (1994), is an extension of the Primal Sketch and describes the characteristics of the blobs and their relations across scale space. Blobs in different scales are linked and form a new structure, the scale space blobs (SSBlob).

Figure 3 presents an example of a tree structure that maps blob relations in scale space. Figure 3.c depicts 5 levels of a scale space built for that image and their corresponding GLBlobs. Figure 3b presents the tree mapping of the relevant events occurring when following the blobs up in the scale space: creation, destruction and merging of blobs. These events are called bifurcations. A SSBlob is the object produced by the union of all GLBlobs that exist in the range between two bifurcations and is represented as a node in the tree. Each SSBlob is associated with a maximum scale (appearing scale) and a minimum scale (disappearing scale). The difference, given as number of scales, between the appearance and disappearance scales is called the life time. The information content of the tree nodes comprises the number of scales a structure survives, its appearance and disappearance scales, contrast with background, position at each survival scale, structure volume at each scale and total volume throughout scales. The linking between nodes reflects the behavior of the blobs during the smoothing process: creation, destruction and merging. This tree model is called Scale Space Primal Sketch.

Stable blobs in the SSPS must represent significant structures in the image (Koenderink, 1984, Lindeberg, 1994). Based on the stability of the SSBlobs significance measurement rates can be defined, allowing the classification of structures. The first significance measure proposed by Lindeberg (1994) was solely based on intrinsic characteristics of the image: blob volume and blob life time. The assumptions for this measurement were general and assumed there was not any *a priori* knowledge about the structures present in the image.

level and can be reached from a local maximum without crossing points with intensity values lower than the blob base level. For 2D images, the gray level blob is defined as the

A more formal definition of the blob concept is: the elliptical region associated to an extreme that is delimited by the above mentioned extreme and a saddle point. Differential geometry theory states that any image characteristic that has a geometrical meaning can be represented by a suitable differential operator. Florack et al (1992) presented an operator for the detection of elliptic regions – or blobs - in the images, the umbilicity operator. In 3D, it is

> *xy xz yz x y x z y z* 22 22 22 2 2 2

*xy z U and L light blob*

0 0 , 0 0

(3)

Regions with positive values of the umbilicity define the gray level blob regions. To

Dark and light blobs are not only prominent in the original image, but also tend to maintain this prominence throughout the scales (Koenderink, 1984). That characteristic makes them a suitable choice for the elements of an image representation containing scale information. The Scale Space Primal Sketch (SSPS), first described by Lindeberg (1994), is an extension of the Primal Sketch and describes the characteristics of the blobs and their relations across scale space. Blobs in different scales are linked and form a new structure,

Figure 3 presents an example of a tree structure that maps blob relations in scale space. Figure 3.c depicts 5 levels of a scale space built for that image and their corresponding GLBlobs. Figure 3b presents the tree mapping of the relevant events occurring when following the blobs up in the scale space: creation, destruction and merging of blobs. These events are called bifurcations. A SSBlob is the object produced by the union of all GLBlobs that exist in the range between two bifurcations and is represented as a node in the tree. Each SSBlob is associated with a maximum scale (appearing scale) and a minimum scale (disappearing scale). The difference, given as number of scales, between the appearance and disappearance scales is called the life time. The information content of the tree nodes comprises the number of scales a structure survives, its appearance and disappearance scales, contrast with background, position at each survival scale, structure volume at each scale and total volume throughout scales. The linking between nodes reflects the behavior of the blobs during the smoothing process: creation, destruction and merging. This tree model

Stable blobs in the SSPS must represent significant structures in the image (Koenderink, 1984, Lindeberg, 1994). Based on the stability of the SSBlobs significance measurement rates can be defined, allowing the classification of structures. The first significance measure proposed by Lindeberg (1994) was solely based on intrinsic characteristics of the image: blob volume and blob life time. The assumptions for this measurement were general and assumed there was not any *a priori* knowledge about the structures present in the image.

 

222

*U and L dark blob*

(2)

tridimensional volume defined by the surface of the gray level and the base level.

discriminate light from dark blobs the laplacian operator is used (Equation 3):

2 2 22 22

22 2 222

given by Equation 2:

*L*

the scale space blobs (SSBlob).

is called Scale Space Primal Sketch.

*U*

Fig. 3. (a) an image that comprises three regions with higher counts than their background. The lower image of (a) shows the image represented as isosurface; (b) SSPS built for the scale space shown in (c); (c) Six levels of the scale space of image (a) and their corresponding GLBlobs.

As an example, the general significance measure can be applied to extract significant structures in thoracic magnetic resonance images, such as cardiac cavities, arteries and veins, shown in Figure 4 left.

Fig. 4. (left): slice of a T1 spin echo MRI thoracic sequence, the arrows point to the five target structures. (right) Dark blobs from 9 levels of the scale space of the image shown in (a). Values of are: 1 = 1.0 (original image), 2= 1.19; 3 = 1.41; 4 = 1.68; 5 = 2.00; 6 = 2.37; <sup>7</sup> = 2.82; 8 = 3.35; 9 = 3.97; 10 = 5.61. (Original image acquired at the Imaging Department - Heart Institute – HCFMUSP- with patient anonymization).

Identification of Structures in Medical Images 17

The concept of suitable scale for processing is based on the idea that the smoothness degree for which an object vanishes in the scale space is related to the object size. This issue links directly to the subject of scale selection, a task that is not trivial and that has been studied by some authors (Lindeberg, 1994; Majer, 2000; Pauly et al, 2003). These authors articles considered the case when image analysis is performed without any *a priori* knowledge about the structures present in the scene. In the present work the suitable scale is determined by constructing the SSPS representation for the pattern images. The analysis of the survival scales for the structure under study and all other structures representing real entities in the image allows the determination of the best scale for the geometrical matching. The following steps are performed: (1) inspect all scales at which the target structure can be detected as a single blob; (2) the levels of the scale space where all blobs representing real structures in the image are present are considered informative and (3) the upper informative scale is

The reasons behind that choice are: (1) the occurrence of spurious structures in lower scales is higher. This fact would highly increase the number of useless calculation of distances and angles; (2) although structures are smoother in higher scales, the determination of angles and distances is based on the geometrical centers of each structure. These points should not change their position under the smoothness process. There is a problem with this strategy when there is not a single level in which all blobs representing real structures exist as a unique structure. This is the case when a small and low contrast structure is present and its corresponding blobs appear only in low scale levels, when bigger objects do not constitute a single blob anymore due to noise. In such cases, the suitable scale is chosen as the higher

The identification of structures modeled in the prototype comprises the following steps: (1) creation of the scale space by using the linear Gaussian filter; (2) detection of GLBlobs through the application of the umbilicity operator; (3) creation of the Scale Space Primal Sketch; (4) comparison of the SSBlob volumes from all nodes with the pattern. Elimination of nodes with values outside a defined range; (5) determination of the distance and angles for the remaining SSBlob in the suitable scale; (6) contextual matching between each candidate node and the features' pattern: support region, angles and distances and (7) the

**Similarity criteria.** In the simplification of the SSBlob tree processing phase, the candidate nodes are defined as those having SSBlob volume values lying in a range defined as a function of the pattern. That range was initially set to 20% around the pattern. The suitability of this choice are analyzed in the experiments. The similarity criterion for contextual matching consists in minimizing the Euclidean distance between each of the

**Pattern creation**. The pattern is obtained by applying the proposed method to a set of data. The following parameters of the blob representing the target structure are stored in the pattern: SSBlob volume, the suitable scale, the support region of the GLBlob in this scale, all distances and angles between the target structure and the others present in this scale, the spatial resolution of the original image, the value of used for the construction

considered the most suitable for the contextual matching.

scale where the blob representing the target structure appears.

node with the best matching is identified as the target structure.

feature values obtained for each node of the test image and the pattern.

**4.1.3 Schema of the prototype application** 

of the scale space.

The thin arrows in Figure 4 right point to the 15 most significant dark blobs extracted using the general significance rate and the scale where the blob is more distinguished. Figure 4 right shows that almost all target structures are present amongst the set of significant blobs (exception is the atrium). Other anatomic structures are also present in the set, like the lungs (two lower arrows in scale 8). The general approach is not able, therefore, to set aside a particular structure from others present in the image. Furthermore, there are some structures that do not have any anatomic meaning, such as disconnected parts of organs in lower scales (lower arrows at scales 5 and 6) and some spurious structures originated from noise in upper scales, identified by the thicker arrows (Rebelo, 2000). In order to overcome the problems found with the application of the general approach, an important feature of medical images can be embedded in the image representation: the knowledge about the general characteristics and expected location of the target structure.

**Including** *a priori* **information on SSPS**. The SSPS is used as an image structure that serves as a guide to subsequent high level processing step where *a priori* knowledge about the desired feature is modeled and included in the representation. The image structure is first used to build patterns of known objects present in the image. A particular object can then be identified in new images by matching with symbolic features described in the pattern. Any kind of information can be embedded in the SSPS structure. The question that arises immediately regards defining the kind of information that is useful to include in the representation. The answer to this question is highly dependent on the problem domain, including the modality of the image under analysis, the kind of structure to be identified and the representation chosen for the biological structures. To validate the proposed method we have built a prototype application, which is described in detail in the next section.

#### **4.1.2 Prototype application**

In a previous work we proposed a prototype, in which geometrical information of the target structure and its relation with other structures present on the scene were included in SSPS (Rebelo, 2007). The set of parameters chosen for building the pattern was: (i) SSBlob volume, or the sum of the support regions of all GLBlobs; (ii) support region of the GLBlob; (iii) contextual information in the form of relative position of the target structure and the others present in the image, specifically distances and angles. For an image containing N GLBlobs at a given scale level the calculation of the set of parameters has a complexity of O(N3). In addition, the calculation has to be performed for the images in all levels of the scale space. A score is determined for each structure present in the image based on the closeness of its feature values to the pattern.

The matching procedure presents, therefore, a problem related to the huge amount of GLBlobs at the lower levels of scale. This can lead to an explosion in the number of parameters to be processed, especially the number of angles, greatly increasing processing time to detect the node with the highest score. In order to decrease the possible combinations for matching, we have developed the following strategy: (1) decrease the number of nodes for the contextual matching phase by performing a selection of the possible nodes, using a global parameter. The prototype uses SSBlob volume, since the previous experiments with thoracic images showed its potential for recognition of relevant structures in medical images. A set of candidate nodes is created by eliminating the nodes with SSBlob volume too distant from the pattern; (2) choosing a suitable scale in the scale space for the contextual matching.

The thin arrows in Figure 4 right point to the 15 most significant dark blobs extracted using the general significance rate and the scale where the blob is more distinguished. Figure 4 right shows that almost all target structures are present amongst the set of significant blobs (exception is the atrium). Other anatomic structures are also present in the set, like the lungs (two lower arrows in scale 8). The general approach is not able, therefore, to set aside a particular structure from others present in the image. Furthermore, there are some structures that do not have any anatomic meaning, such as disconnected parts of organs in lower scales (lower arrows at scales 5 and 6) and some spurious structures originated from noise in upper scales, identified by the thicker arrows (Rebelo, 2000). In order to overcome the problems found with the application of the general approach, an important feature of medical images can be embedded in the image representation: the knowledge about the

**Including** *a priori* **information on SSPS**. The SSPS is used as an image structure that serves as a guide to subsequent high level processing step where *a priori* knowledge about the desired feature is modeled and included in the representation. The image structure is first used to build patterns of known objects present in the image. A particular object can then be identified in new images by matching with symbolic features described in the pattern. Any kind of information can be embedded in the SSPS structure. The question that arises immediately regards defining the kind of information that is useful to include in the representation. The answer to this question is highly dependent on the problem domain, including the modality of the image under analysis, the kind of structure to be identified and the representation chosen for the biological structures. To validate the proposed method we have built a prototype application, which is described in detail in the next section.

In a previous work we proposed a prototype, in which geometrical information of the target structure and its relation with other structures present on the scene were included in SSPS (Rebelo, 2007). The set of parameters chosen for building the pattern was: (i) SSBlob volume, or the sum of the support regions of all GLBlobs; (ii) support region of the GLBlob; (iii) contextual information in the form of relative position of the target structure and the others present in the image, specifically distances and angles. For an image containing N GLBlobs at a given scale level the calculation of the set of parameters has a complexity of O(N3). In addition, the calculation has to be performed for the images in all levels of the scale space. A score is determined for each structure present in the image based on the closeness of its

The matching procedure presents, therefore, a problem related to the huge amount of GLBlobs at the lower levels of scale. This can lead to an explosion in the number of parameters to be processed, especially the number of angles, greatly increasing processing time to detect the node with the highest score. In order to decrease the possible combinations for matching, we have developed the following strategy: (1) decrease the number of nodes for the contextual matching phase by performing a selection of the possible nodes, using a global parameter. The prototype uses SSBlob volume, since the previous experiments with thoracic images showed its potential for recognition of relevant structures in medical images. A set of candidate nodes is created by eliminating the nodes with SSBlob volume too distant from the pattern; (2) choosing a suitable scale in

general characteristics and expected location of the target structure.

**4.1.2 Prototype application** 

feature values to the pattern.

the scale space for the contextual matching.

The concept of suitable scale for processing is based on the idea that the smoothness degree for which an object vanishes in the scale space is related to the object size. This issue links directly to the subject of scale selection, a task that is not trivial and that has been studied by some authors (Lindeberg, 1994; Majer, 2000; Pauly et al, 2003). These authors articles considered the case when image analysis is performed without any *a priori* knowledge about the structures present in the scene. In the present work the suitable scale is determined by constructing the SSPS representation for the pattern images. The analysis of the survival scales for the structure under study and all other structures representing real entities in the image allows the determination of the best scale for the geometrical matching. The following steps are performed: (1) inspect all scales at which the target structure can be detected as a single blob; (2) the levels of the scale space where all blobs representing real structures in the image are present are considered informative and (3) the upper informative scale is considered the most suitable for the contextual matching.

The reasons behind that choice are: (1) the occurrence of spurious structures in lower scales is higher. This fact would highly increase the number of useless calculation of distances and angles; (2) although structures are smoother in higher scales, the determination of angles and distances is based on the geometrical centers of each structure. These points should not change their position under the smoothness process. There is a problem with this strategy when there is not a single level in which all blobs representing real structures exist as a unique structure. This is the case when a small and low contrast structure is present and its corresponding blobs appear only in low scale levels, when bigger objects do not constitute a single blob anymore due to noise. In such cases, the suitable scale is chosen as the higher scale where the blob representing the target structure appears.

## **4.1.3 Schema of the prototype application**

The identification of structures modeled in the prototype comprises the following steps: (1) creation of the scale space by using the linear Gaussian filter; (2) detection of GLBlobs through the application of the umbilicity operator; (3) creation of the Scale Space Primal Sketch; (4) comparison of the SSBlob volumes from all nodes with the pattern. Elimination of nodes with values outside a defined range; (5) determination of the distance and angles for the remaining SSBlob in the suitable scale; (6) contextual matching between each candidate node and the features' pattern: support region, angles and distances and (7) the node with the best matching is identified as the target structure.

**Similarity criteria.** In the simplification of the SSBlob tree processing phase, the candidate nodes are defined as those having SSBlob volume values lying in a range defined as a function of the pattern. That range was initially set to 20% around the pattern. The suitability of this choice are analyzed in the experiments. The similarity criterion for contextual matching consists in minimizing the Euclidean distance between each of the feature values obtained for each node of the test image and the pattern.

**Pattern creation**. The pattern is obtained by applying the proposed method to a set of data. The following parameters of the blob representing the target structure are stored in the pattern: SSBlob volume, the suitable scale, the support region of the GLBlob in this scale, all distances and angles between the target structure and the others present in this scale, the spatial resolution of the original image, the value of used for the construction of the scale space.

Identification of Structures in Medical Images 19

After the construction of the scale space, using = 0.19, the GLBlobs were detected in 12 levels of scale. Figure 6 depicts the GLBlobs detected for the images of Set 1 and Set 2, presented in Figure 5. In Figure 6 it is possible to observe that the number of blobs in lower scales increases for higher noise images. This effect tends to disapear for upper scale levels. The application of the proposed method led to the successful identification of the chosen

structures in most experiments, exception made to two with the highest noise level.

Fig. 6. Set of GLBlobs determined in 12 levels of scale for two groups of images shown in Figure 5. The set from Set 1 (original without noise, low level noise, mid level noise and high

**Behavior for different contrast levels.** The contrast between a structure and its background is one of the factors affecting the survival of the structure in the scale space, or, in other words, it affects the shape of the SSPS. In order to test the application of the method at several levels of contrast a simpler version of image 3 from Set 1, shown in Figure 5 (first line), was used. From the original image, two sets of images were created to assess how contrast influences SSPS shape. The first set was created by changing the counts of the

level noise) is shown on the left and the set from Set 2 is shown on the right.

#### **4.1.4 Experiments**

The Mathematical Cardiac Torso (MCAT) phantom is an anthropomorphic phantom, developed at the University of North Carolina that models size, shape and configurations of the major thoracic structures and organs by using of mathematical formulae (Tsui, 1993). A set of experiments was performed with two-dimensional images obtained from manipulations - adding and mirroring - of MCAT images. The manipulation aimed at creating frames with more organs, to make the matching step more complex. The goal was to analyze the ability of the method to identify organs at different levels of noise and contrast. The results for each case are described in the following sections.

**Behavior for different noise levels**. Five images were generated from the MCAT frames without noise. Three different levels of additive Gaussian noise with zero mean were then added to each image. For computing the noise level, we used the signal-to-noise ratio (SNR) described by Rangayan (2005).

$$\text{SNR} = 20^{\text{s}} \log\_{10} \left( \frac{\sigma\_{signal}}{\sigma\_{noise}} \right) \text{ dB} \tag{4}$$

In equation 4, *signal* is the standard deviation of the intensity values of the original image (not corrupted with noise); *noise* is the standard deviation of the intensity values of the noise image added to the original to generate the final corrupted image. The signal-to-noise ratio of the three different levels were 8dB (high noise level), 19 dB (medium level) and 33 dB (low level).

The five groups of images were classified into two distinct sets, according to their geometry. Figure 5 shows two images representatives of these sets: an image from Set 1 in first line and Set 2 in second line. In each set, one structure was chosen to be identified: structure 4 in Set1 and structure 5 in Set 2. The pattern was built based on the original images without noise. In Figure 5 the original images (not corrupted by noise) are shown in column A. The noisy images are presented in column B (low level of noise), C (medium level of noise) and D (high level of noise).

Fig. 5. Two images representative of the two Sets (image 3 from Set 1 and image 4 from Set 2). Column (A) shows the original images (without noise), column (B) the low noise images, column (C) medium noise images and column (D) high noise images.

The Mathematical Cardiac Torso (MCAT) phantom is an anthropomorphic phantom, developed at the University of North Carolina that models size, shape and configurations of the major thoracic structures and organs by using of mathematical formulae (Tsui, 1993). A set of experiments was performed with two-dimensional images obtained from manipulations - adding and mirroring - of MCAT images. The manipulation aimed at creating frames with more organs, to make the matching step more complex. The goal was to analyze the ability of the method to identify organs at different levels of noise and

**Behavior for different noise levels**. Five images were generated from the MCAT frames without noise. Three different levels of additive Gaussian noise with zero mean were then added to each image. For computing the noise level, we used the signal-to-noise ratio (SNR)

*SNR* 20\* log10 *dB*

noise image added to the original to generate the final corrupted image. The signal-to-noise ratio of the three different levels were 8dB (high noise level), 19 dB (medium level) and 33

The five groups of images were classified into two distinct sets, according to their geometry. Figure 5 shows two images representatives of these sets: an image from Set 1 in first line and Set 2 in second line. In each set, one structure was chosen to be identified: structure 4 in Set1 and structure 5 in Set 2. The pattern was built based on the original images without noise. In Figure 5 the original images (not corrupted by noise) are shown in column A. The noisy images are presented in column B (low level of noise), C (medium level of noise) and D

Fig. 5. Two images representative of the two Sets (image 3 from Set 1 and image 4 from Set 2). Column (A) shows the original images (without noise), column (B) the low noise images,

column (C) medium noise images and column (D) high noise images.

*signal noise*

(4)

*signal* is the standard deviation of the intensity values of the original image

*noise* is the standard deviation of the intensity values of the

contrast. The results for each case are described in the following sections.

**4.1.4 Experiments** 

described by Rangayan (2005).

(not corrupted with noise);

In equation 4,

dB (low level).

(high level of noise).

After the construction of the scale space, using = 0.19, the GLBlobs were detected in 12 levels of scale. Figure 6 depicts the GLBlobs detected for the images of Set 1 and Set 2, presented in Figure 5. In Figure 6 it is possible to observe that the number of blobs in lower scales increases for higher noise images. This effect tends to disapear for upper scale levels.

The application of the proposed method led to the successful identification of the chosen structures in most experiments, exception made to two with the highest noise level.

Fig. 6. Set of GLBlobs determined in 12 levels of scale for two groups of images shown in Figure 5. The set from Set 1 (original without noise, low level noise, mid level noise and high level noise) is shown on the left and the set from Set 2 is shown on the right.

**Behavior for different contrast levels.** The contrast between a structure and its background is one of the factors affecting the survival of the structure in the scale space, or, in other words, it affects the shape of the SSPS. In order to test the application of the method at several levels of contrast a simpler version of image 3 from Set 1, shown in Figure 5 (first line), was used. From the original image, two sets of images were created to assess how contrast influences SSPS shape. The first set was created by changing the counts of the

Identification of Structures in Medical Images 21

data should be carried out in order to assess the full clinical usefulness of the method. The presented method is general and can be classified as a third generation method, according to the classification presented in the review, as it is a second generation multiresolution representation capable of incorporating knowledge to be used in a specific application. Although the representation is generic, the information chosen for definition of the pattern will depend on particular applications, and can be easily inserted in the data structure.

The review presented in this chapter was intended to give the reader a general scenario of medical applications. We focused on clinical applications and the issues for identification of particular organs and not on detailed application and modality-dependent aspects. Methods and applications that have appeared in the recent literature were briefly described. The main goal was to to give the readers a glimpse of this broad and extremely active research field.

In spite of the huge number of existing methods, the problem of identification in medical images is still challenging, with no general and unique solution and remains an open and promising area in the field of image processing. The growing research in the clinical areas makes the number of objects of interest in medical images to grow constantly, which generates a continuous demand for automatic solutions to aid clinicians. On the other side, fast advances in radiological imaging systems result in increasingly higher volume (3D) images to be processed. Processing of these images in radiological diagnostic systems requires accurate and fast algorithms. Intense use of apparel techniques must be studied, and the use of new computing platforms, such as computational grids, must be explored.

The goal of automated systems is not to replace the experts, whose experience and knowledge are essential in any automatic systems for identification or analysis. Instead, the development of automated systems to assist the clinicians is intended to reduce their workload in the step of analyzing patient data, saving them time to other valuable tasks as

This work is supported by National Council for Scientific and Technological Development, The National Institute for Science and Technology - Medicine Assisted by Scientific

Alattar, M. A., Osman, N.F. & Fahmy, A. S. (2010). Myocardial Segmentation Using

Allender, S., Scarborough, P., Peto, V., Rayner, M., Leal, J., Luengo-Fernandez, R. & Gray, A.

Amira, A., Chandrasekarana, S., Montgomery, D.W. & Uzunb, I.S. (2008) A segmentation

Andra, H., Battiato, S., Bilotta, G., Farinella, G.M., Impoco, G., Orlik, J., Russo, G. & Zemitis,

*Image Analysis and Recognition*, vol. 6112/2010, pp. 89-98.

*Neurocomputing* , vol.71 (2008), pp. 1954– 1965.

*Sensors*, vol 8 (2008), pp. 5897-5926.

Constrained Multi-Seeded Region Growing. *Lecture Notes in Computer Science -* 

(2008). *European cardiovascular disease statistics: 2008 edition*, British Heart

concept for positron emission tomography imaging using multiresolution analysis.

A. (2008). Structural Simulation of a Bone-Prosthesis System of the Knee Joint.

strategies and decision making.

Computing (INCT-MAAC) and Zerbini Foundation.

Foundation, London.

**6. Acknowledments** 

**7. References** 

thoracic region. The second set was created by changing both the counts of the thoracic region and the counts of the background, in such a way that the contrast between thorax and background remained constant. The contrast C(s,b) between a target structure S and its local background B, was calculated by Equation 5:

$$\mathbf{C} = \frac{\mu\_{\rm S} - \mu\_{\rm B}}{\mu\_{\rm B}} \ . \tag{5}$$

µS is the mean value of intensity inside the structure; µB is the mean value of intensity in its background B. For each contrast manipulated image a set of noisy images with three different noise levels was created. For this experiment, noise was quantified by the contrast to noise ratio, defined in relation to the target structure S by Equation 6 (Rangayan, 2005):

$$\text{CNNR} = \frac{|\mu\_S - \mu\_B|}{\sigma\_B} \tag{6}$$

where B is the standard deviation of intensity in the background.

The structure number 2 (Figure 5 upper A) was chosen as the target. An interesting observation was that when the contrast between thorax and background remained at the same value, the blobs of the images with low contrast between the internal structures and the thorax had a behavior not very different from the one observed in the images with high contrast. However, all internal structures survived in a lower number of levels as the contrast decreased.

The results of identification were satisfactory. In 87% of the experiments the target structure was identified correctly. The pattern was built based on the original image with the lowest contrast between the internal structures and the thorax.

## **5. Conclusion**

In this chapter we discussed the challenges, methods and clinical examples of medical image identification. We pointed out the importance of this task as a previous step for medical image applications such as visualization and the quantitative analysis of organs and systems. A classification for the methods described in the literature and a brief review of their clinical applications were presented.

It was shown that identification of structures in medical images is a very difficult task due to the complexity of the anatomical structures and their changing with different physiological states. A simplification of the task can be achieved by dividing it in two phases: first, identification of the desired structures that includes a rough segmentation; then a postprocessing phase for segmentation refinement.

In this text we presented a method for the first phase that uses an image representation based on a scale space approach that embeds *a priori* knowledge in it. One of the main goals of this proposal was to develop a powerful representation for images that would be suitable for application of high level processing routines, such as the matching used in the prototype implemented. A study with MCAT phantom images was performed in order to assess the method under controlled conditions reproducing anatomical structures. The results we have obtained were quite encouraging. However, further intensive investigation with real patient

thoracic region. The second set was created by changing both the counts of the thoracic region and the counts of the background, in such a way that the contrast between thorax and background remained constant. The contrast C(s,b) between a target structure S and its

> *S B B*

µS is the mean value of intensity inside the structure; µB is the mean value of intensity in its background B. For each contrast manipulated image a set of noisy images with three different noise levels was created. For this experiment, noise was quantified by the contrast to noise ratio, defined in relation to the target structure S by Equation 6 (Rangayan, 2005):

*S B B*

The structure number 2 (Figure 5 upper A) was chosen as the target. An interesting observation was that when the contrast between thorax and background remained at the same value, the blobs of the images with low contrast between the internal structures and the thorax had a behavior not very different from the one observed in the images with high contrast. However, all internal structures survived in a lower number of levels as the

The results of identification were satisfactory. In 87% of the experiments the target structure was identified correctly. The pattern was built based on the original image with the lowest

In this chapter we discussed the challenges, methods and clinical examples of medical image identification. We pointed out the importance of this task as a previous step for medical image applications such as visualization and the quantitative analysis of organs and systems. A classification for the methods described in the literature and a brief review of

It was shown that identification of structures in medical images is a very difficult task due to the complexity of the anatomical structures and their changing with different physiological states. A simplification of the task can be achieved by dividing it in two phases: first, identification of the desired structures that includes a rough segmentation; then a post-

In this text we presented a method for the first phase that uses an image representation based on a scale space approach that embeds *a priori* knowledge in it. One of the main goals of this proposal was to develop a powerful representation for images that would be suitable for application of high level processing routines, such as the matching used in the prototype implemented. A study with MCAT phantom images was performed in order to assess the method under controlled conditions reproducing anatomical structures. The results we have obtained were quite encouraging. However, further intensive investigation with real patient

. (5)

(6)

*<sup>C</sup>* 

*CNR* 

where B is the standard deviation of intensity in the background.

contrast between the internal structures and the thorax.

their clinical applications were presented.

processing phase for segmentation refinement.

local background B, was calculated by Equation 5:

contrast decreased.

**5. Conclusion** 

data should be carried out in order to assess the full clinical usefulness of the method. The presented method is general and can be classified as a third generation method, according to the classification presented in the review, as it is a second generation multiresolution representation capable of incorporating knowledge to be used in a specific application. Although the representation is generic, the information chosen for definition of the pattern will depend on particular applications, and can be easily inserted in the data structure.

The review presented in this chapter was intended to give the reader a general scenario of medical applications. We focused on clinical applications and the issues for identification of particular organs and not on detailed application and modality-dependent aspects. Methods and applications that have appeared in the recent literature were briefly described. The main goal was to to give the readers a glimpse of this broad and extremely active research field.

In spite of the huge number of existing methods, the problem of identification in medical images is still challenging, with no general and unique solution and remains an open and promising area in the field of image processing. The growing research in the clinical areas makes the number of objects of interest in medical images to grow constantly, which generates a continuous demand for automatic solutions to aid clinicians. On the other side, fast advances in radiological imaging systems result in increasingly higher volume (3D) images to be processed. Processing of these images in radiological diagnostic systems requires accurate and fast algorithms. Intense use of apparel techniques must be studied, and the use of new computing platforms, such as computational grids, must be explored.

The goal of automated systems is not to replace the experts, whose experience and knowledge are essential in any automatic systems for identification or analysis. Instead, the development of automated systems to assist the clinicians is intended to reduce their workload in the step of analyzing patient data, saving them time to other valuable tasks as strategies and decision making.

#### **6. Acknowledments**

This work is supported by National Council for Scientific and Technological Development, The National Institute for Science and Technology - Medicine Assisted by Scientific Computing (INCT-MAAC) and Zerbini Foundation.

## **7. References**


Identification of Structures in Medical Images 23

Gooya, A., Liao, H., Matsumiya, K., Masamune, K., Masutani, Y. & Dohi, T. (2008). A

Medical Images. *IEEE Trans on Imag Proc*, vol. 17, no 8 (2008), pp. 1295-1312. Hegedüs, L. (2004) Thyroid ultrasound as a screening tool for thyroid disease. *Thyroid*, vol.

Heimann, T., van Ginneken, B., Styner, M.A. et al (2009). Comparison and Evaluation of

Ishii, K., Kanda, T., Uemura, T., Miyamoto, N., Yoshikawa, T., Shimada, K., Ohkawa, S. &

Jain, A.K. (1989). *Fundamentals of Digital Image Processing*, Prentice Hall Englewood Cliffs. Kirbas, C. & Quek, K. (2002). A Review of Vessel Extraction Techniques and Algorithms,

Koenderink, J.J. (1984). The structure of images. *Biological Cybernetics*, vol. 50 (1984), pp. 363-

Kohlberger, T., Uzunba, M.G, Alvino, C. & Kadir, T. (2009) Organ Segmentation with Level

Kollorz, E.N.K., Hahn, D.A., Linke, R., Goecke, T.W., Hornegger, J. & Kuwert, T. (2008)

Kubota, T., Jerebko, A.K., Dewan, M., Salganicoff, M. & Krishnan, A. (2011) Segmentation of

Assessment in MRI. *IEEE Trans on Med Imag*, vol 30, no 1 (2011), pp. 52-68. Lesage, D., Angelini, E.D., Bloch, I. & Funka-Lea, G. (2009) A review of 3D vessel lumen

Lindeberg, T. (1994). Scale-space theory: a basic tool for analyzing structures at different

Linguraru, M.G., Sandberg, J.K., Li, Z., Pura, J.A. & Summers, R.M. (2009) Atlas-Based

Estimation. *MICCAI 2009, Part II, LNCS*, vol. 5762/2009, pp. 1001–1008. Lopes, R., Dubois, P., Makni, N., Szurhaj, W., Maouche, S. & Betrouni, N. (2008)

epilepsy detection. *International Journal of CARS 2008*, vol 3, pp. 341–346. Lynch, M., Ghita, O. & Whelan, P.F. (2008) Segmentation of the Left Ventricle of the Heart in

Ma, Z., Tavares, J.M.R.S. & Jorge, R.N. (2008) Segmentation of Structures in Medical Images:

Majer, P.A. (2000) *Statistical Approach to Feature Detection and Scale Selection in Images*, PhD

scales, *Journal of Applied Statistics*, vol 21, (1994), p. 225-270.

convexity models. *Medical Image Analysis*, vol. 15 (2011), pp. 133–154. Lekadir, K., Keenan, N.G., Pennell, D.J. & Yang, G-Z. (2011) An Inter-Landmark Approach

14, no 11 (2004), pp. 879–880.

no 8 (2009), pp. 1251-1265.

370.

5762, pp. 34–42.

*Molecular Imaging*, vol 36 (2009), pp. 831–840.

*Med Imag*, vol 27, no 4 (2008), pp. 457-466.

*Analysis*, vol. 13 (2009), pp. 819–845.

*Imag*, vol. 27, no 2, pp. 195-203.

Thesis, University of Göttingen, Germany, 122p.

Portugal, Feb-March 2008.

*ACM Computing Surveys*, vol-36 (2002), pp. 81-121.

Variational Method for Geometric Regularization of Vascular Segmentation in

Methods for Liver Segmentation from CT Datasets, *IEEE Trans on Med Imag*, vol 28,

Minoshima, S. (2009) Computer-assisted diagnostic system for neurodegenerative dementia using brain SPECT and 3D-SSP. *European Journal of Nuclear Medicine and* 

Sets Using Local Shape and Appearance Priors. *MICCAI 2009, Part II, LNCS*, vol.

Quantification of Thyroid Volume Using 3-D Ultrasound Imaging. *IEEE Trans on* 

pulmonary nodules of various densities with morphological approaches and

to 4-D Shape Extraction and Interpretation: Application to Myocardial Motion

segmentation techniques: Models, features and extraction schemes. *Medical Image* 

Automated Segmentation of Spleen and Liver Using Adaptive Enhancement

Classification of brain SPECT imaging using 3D local multifractal spectrum for

3-D+t MRI Data Using an Optimized Nonrigid Temporal Model. *IEEE Trans on Med* 

Review and a New Computational Framework. *Proceedings of International Symposium on Computer Methods in Biomechanics and Biomedical Engineering*, Porto,


Antunes, S.G., Silva, J.S. & Santos, J.B. (2010) A Level Set Segmentation Method of the Four

Ashburner, J., Csernansky, J.G., Davatzikos, C., Fox, N.C., Frisoni, G.B. and Thompson, P.M.

Babaud, J., Witkin, A.P., Baudin, M. & Duda, R.O. (1986). Uniqueness of the Gaussian kernel

Bernard, O. & Friboulet, D. (2009) Fast medical image segmentation through an

Bresser, J., Portegies, M.P., Leemans, A., Biessels, G.J., Kappelle, L.J. & Viergever, M.A.

Campadelli, P., Casiraghi, E. & Esposito, A. (2009) Liver segmentation from computed

Campadelli, P., Casiraghi, E. & Pratissoli, S. (2010) A segmentation framework for abdominal organs from CT scans. *Artificial Intelligence in Medicine*, vol. 50 (2010), pp. 3–11. Cheng, H.D., Cai, X. , Chen, X., Hu, L. & Lou, X. (2003). Computer-aided detection and

Cordero-Grande, L., Vegas-Sánchez-Ferrero. G., Casaseca-de-la-Higuera, P., San-Román-

Duncan, J.S., Ayache, N (2000). Medical Image Analysis: Progress over Two Decades and the

Ecabert, O., Peters, J., Schramm, H., Lorenz, C., von Berg, J., Walker, M.J., Vembar, M.,

Florack, L.M.J., Romeny, B.M.T.H., Koenderink, J.J. & Viergever, M.A.. (1992). Scale and the

Gonzales, R., Woods, R.E. (2000). *Digital Image Processing*. Editora Edgard Blücher Ltda. São

deformable model. *Medical Image Analysis*, vol. 15 (2011), pp. 283–301. Crum, W.R., Phil, D., Hartkens, T. & Hill, D.L.G. (2004) Non-rigid image registration: theory and practice. *The British Journal of Radiology*, vol. 77 (2004),pp. 140–153. Destrempes, F., Meunier, J., Giroux, M.F., Soulez, G., Cloutier, G. (2009) Segmentation in

Ballard, D.H., Brown, C.M. (1982). *Computer Vision*. Prentice-Hall, Inc. Englewood Cliffs Bandyopadhyay, S.K. (2011). A Survey on Brain Image Segmentation Methods. *Journal of* 

vol. 6112/2010 (2010), pp.99-107.

pp. 1223 – 1230.

vol. 45 (2009), pp.185—196.

vol. 36 (2003), pp. 2967-2991.

28, no 2 (2009), pp. 215-229.

9 (2008), pp. 1189-1201.

85-106.

Paulo

brains. *Lancet Neurology*, vol. 2 2003, pp. 79–88.

for scale space filtering, *IEEE PAMI 8*, (1986), pp. 26-33.

*Global Research in Computer Science*, vol 2, no 2 (2011), pp. 4-7.

atrophy progression. *NeuroImage*, vol 54 (2011), pp. 760–768.

Heart Cavities in Pediatric Ultrasound Images. *Lecture Notes in Computer Science*,

(2003). Computer-assisted imaging to assess brain structure in healthy and diseased

approximation of narrow-band B-spline level-set and multiresolution. *IEEE International Symposium on Biomedical Imaging: From Nano to Macro*, pp. 45 – 48. Bouma, H., Sonnemans, J.J., Vilanova, A., Gerritsen, F.A. (2009) Automatic Detection of

Pulmonary Embolism in CTA Images. *IEEE Trans on Med Imag*, vol. 28, no 8 (2009),

(2011). A comparison of MR based segmentation methods for measuring brain

tomography scans: A survey and a new algorithm. *Artificial Intelligence in Medicine*,

classification of microcalcifications in mammograms: a survey, *Pattern Recognition*,

Calvar. J.A., Revilla-Orodea, A., Martín-Fernández, M. & Alberola-López, C. (2011) Unsupervised 4D myocardium segmentation with a Markov Random Field based

Ultrasonic B-Mode Images of Healthy Carotid Arteries Using Mixtures of Nakagami Distributions and Stochastic Optimization. *IEEE Trans on Med Imag*, vol

Challenges Ahead, *IEEE Trans on Pat Anal and Mach Intell*, vol 22, no 1 (2000), pp.

Olszewski, M.E., Subramanyan, K., Lavi, G. & Weese, J. (2008) Automatic Model-Based Segmentation of the Heart in CT Images. *IEEE Trans on Med Imag*, vol. 27, no

differential structure of images, *Image and Vision Computing*, vol 10 (1992), pp. 376-388.


Identification of Structures in Medical Images 25

Ramakrishna, B., Liu, W., Saiprasad, G., Safdar, N., Chang, C.-I., Siddiqui, K., Kim, W.,

Rangayan, R.M. (2005). *Biomedical Image Analysis*, 1st edition Boca Raton Florida © CRC

Rani. U.N., Subbaiah, P.V. & Rao, D.V. (2011). Segmentation of Brain Tumors from Medical

Rebelo, M.S., Furuie, S.S., Gutierrez, M.A., Costa, E.T., Moura, L.A. (2007). Multiscale

Rebelo, M.S., Gutierrez, M.A., Furuie, S.S. & Moura, L.. (2000). Extraction of cardiac

Ribbens, A., Hermans, J., Maes, F., Vandermeulen, D. & Suetens, P., (2010). SPARC: Unified

Rouchdy, Y., Pousin, J., Schaerer, J. & Clarysse, P. (2007). A nonlinear elastic deformable

Ruskó, L., Bekes, G. & Fidrich, M. (2009). Automatic segmentation of the liver from multi-

Samulski, M. & Karssemeijer, N. ( 2011). Optimizing case-based detection performance in a

Schaerer, J., Casta, C., Pousin, J. & Clarysse, P. (2010). A dynamic elastic model for

Scherrer, B., Forbes, F., Garbay, C. & Dojat, M. (2009). Distributed Local MRF Models for

Shiee, N., Bazin, P-R., Ozturk, A., Reich, D.S., Calabresi, P.A. & Pham, D.L. (2010). A

Sonka, M., Hlavac, V., Boyle, R. (1998). *Image processing, analysis, and machine vision*.

Suetens, P., Fua, P. & Hanson, A.J. (1992). Computational strategies for object recognition.

Timp, S., Varela, C. & Karssemeijer, N. (2007). Temporal Change Analysis for

Tsui, B.M.W., Terry, J.A. & Gullberg, G.T. (1993). Evaluation of cardiac cone-beam SPECT

sclerosis lesions. *NeuroImage*, vol 49 (2010), pp. 1524–1535.

Brooks/Cole Publishing Company, Pacific Grove.

*ACM Computing Surveys*, vol. 24, no 1 (1992), pp. 5-55.

MRI. *Inverse Problems*, vol. 23 (2007), pp. 1017–1035. (checar autores) Rousset, O., Rahmim, A., Alavi, A. & Zaidi, H. (2007). Partial Volume Correction Strategies

*Med Imag*, vol. 28, no 8 (2009), pp. 1308 - 1316. (verificar autores)

*Computers in Biology and Medicine,* vol. 37, no 8, pp. 1183-1193

*Biomedical Imaging: From Nano to Macro (2010)*, pp. 856 - 859.

in PET. *PET Clinics*, vol. 2 (2007), pp. 235–249.

(2009), pp. 871–882.

(2011), pp. 1001–1009.

pp. 1278–1295.

*Analysis*, vol. 14 (2010), pp. 738–749.

26, no 7 (2007), pp. 945-953.

vol. 28, no 12 (1993), pp. 1101-12.

approach, *Proc. Computers in Cardiology*, vol 27 (2000), pp. 611-614.

*Science and Security (IJCSS),* vol. 1, no 3, 7p.

Press, 1272p.

Siegel, E., Chai,I.W., Chen, C., Lee, S-K. (2009). An Automatic Computer-Aided Detection System for Meniscal Tears on Magnetic Resonance Images. *IEEE Trans on* 

Scan Images Using DRLSE Variational Levelsets. *International Journal of Computer* 

representation for automatic identification of structures in medical images.

structures through the incorporation of *a priori* knowledge in a multi-scale

Framework for Automatic Segmentation, Probabilistic Atlas Construction, Registration and Clustering of Brain MR Images. *IEEE International Symposium on* 

template for soft structure segmentation: application to the heart segmentation in

and single-phase contrast-enhanced CT images. *Medical Image Analysis*, vol. 13

multi-view CAD system for mammography. *IEEE Trans on Med Imag*, vol. 30, no 4

segmentation and tracking of the heart in MR image sequences. *Medical Image* 

Tno and Structure Brain Segmentation. *IEEE Trans on Med Imag*, vol. 28, no 8 (2009),

topology-preserving approach to the segmentation of brain images with multiple

Characterization of Mass Lesions in Mammography. *IEEE Trans on Med Imag*, vol.

using observer performance experiments and ROC analysis, *Investigative Radiology*.


Marr, D. (2010). *Vision: A Computational Investigation into the Human Representation and* 

Masuda, Y., Foruzan, A.H., Tateyama, T. & Chen, Y.W. (2010) Automatic liver tumor

Murphy, K., van Ginneken, B., Schilham , A.M.R., de Hoop, B.J., Gietema, H.A. & Prokop,

Niessen, W. (1997) *Medical Image Analysis*. PhD Thesis. Utrecht University, The Netherlands. Noble, J.A. (2010) Ultrasound image segmentation and tissue characterization. *Proceedings of* 

O'Brien, S.P., Ghita, O. & Whelan, P.F. (2011). A Novel Model-Based 3D Time Left

O'Donnell, T., Funka-Lea, G., Tek, H., Jolly, M.-P. & Rasch, M. (2006). Comprehensive

Oliver, A., Freixenet, J., Martí, J., Pérez, E., Pont, J., Denton, E.R.E. & Zwiggelaar, R. (2010). A

Pauly, M., Keiser, R. & Gross, M. (2003) Multi-scale Feature Extraction on Point-sampled

Peters, J., Lessick, J., Kneser, R., Wächter, I., Vembar, M., Ecabert, O. & Weese, J. (2010).

Petitjean, C. & Dacher, J-N. (2011). A review of segmentation methods in short axis cardiac

Pu, J., Fuhrman, C., Good, W.F., Sciurba, F.C. & Gur, D. (2011). A Differential Geometric

Pu, J., Leader, J.K., Zheng, B., Knollmann, F., Fuhrman, C., Sciurba, F.C. & Gur, D. (2009). A

in CT Examinations. *IEEE Trans on Med Imag*, vol 28, no 5 (2009), pp. 710-719. Qian, X., Brennan, M.P., Dione, D.P., Dobrucki, W.L., Jackowski, M.P., Breuer, C.K., Sinusas,

MR images. *Medical Image Analysis*, vol. 15, no 2, pp. 169-184.

*International Journal of Computer Vision*, vol. 70, no 2 (2006), pp. 165–178. Oguro, S., Tuncali, K., Elhawary, H., Morrison, P.R., Hata, N. & Silverman, S.G. (2011).

MIT Press. Cambridge, MA.

*Image Analysis*, vol. 13 (2009), pp. 757–770.

*Medical Image Analysis*, vol.14 (2010), pp. 87–110.

Surfaces, *EUROGRAPHICS 2003*, pp. 22.

*Imag*, vol. 30, no 2 (2011), pp. 266-278.

5496/2009, pp. 58-68.

pp. 224-307.

pp. 461-474.

vol. 6 (2011), pp. 111–117.

*Processing of Visual Information*. The 2010 Edition. ISBN-13: 978-0-262-51462-0 The

detection using EM/MPM algorithm and shape information. *2nd International Conference on Software Engineering and Data Mining (SEDM)* 2010, pp. 692–695. Mora, M., Leiva, J. & Olivares, M. (2009) Heart Cavity Segmentation in Ultrasound Images

Based on Supervised Neural Networks. *Lecture Notes in Computer Science*, vol.

M. (2009) A large-scale evaluation of automatic pulmonary nodule detection in chest CT using local image features and k-nearest-neighbour classification. *Medical* 

*the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine* 2010,

Ventricular Segmentation Technique. *IEEE Trans on Med Imag*, vol 30, no 2 (2011),

cardiovascular image analysis using MR and CT at Siemens Corporate Research.

Image registration of pre-procedural MRI and intra-procedural CT images to aid CT-guided percutaneous cryoablation of renal tumors. *International Journal CARS*,

review of automatic mass detection and segmentation in mammographic images.

Accurate Segmentation of the Left Ventricle in Computed Tomography Images for Local Wall Thickness Assessment. *MICCAI 2010, Part I, LNCS*, vol. 6361, pp. 400–408.

Approach to Automated Segmentation of Human Airway Tree. *IEEE Trans on Med* 

Computational Geometry Approach to Automated Pulmonary Fissure Segmentation

A.J. & Papademetris, X. (2009). A non-parametric vessel detection method for complex vascular structures. *Medical Image Analysis*, vol. 13 (2009), pp. 49–61. Rahman, M., Uddin, S. & Hasan, M. (2010). 3D Segmentation and Visualization of Left

Coronary Arteries of Heart Using CT Images. *IJCA Special Issue on "Computer Aided Soft Computing Techniques for Imaging and Biomedical Applications"*, 2010, pp. 88-92.


**2** 

*Tunisia* 

**Detection of Abnormalities in a Biological** 

Diffuse optical tomography (DOT) has a real potential as a new medical diagnostic tool. This fact has stimulated considerable research interest in the last years. When it is possible, this technique has some advantages not available in classical medical imaging methods. Among the benefits of this technique, nonionizing radiation, low-cost instrumentation, mobility and possible functional imaging are the most important. Also, DOT could be used in combination with other imaging methods to provide high-resolution structural information. The transparency of tissues is related to absorption and scattering of light and it reaches its maximum in the near-infrared. Interaction of radiation from this range with biological tissues is widely studied [1-5]. In particular, some works was interested to detection of

Most of models of light interaction with biological matter are based on radiative transfer theory [9-13]. The most used parameters in modelling Laser radiation interaction with biological tissue is absorption and scattering. However some other studies evoked a significant variation of refractive index of abnormal biological tissues especially in the near infrared range. More precisely, experimental results [14-17] showed that the tissue of malignant tumors could manifest an increase of the refractive index which could attain until 10 % of that of a normal tissue which encircles them. So, medical imaging by diffuse optical tomography should take advantage from the emergence of a third contrast parameter which is the refractive index. This drove to the appearance of a big number of numerical and fundamental works in the field of radiative transfer in a varying refractive index medium. Among them some papers are interested to varying refractive index

In this chapter, infrared laser radiation interaction with a biological tissue including a spatially variation of refractive index is studied through a time-dependent radiative transfer model. Our general concern is to contribute at the usability of the radiative transfer theory in a potential optical tomography setting in medical imaging. At this level, studying the effect of refractive index on the transmitted light through a biological rectangular layer is crucial. This should be a contribution to obtaining information about

**1. Introduction** 

biological tissues [18-22].

tumor-like inclusions within a biological tissue [6-8].

**Tissue by Diffuse Optical Tomography:** 

*University of Tunis El Manar, Faculté des Sciences de Tunis Unité de Rayonnement, Département de Physique, Tunis* 

 **A Computational Study** 

H. Trabelsi, M. Gantri and E. Sediki

