**Meet the editor**

Muhammad Tanvir Afzal, PhD., is an Assistant Professor in the department of Computer Science at Mohammad Ali Jinnah University, Islamabad, Pakistan, adjunct professor at Graz University of Technology, Graz, Austria, editor-in-chief of a reputed journal such as: Journal of Universal Computer Science, and group leader of an active research group: Centre for Distributed and Se-

mantic Computing. He earned his PhD with distinction from Graz University of Technology, Austria and secured Gold Medal in his Masters from Quaid-i-Azam University, Islamabad, Pakistan. He has published over 50 research articles in reputed journals and conferences. His research activities encompass the area of Semantic Web, Social Web, Sentiment Analysis, Digital Libraries, eLearning, and Information Integration. He is editor/reviewer/session chair of many journals and conferences. He is supervising/ co-supervising number of PhD students in Austria, Pakistan, and Malaysia, and is involved in number of international research projects.

Contents

**Preface IX** 

**Section 1 Software Engineering 1** 

Chapter 2 **Program Slicing Based** 

Yingzhou Zhang

**Section 2 Applications: Semantic Cache,** 

Chapter 4 **Semantic Cache System 87** 

Chapter 5 **Semantic Interoperability** 

Xueming Qian

Chapter 1 **Using Model Transformation** 

Samuel A. Ajila, Dorina Petriu and Pantanowitz Motshegwa

**on Monadic Semantics 41** 

Chapter 3 **CCMF, Computational Context Modeling** 

**Framework – An Ontological Approach** 

Munir Ahmad, Muhammad Abdul Qadir, Tariq Ali, Muhammad Azeem Abbas and Muhammad Tanvir Afzal

**in E-Health for Improved Healthcare 107** 

Saman Iftikhar, Wajahat Ali Khan, Farooq Ahmad and Kiran Fatima

Chapter 6 **Semantic Based Sport Video Browsing 139** 

Chapter 7 **Intelligent Self-Describing Power Grids 163**  Andrea Schröder and Inaki Laresgoiti

**Language Semantics for Aspects Composition 3** 

**to Develop Context-Aware Web Applications 63**  Luis Paulo Carvalho and Paulo Caetano da Silva

**E-Health, Sport Video Browsing, and Power Grids 85** 

## Contents

### **Preface** XI


#### X Contents

### **Section 3 Visualization 187**


## Preface

Semantics is the research area touching the diversified domains such as: Philosophy, Information Science, Linguistics, Formal Semantics, Philosophy of Language and its constructs, Query Processing, Semantic Web, Pragmatics, Computational Semantics, Programming Languages, and Semantic Memory etc. The current book is a pleasant combination of number of great ideas, applications, case studies, and practical systems in diversified domains. The book has been divided into two volumes. The current one is the second volume which highlights the state-of-the-art areas in the domain of Semantics. This volume has been divided into four sections and ten chapters. The sections include: 1) Software Engineering, 2) Applications: Semantic Cache, E-Health, Sport Video Browsing, and Power Grids, 3) Visualization, and 4) Natural Language Disambiguation.

Section 1 presents work in the domain of Software Engineering. This section includes three chapters. First chapter employs model transformation language semantics for aspect composition. The presented work highlights state-of-the-art in the domain, proposes model composition semantics, and provides formal notions and algorithms. The implementation details using ATL semantics and evaluations have been provided. Second chapter presents a static slice monad transformer for program slicing based on monadic semantics. The work is an extended version of the previously published papers by the authors in the same area. Third Chapter presents a framework for modeling and generation of web service oriented context aware system to improve human-machine interaction.

Section 2 highlights applications of semantics in four areas such as: semantic cache, ehealth, video browsing, and power grid. The section has been divided into four chapters. First chapter presents a semantic cache system. The state-of-the-art provides insights into contemporary systems. Subsequently, a new semantic cache system such as: sCacheQP, has been proposed. The algorithms have been discussed in details with a case study highlighting the major contributions and challenges in the area. Second chapter highlights the use of Semantics in the area of eHealth for providing semantic interoperability. The prototype has been successfully implemented and tested for a local laboratory in Pakistan. Third chapter presents a semantic approach for sport video browsing. Soccer video high-level semantics detection approaches have been described and evaluated. Subsequently, from the detected highlights, a semantic based

#### XII Preface

soccer video browsing approach is proposed which carries out video content browsing using a book-like structure. Fourth chapter describes a futuristic intelligent selfdescribing power grid based on ontologies. The prototype applications demonstrate that the rule based techniques when applied on domain ontologies, are capable to facilitate the coding of industrial applications and their customization for user needs.

Section 3 presents innovative visualization techniques to infer semantics. This section comprises of two chapters. First chapter is a study to propose a methodology to visualize conflict structures of public debate by analyzing debate minutes based on corpus-based discourse analysis. An innovative visualization has been proposed and demonstrated for a data which is able to extract insights from public debate domain, for example, do citizens agree with each other?, do experts and administration have interest conflict? etc. Second chapter elaborates an idea of visualizing program semantics. With enough details, examples, theories, and visualization, the chapter argues that the Petri nets can be extended to formally visualize the semantics of a program.

Section 4, the last section of this book, discusses about natural language disambiguation for resolving topic-focus ambiguities using natural language semantics. The procedural semantics of TIL were shown to provide rigorous analyses such that sentences differing only in their topic-focus articulation were assigned different constructions producing different propositions (truth-conditions) and having different consequences. However, the proposed approach is not able to dictate *which*  disambiguation is the intended one, thus leaving room for pragmatics.

I would like to thank authors who participated to conclude such a nice worth-reading book. I am also thankful to In-Tech Open access Initiative for making accessible all of the chapters online free of cost to the scientific community.

> **Dr. Muhammad Tanvir Afzal** Department of Computer Science Mohammad Ali Jinnah University, Islamabad, Pakistan

## **Section 1**

**Software Engineering** 

**1** 

*Canada* 

**Using Model Transformation Language** 

Samuel A. Ajila, Dorina Petriu and Pantanowitz Motshegwa

Modern software systems are huge, complex, and greatly distributed. In order to design and model such systems, software architects are faced with the problem of cross-cutting concerns much earlier in the development process. At this level, cross-cutting concerns result in model elements that cross-cut the structural and behavioral views of the system. Research has shown that Aspect Oriented (AO) techniques can be applied to software design models. This can greatly help software architects and developers to isolate, reason, express, conceptualize, and work with cross-cutting concerns separately from the core functionality (Ajila et al., 2010; Petriu et al, 2007). This application of AO techniques much earlier in the development process has spawned a new field of study called Aspect-Oriented Modeling (AOM). In AOM, the aspect that encapsulates the cross-cutting behavior or structure is a model, just like the base system model it cross-cuts. A system been modeled has several views including structural and behavioral views. Therefore, a definition of an aspect depends on the view of interest. Unified Modeling Language (UML) provides different diagrams to describe the different views. Class, Object, Composite Structure, Component, Package, and Deployment diagrams can be used to represent the structural view of a system or aspect. On the other hand, Activity, State Machine, and Interaction diagrams are used to model the behavioral view. Interaction diagrams include Sequence, Interaction Overview, Communication, and Timing

After reasoning and working with aspects in isolation, the aspect models eventually have to be combined with the base system model to produce an integrated system model. This merging of the aspect model with the base model is called Aspect Composition or Weaving. Several approaches have been proposed for aspect composition using different technologies/methodologies such as graph transformations (Wittle & Jayaraman, 2007), matching and merging of model elements (Fleury et al., 2007), weaving models (Didonet et al., 2006) and others. The goal of this research is to compose aspect models represented as UML sequence diagrams using transformation models written in Atlas Transformation

Composing behavioral models (views) represented as UML Sequence diagrams is more complex than composing structural views. Not only is the relationships between the

**1. Introduction** 

diagrams.

Language (ATL).

**Semantics for Aspects Composition** 

*Department of Systems and Computer Engineering,* 

*Carleton University, Ottawa, ON,* 

## **Using Model Transformation Language Semantics for Aspects Composition**

Samuel A. Ajila, Dorina Petriu and Pantanowitz Motshegwa *Department of Systems and Computer Engineering, Carleton University, Ottawa, ON, Canada* 

## **1. Introduction**

Modern software systems are huge, complex, and greatly distributed. In order to design and model such systems, software architects are faced with the problem of cross-cutting concerns much earlier in the development process. At this level, cross-cutting concerns result in model elements that cross-cut the structural and behavioral views of the system. Research has shown that Aspect Oriented (AO) techniques can be applied to software design models. This can greatly help software architects and developers to isolate, reason, express, conceptualize, and work with cross-cutting concerns separately from the core functionality (Ajila et al., 2010; Petriu et al, 2007). This application of AO techniques much earlier in the development process has spawned a new field of study called Aspect-Oriented Modeling (AOM). In AOM, the aspect that encapsulates the cross-cutting behavior or structure is a model, just like the base system model it cross-cuts. A system been modeled has several views including structural and behavioral views. Therefore, a definition of an aspect depends on the view of interest. Unified Modeling Language (UML) provides different diagrams to describe the different views. Class, Object, Composite Structure, Component, Package, and Deployment diagrams can be used to represent the structural view of a system or aspect. On the other hand, Activity, State Machine, and Interaction diagrams are used to model the behavioral view. Interaction diagrams include Sequence, Interaction Overview, Communication, and Timing diagrams.

After reasoning and working with aspects in isolation, the aspect models eventually have to be combined with the base system model to produce an integrated system model. This merging of the aspect model with the base model is called Aspect Composition or Weaving. Several approaches have been proposed for aspect composition using different technologies/methodologies such as graph transformations (Wittle & Jayaraman, 2007), matching and merging of model elements (Fleury et al., 2007), weaving models (Didonet et al., 2006) and others. The goal of this research is to compose aspect models represented as UML sequence diagrams using transformation models written in Atlas Transformation Language (ATL).

Composing behavioral models (views) represented as UML Sequence diagrams is more complex than composing structural views. Not only is the relationships between the

Using Model Transformation Language Semantics for Aspects Composition 5

Fig. 2.1. Overview of Model Transformation Adopted from (ATL-User-Manual, 2009).

middleware (e.g., CORBA), operating system (e.g., Linux), hardware, etc.

**2.1 Aspect-Oriented (AO) techniques/technologies** 

software systems (Klein et al., 2007).

OMG's Model-Driven Architecture (MDA) is a term copyrighted by OMG that describes an MDE approach supported by OMG standards; namely, UML, Meta-Object Facility (MOF), MOF-Query/View/Transformation (QVT), XML Metadata Interchange (XMI) and Common Warehouse Metamodel (CWM). MDA decouples the business and application logic from the underlying platform technology through the use of the Platform Independent Model (PIM), Platform Specific Model (PSM) and model transformations. The PIM describes a software system independently of the platform that supports it while PSM expresses how the core application functionality is realized on a specific platform. Given a specific platform, the PIM is transformed to PSM. Platform in this case refers to technological and engineering details that are independent of the core functionality of the application. For example,

The size of modern software systems has increased tremendously. Software architects and developers have to design and develop systems that are not only enormous, but are more complex, and greatly distributed. These systems naturally have many cross-cutting concerns (requirements) whose solutions tend to cross-cut the base architecture and system behavior. Such concerns include security, persistence, system logging, new features in software product lines, and many others. Aspect-Oriented techniques allow software developers and architects to conceptualize and work with multiple concerns separately (Groher & Voelter, 2007; Kienzle et al., 2009; Petriu et al., 2007). These techniques allow us to modularize concerns that we cannot easily modularize with current Object-Oriented (OO) techniques (Whittle & Jayaraman, 2007). The final system is then produced by weaving or composing solutions from separate concerns (Petriu et al., 2007). Klein et al. point out that dividing and conquering these cross-cutting concerns also allows us to better maintain and evolve

Aspect Oriented Programming (AOP) applies AO techniques at code level (France et al., 2004; Petriu et al., 2007). AOP was introduced to enhance Object-Oriented Programming to better handle cross-cutting concerns that cause code scattering and tangling, which leads to code that is very difficult to maintain and impossible to reuse or modify. AOP addresses these issues by introducing a class like programming construct called an *aspect* which is used to encapsulate cross-cutting concerns. Just like a class, an aspect has attributes (state) and methods (behavior). An aspect also introduces concepts well known to AO community; namely, *join points*, *advice*, and *pointcut*. Join points are points in the code where the crosscutting behavior is to be inserted. AspectJ (a popular AOP Java tool) supports join points for

model elements important but the order is equally paramount. Several approaches have been proposed for composing behavioral aspects with core system behavior, and these include graphs transformations [Whittle et al., 2007] and generic weavers [Morin et al., 2008]. In this research work we view composition as a form of model transformation. Aspect composition can be considered a model transformation since it transforms aspect and primary models to an integrated system model. Toward this end, we propose and focus on an approach that uses model transformations to compose both primary and aspect models represented as UML Sequence diagrams (SDs). SDs modeling the primary model and generic aspect models is created using Graphical UML modeling tools like Rational Software Architect (RSA). Model transformations are then used to instantiate the generic aspect models in the context of the application to produce context specific aspect models. Binding rules used for instantiating the generic aspect are represented as mark models that conform to a metamodel. Using other model transformations, the context specific aspect models are then composed with the primary model to produce an integrated system model. Verification and validation is performed, not only to verify that composition was successful, but also to ensure that the composed model is a valid UML model that can be processed further and shared with other researchers.

The rest of this chapter is structured as follows. Section two presents Model Driven approach, Aspect-Oriented techniques and technologies, and Atlas Transformation Language (ATL). We present our approach to model composition in section three starting with an example. We introduce our model composition semantics and definitions in section four – giving formal notions and three major algorithms (pointcut detection, advice composition, and complete composition) that define the basis of our work. Section five presents the design and implementation of our model composition using ATL semantics. We introduce and analyze a case study based on phone call features in section six. Section seven gives our conclusion, limitations, and future work.

## **2. Model Driven Engineering/Development/Architecture (MDE/MDD/MDA)**

In Model Driven Engineering (MDE) everything is a model. A model refers to a simplified view of a real world system of interest; that is, an abstraction of a system. MDE considers models as the building blocks or first class entities (Didonet et al, 2006). A model conforms to a metamodel while a metamodel conforms to a metametamodel. MDE is mainly concerned with the evolution of models as a way of developing software by focusing on models. With this new paradigm of software development, the code will be generated automatically by model to code transformations. Model Driven Development (MDD) is copyrighted term by Object Management Group (OMG). One of the most important operations in MDE/MDD is model transformation. There are different kinds of model transformations including model to code, model to text, and model to model. Our interest in this paper is in model to model transformations. Figure 2.1 shows the process of model transformation. Since every artifact in MDE is a model, the model transformation is also a model that conforms to a metamodel. The transformation model defines how to generate models that conform to a particular metamodel from models that conform to another metamodel or the same metamodel. In Figure 2.1, the transformation model Mt transforms Ma to Mb. Mt conforms to MMt while Ma and Mb conform to MMa and MMb respectively. The three metamodels conform to a common metametamodel MMM.

model elements important but the order is equally paramount. Several approaches have been proposed for composing behavioral aspects with core system behavior, and these include graphs transformations [Whittle et al., 2007] and generic weavers [Morin et al., 2008]. In this research work we view composition as a form of model transformation. Aspect composition can be considered a model transformation since it transforms aspect and primary models to an integrated system model. Toward this end, we propose and focus on an approach that uses model transformations to compose both primary and aspect models represented as UML Sequence diagrams (SDs). SDs modeling the primary model and generic aspect models is created using Graphical UML modeling tools like Rational Software Architect (RSA). Model transformations are then used to instantiate the generic aspect models in the context of the application to produce context specific aspect models. Binding rules used for instantiating the generic aspect are represented as mark models that conform to a metamodel. Using other model transformations, the context specific aspect models are then composed with the primary model to produce an integrated system model. Verification and validation is performed, not only to verify that composition was successful, but also to ensure that the composed model is a valid UML

The rest of this chapter is structured as follows. Section two presents Model Driven approach, Aspect-Oriented techniques and technologies, and Atlas Transformation Language (ATL). We present our approach to model composition in section three starting with an example. We introduce our model composition semantics and definitions in section four – giving formal notions and three major algorithms (pointcut detection, advice composition, and complete composition) that define the basis of our work. Section five presents the design and implementation of our model composition using ATL semantics. We introduce and analyze a case study based on phone call features in section six. Section

**2. Model Driven Engineering/Development/Architecture (MDE/MDD/MDA)** 

In Model Driven Engineering (MDE) everything is a model. A model refers to a simplified view of a real world system of interest; that is, an abstraction of a system. MDE considers models as the building blocks or first class entities (Didonet et al, 2006). A model conforms to a metamodel while a metamodel conforms to a metametamodel. MDE is mainly concerned with the evolution of models as a way of developing software by focusing on models. With this new paradigm of software development, the code will be generated automatically by model to code transformations. Model Driven Development (MDD) is copyrighted term by Object Management Group (OMG). One of the most important operations in MDE/MDD is model transformation. There are different kinds of model transformations including model to code, model to text, and model to model. Our interest in this paper is in model to model transformations. Figure 2.1 shows the process of model transformation. Since every artifact in MDE is a model, the model transformation is also a model that conforms to a metamodel. The transformation model defines how to generate models that conform to a particular metamodel from models that conform to another metamodel or the same metamodel. In Figure 2.1, the transformation model Mt transforms Ma to Mb. Mt conforms to MMt while Ma and Mb conform to MMa and MMb respectively.

model that can be processed further and shared with other researchers.

seven gives our conclusion, limitations, and future work.

The three metamodels conform to a common metametamodel MMM.

Fig. 2.1. Overview of Model Transformation Adopted from (ATL-User-Manual, 2009).

OMG's Model-Driven Architecture (MDA) is a term copyrighted by OMG that describes an MDE approach supported by OMG standards; namely, UML, Meta-Object Facility (MOF), MOF-Query/View/Transformation (QVT), XML Metadata Interchange (XMI) and Common Warehouse Metamodel (CWM). MDA decouples the business and application logic from the underlying platform technology through the use of the Platform Independent Model (PIM), Platform Specific Model (PSM) and model transformations. The PIM describes a software system independently of the platform that supports it while PSM expresses how the core application functionality is realized on a specific platform. Given a specific platform, the PIM is transformed to PSM. Platform in this case refers to technological and engineering details that are independent of the core functionality of the application. For example, middleware (e.g., CORBA), operating system (e.g., Linux), hardware, etc.

#### **2.1 Aspect-Oriented (AO) techniques/technologies**

The size of modern software systems has increased tremendously. Software architects and developers have to design and develop systems that are not only enormous, but are more complex, and greatly distributed. These systems naturally have many cross-cutting concerns (requirements) whose solutions tend to cross-cut the base architecture and system behavior. Such concerns include security, persistence, system logging, new features in software product lines, and many others. Aspect-Oriented techniques allow software developers and architects to conceptualize and work with multiple concerns separately (Groher & Voelter, 2007; Kienzle et al., 2009; Petriu et al., 2007). These techniques allow us to modularize concerns that we cannot easily modularize with current Object-Oriented (OO) techniques (Whittle & Jayaraman, 2007). The final system is then produced by weaving or composing solutions from separate concerns (Petriu et al., 2007). Klein et al. point out that dividing and conquering these cross-cutting concerns also allows us to better maintain and evolve software systems (Klein et al., 2007).

Aspect Oriented Programming (AOP) applies AO techniques at code level (France et al., 2004; Petriu et al., 2007). AOP was introduced to enhance Object-Oriented Programming to better handle cross-cutting concerns that cause code scattering and tangling, which leads to code that is very difficult to maintain and impossible to reuse or modify. AOP addresses these issues by introducing a class like programming construct called an *aspect* which is used to encapsulate cross-cutting concerns. Just like a class, an aspect has attributes (state) and methods (behavior). An aspect also introduces concepts well known to AO community; namely, *join points*, *advice*, and *pointcut*. Join points are points in the code where the crosscutting behavior is to be inserted. AspectJ (a popular AOP Java tool) supports join points for

Using Model Transformation Language Semantics for Aspects Composition 7

sub-types of the *OCLAny* abstract super-type. An ATL module or program, like Families2Persons.atl in the previous example, defines a model to model transformation (Jouault & Kurtev, 2005). It consists of a header, helpers (attribute and operation helpers) and transformation rules (matched, called and lazy rules) (Jouault & Kurtev, 2005). The header defines the module's name, and the input and target models. ATL operation helpers are more like functions or Java methods, and can be invoked from rules and other helpers. Attribute helpers unlike operation helpers do not take any arguments. All helpers are, however, recursive and must have a return value. Rules define how input models are transformed to target models. They are the core construct in ATL (Jouault & Kurtev, 2005). ATL supports both declarative and imperative rules. Declarative rules include matched rules and lazy rules. Lazy rules are similar to matched rules but can only be invoked from other rules. A matched rule consists of source pattern and target pattern (Jouault & Kurtev, 2005). A source pattern is defined as an OCL expression and defines what type of input elements will be matched (ATL-User-Guide, 2009). An ATL model is compiled, and then executed on the ATL engine that has two model handlers; namely, EMF (Eclipse Modeling framework) and MDR (Meta Data repository) (ATL-User-Guide, 2009). Model handlers provide a programming interface for developers to manipulate models (Jouault & Kurtev, 2005). The EMF handler allows for manipulation of Ecore models while MDR allows the ATL engine to handle models that conform to the MOF 1.4 (Meta Object Facility) metametamodel (ATL-User-Guide, 2009). For example, the ATL transformation in Figure 4.1

would require an EMF model handler since the metamodels conform to Ecore.

Fig. 3.1. Our AOM Composition Approach.

Fig. 2.2. Overview of the Family to Person ATL Transformation.

method invocations, initializing of attributes, exception handling, etc (Colyer et al., 2004). A *pointcut* is used to describe a condition that matches join points, that is it is a way of defining join points of interest where we want to insert the cross-cutting functionality. This crosscutting behavior to be inserted at a join point is defined in the *advice*. AspectJ supports *before*, *after*, and *around* advices (Colyer et al., 2004).

Recent work in AO has focused on applying AO techniques much earlier in the development process (Kienzle et al., 2009; Klein et al., 2007; Morin et al., 2008; Petriu et al., 2007). In Aspect Oriented Modeling (AOM), Aspect-Oriented techniques are applied to models (unlike AOP). Whittle et al. define an AO model as "a model that cross-cuts other models *at the same level of abstraction*" (Whittle et al., 2006). Aspects are considered models as well; hence, it makes sense to define (or abstract) other concepts such as pointcuts and advices as models. However, the precise definition of a joint point, pointcut or advice depends on our modeling view. For example, in a structural view, such as a class diagram, an aspect is defined in terms of classes and operations/methods whereas in a behavioral view, such as a sequence diagram, an aspect is defined in terms of messages and lifelines. This has resulted in several approaches to AOM most of which have focused on separation and weaving (composition) of structural (class diagrams) and behavioral views (sequence, activity, and state diagrams) (Klein et al., 2007; Morin et al., 2008; Petriu et al., 2007). Several approaches have been proposed for composing aspects in AOM. These include using graph transformations (Gong, 2008; Whittle & Jayaraman, 2007), semantics (Klein et al., 2006), executable class diagrams (ECDs), weaving models (Didonet et al., 2006), generic approaches and frameworks (Fleury et al., 2008; Morin et al., 2008), etc. Composition primarily involves deciding what has to be composed, where to compose, and how to compose (Didonet et al., 2006). Aspect composition can either be symmetric or asymmetric. In symmetric composition, there is a clear distinction between the models to be composed; that is, one model plays the role of a base model while the other is declared an aspect model (Jeanneret et al, 2008). This distinction is absent in asymmetric composition.

## **2.2 Atlas transformation language**

The Atlas Transformation Language (ATL) is a model transformation language from the ATLAS INRIA & LINA research group (ATL-User-Guide, 2009). The language is both declarative and imperative, and allows developers to transform a set of input models to a number of output target models. In ATL, source or input models can only be navigated but cannot be modified (Jouault & Kurtev, 2005) whereas target models are write-only and cannot be navigated. Figure 2.2 below shows an overview of an example of an ATL transformation (Family2Person) from [ATL\_Examples] that transforms a Family model to a Person model. The Family model conforms to an *MMFamily* metamodel whereas the Person model conforms to an *MMPerson* metamodel. The ATL, *MMFamily*, and *MMPerson* metamodels all conform to the Ecore metametamodel which is a metamodel for the Eclipse Modeling Framework. Families2Persons.atl is an ATL model or program that transforms a Family model to a Person model.

ATL has three types of units that are defined on separate files (ATL-User-Guide, 2009); namely, ATL modules, queries and libraries. ATL has types and expressions that are based on the Object Constraint Language (OCL) from OMG. ATL has primitive types (Numeric, String, Boolean), collection type (sets, sequences and bags) and other types, all of which are

method invocations, initializing of attributes, exception handling, etc (Colyer et al., 2004). A *pointcut* is used to describe a condition that matches join points, that is it is a way of defining join points of interest where we want to insert the cross-cutting functionality. This crosscutting behavior to be inserted at a join point is defined in the *advice*. AspectJ supports *before*,

Recent work in AO has focused on applying AO techniques much earlier in the development process (Kienzle et al., 2009; Klein et al., 2007; Morin et al., 2008; Petriu et al., 2007). In Aspect Oriented Modeling (AOM), Aspect-Oriented techniques are applied to models (unlike AOP). Whittle et al. define an AO model as "a model that cross-cuts other models *at the same level of abstraction*" (Whittle et al., 2006). Aspects are considered models as well; hence, it makes sense to define (or abstract) other concepts such as pointcuts and advices as models. However, the precise definition of a joint point, pointcut or advice depends on our modeling view. For example, in a structural view, such as a class diagram, an aspect is defined in terms of classes and operations/methods whereas in a behavioral view, such as a sequence diagram, an aspect is defined in terms of messages and lifelines. This has resulted in several approaches to AOM most of which have focused on separation and weaving (composition) of structural (class diagrams) and behavioral views (sequence, activity, and state diagrams) (Klein et al., 2007; Morin et al., 2008; Petriu et al., 2007). Several approaches have been proposed for composing aspects in AOM. These include using graph transformations (Gong, 2008; Whittle & Jayaraman, 2007), semantics (Klein et al., 2006), executable class diagrams (ECDs), weaving models (Didonet et al., 2006), generic approaches and frameworks (Fleury et al., 2008; Morin et al., 2008), etc. Composition primarily involves deciding what has to be composed, where to compose, and how to compose (Didonet et al., 2006). Aspect composition can either be symmetric or asymmetric. In symmetric composition, there is a clear distinction between the models to be composed; that is, one model plays the role of a base model while the other is declared an aspect model

(Jeanneret et al, 2008). This distinction is absent in asymmetric composition.

The Atlas Transformation Language (ATL) is a model transformation language from the ATLAS INRIA & LINA research group (ATL-User-Guide, 2009). The language is both declarative and imperative, and allows developers to transform a set of input models to a number of output target models. In ATL, source or input models can only be navigated but cannot be modified (Jouault & Kurtev, 2005) whereas target models are write-only and cannot be navigated. Figure 2.2 below shows an overview of an example of an ATL transformation (Family2Person) from [ATL\_Examples] that transforms a Family model to a Person model. The Family model conforms to an *MMFamily* metamodel whereas the Person model conforms to an *MMPerson* metamodel. The ATL, *MMFamily*, and *MMPerson* metamodels all conform to the Ecore metametamodel which is a metamodel for the Eclipse Modeling Framework. Families2Persons.atl is an ATL model or program that transforms a

ATL has three types of units that are defined on separate files (ATL-User-Guide, 2009); namely, ATL modules, queries and libraries. ATL has types and expressions that are based on the Object Constraint Language (OCL) from OMG. ATL has primitive types (Numeric, String, Boolean), collection type (sets, sequences and bags) and other types, all of which are

*after*, and *around* advices (Colyer et al., 2004).

**2.2 Atlas transformation language** 

Family model to a Person model.

Fig. 2.2. Overview of the Family to Person ATL Transformation.

sub-types of the *OCLAny* abstract super-type. An ATL module or program, like Families2Persons.atl in the previous example, defines a model to model transformation (Jouault & Kurtev, 2005). It consists of a header, helpers (attribute and operation helpers) and transformation rules (matched, called and lazy rules) (Jouault & Kurtev, 2005). The header defines the module's name, and the input and target models. ATL operation helpers are more like functions or Java methods, and can be invoked from rules and other helpers. Attribute helpers unlike operation helpers do not take any arguments. All helpers are, however, recursive and must have a return value. Rules define how input models are transformed to target models. They are the core construct in ATL (Jouault & Kurtev, 2005). ATL supports both declarative and imperative rules. Declarative rules include matched rules and lazy rules. Lazy rules are similar to matched rules but can only be invoked from other rules. A matched rule consists of source pattern and target pattern (Jouault & Kurtev, 2005). A source pattern is defined as an OCL expression and defines what type of input elements will be matched (ATL-User-Guide, 2009). An ATL model is compiled, and then executed on the ATL engine that has two model handlers; namely, EMF (Eclipse Modeling framework) and MDR (Meta Data repository) (ATL-User-Guide, 2009). Model handlers provide a programming interface for developers to manipulate models (Jouault & Kurtev, 2005). The EMF handler allows for manipulation of Ecore models while MDR allows the ATL engine to handle models that conform to the MOF 1.4 (Meta Object Facility) metametamodel (ATL-User-Guide, 2009). For example, the ATL transformation in Figure 4.1 would require an EMF model handler since the metamodels conform to Ecore.

Fig. 3.1. Our AOM Composition Approach.

Using Model Transformation Language Semantics for Aspects Composition 9

To create a generic aspect we adopt the use of template parameters used by France et al. (France et al., 2004) and others (Kienzle et al., 2009; Petriu et al., 2007) to define generic roles played by the participants and messages in the generic aspect model. These generic roles are then bound to specific roles (names) when the generic aspect is instantiated. Figure 3.3 shows the pointcut and advice that make up our generic security aspect model. It should be noted that in case of multiple aspects, each aspect will be modeled separately. The lifelines (participants) and messages in the model are made generic. The pointcut in Figure 3.3a defines that the aspect detects any sequence of messages between a lifeline that plays the role of |client and lifeline that plays the role of |server such that |client sends a message tied to the role |operation and |server responds with |retry. During instantiation these template parameters (roles) will be set (bound) to concrete names of appropriate lifelines

As already mentioned, the advice represents the new or additional behavior we want executed if the pointcut matches, that is, if we find the sequence of messages defined in the pointcut in our primary model. The advice in Figure 3.3b declares that we want |server to invoke a self call after receiving |operation and before sending |retry to |client. So our advice in this case adds new behavior (the |handle\_error self call). The idea is that during composition, as we shall see later, we replace whatever was matched by pointcut with what

Before an aspect can be composed with the primary model, the generic aspect model must first be instantiated to the context of the application to produce a Context Specific Aspect Model. This is achieved by "binding" the template parameters to application specific values. For example, we want to bind "customer" to |client because in our primary model,

Instantiating our generic aspect model using the bindings in Table 1, we obtain the context specific aspect model shown in Figure 3.4. The pointcut from the context specific aspect will then match the sending of a *login* message from *customer* to *server* and a try\_*again* message from *server* back to *customer*, which is what we want. Its advice declares that the *save\_bad\_attempt* self call will be added to the server hopefully for the server to do

(a) pointcut (b) Advice

and messages.

is defined in the advice.

customer plays the role of |client.

something useful and security related.

Fig. 3.3. Generic Aspect Model.

## **3. Our approach to model composition**

Our approach shown in the Figure 3.1 is an adaptation of the approach proposed by Petriu et al. in (Petriu et al., 2007). Using a UML modeling tool, like RSA (Rational Software Architect) from IBM, the primary and generic aspect models are modeled in UML and then exported to a UML 2.1 (.uml) format file. The mark model is created in an XMI file. The *Instantiate* and *Compose* operations in Figure 3.1 are defined as ATL model transformations. We first instantiate a generic aspect model to the context of the application by using a model transformation that takes the primary, generic aspect and mark models as input, and transforms them to a context specific aspect model. We then invoke a second transformation that will take as input the newly created context specific aspect model and the primary model, and then output a composed target model.

## **3.1 Example**

Let us introduce a simple example to provide a better view of our approach and the definitions of the various concepts used in the approach. This example is adapted from Klein et al. (Klein et al., 2007). It illustrates the weaving of a simple security aspect into a primary model. The primary model consists of a single scenario. In fact, our composition approach assumes that the primary model has only one sequence diagram (SD) and models a particular instance of a use case. This example consists of a login scenario shown in Figure 3.2 below. The model shows a simple iteration between instances of Server and Customer. The Customer attempts to log into the Server by sending a *login* message. The Customer's login details are incorrect; hence, the Server sends a *try\_again* message to the Customer to attempt another login. The Customer then sends a *login* message with the correct details this time and the Server responds with an *ok* message.

The primary model does not have any security features. So, we want to add some security mechanism to the scenario so that when a customer attempts login and fails, the system should do something about that exception. We can model this security mechanism as a Security Aspect model that will detect a presence of a message from the Customer to the Server, and a reply from the Server back to the Customer. The presence of this sequence of messages is defined in the aspect's pointcut. The new behavior we want to add to the primary model in order to enhance security is defined in the aspect's advice. However, to make the aspect reusable and more useful, it has to be generic but not specific to our example or situation. This way we can reuse the aspect and in different situations and scenarios.

Fig. 3.2. The Primary Model - A Login Scenario for a Security Aspect Example.

Our approach shown in the Figure 3.1 is an adaptation of the approach proposed by Petriu et al. in (Petriu et al., 2007). Using a UML modeling tool, like RSA (Rational Software Architect) from IBM, the primary and generic aspect models are modeled in UML and then exported to a UML 2.1 (.uml) format file. The mark model is created in an XMI file. The *Instantiate* and *Compose* operations in Figure 3.1 are defined as ATL model transformations. We first instantiate a generic aspect model to the context of the application by using a model transformation that takes the primary, generic aspect and mark models as input, and transforms them to a context specific aspect model. We then invoke a second transformation that will take as input the newly created context specific aspect model and the primary

Let us introduce a simple example to provide a better view of our approach and the definitions of the various concepts used in the approach. This example is adapted from Klein et al. (Klein et al., 2007). It illustrates the weaving of a simple security aspect into a primary model. The primary model consists of a single scenario. In fact, our composition approach assumes that the primary model has only one sequence diagram (SD) and models a particular instance of a use case. This example consists of a login scenario shown in Figure 3.2 below. The model shows a simple iteration between instances of Server and Customer. The Customer attempts to log into the Server by sending a *login* message. The Customer's login details are incorrect; hence, the Server sends a *try\_again* message to the Customer to attempt another login. The Customer then sends a *login* message with the correct details this

The primary model does not have any security features. So, we want to add some security mechanism to the scenario so that when a customer attempts login and fails, the system should do something about that exception. We can model this security mechanism as a Security Aspect model that will detect a presence of a message from the Customer to the Server, and a reply from the Server back to the Customer. The presence of this sequence of messages is defined in the aspect's pointcut. The new behavior we want to add to the primary model in order to enhance security is defined in the aspect's advice. However, to make the aspect reusable and more useful, it has to be generic but not specific to our example or situation. This

**3. Our approach to model composition** 

model, and then output a composed target model.

time and the Server responds with an *ok* message.

way we can reuse the aspect and in different situations and scenarios.

Fig. 3.2. The Primary Model - A Login Scenario for a Security Aspect Example.

**3.1 Example** 

To create a generic aspect we adopt the use of template parameters used by France et al. (France et al., 2004) and others (Kienzle et al., 2009; Petriu et al., 2007) to define generic roles played by the participants and messages in the generic aspect model. These generic roles are then bound to specific roles (names) when the generic aspect is instantiated. Figure 3.3 shows the pointcut and advice that make up our generic security aspect model. It should be noted that in case of multiple aspects, each aspect will be modeled separately. The lifelines (participants) and messages in the model are made generic. The pointcut in Figure 3.3a defines that the aspect detects any sequence of messages between a lifeline that plays the role of |client and lifeline that plays the role of |server such that |client sends a message tied to the role |operation and |server responds with |retry. During instantiation these template parameters (roles) will be set (bound) to concrete names of appropriate lifelines and messages.

As already mentioned, the advice represents the new or additional behavior we want executed if the pointcut matches, that is, if we find the sequence of messages defined in the pointcut in our primary model. The advice in Figure 3.3b declares that we want |server to invoke a self call after receiving |operation and before sending |retry to |client. So our advice in this case adds new behavior (the |handle\_error self call). The idea is that during composition, as we shall see later, we replace whatever was matched by pointcut with what is defined in the advice.

Before an aspect can be composed with the primary model, the generic aspect model must first be instantiated to the context of the application to produce a Context Specific Aspect Model. This is achieved by "binding" the template parameters to application specific values. For example, we want to bind "customer" to |client because in our primary model, customer plays the role of |client.

Instantiating our generic aspect model using the bindings in Table 1, we obtain the context specific aspect model shown in Figure 3.4. The pointcut from the context specific aspect will then match the sending of a *login* message from *customer* to *server* and a try\_*again* message from *server* back to *customer*, which is what we want. Its advice declares that the *save\_bad\_attempt* self call will be added to the server hopefully for the server to do something useful and security related.

Fig. 3.3. Generic Aspect Model.

Using Model Transformation Language Semantics for Aspects Composition 11

A sequence diagram shows the order in which messages are exchanged among participants; hence, order is crucial [Hamilton & Miles, 2006; Pilone & Pitman, 2005). The messages or interactions, to be precise, on a specific lifeline are totally ordered but interactions between two lifelines are partially ordered. The most important model elements in a SD are probably lifelines (participants), messages, message ends, and the enclosing interaction. Figure 4.1 is a simplified class diagram of the Interactions Metamodel showing the relationships among

A complete description of each metaclass can be obtained from the UML specification (OMG, 2009). The *InteractionFragment* abstract class represents a general concept of an interaction (OMG, 2009). An *Interaction* is a sub class of *InteractionFragment* that represents the modeled behavior or interactions (exchange of messages) between participants (lifelines)[OMG09]. An *Interaction* essentially encloses *Messages*, *Lifelines* and other *InteractionFragments*. The enclosed *InteractionFragments* are stored in an ordered list referenced by the *fragment* role. This ordering is exploited in our algorithms for matching and composing SDs. A *Message* models the kind of communication between participants [OMG09]. There are five main types of messages; namely, synchronous, asynchronous, delete, create, and reply messages [Hamilton+06]. Each message is accompanied by a pair of MessageOccurrenceSpecifications (MOSs). The *sendEvent* MOS represents the sending of the message while *receiveEvent* MOS models the reception of the message. Each MOS also has a reference to the lifeline for which the message is received or sent from through the *covered* association. In return, each *Lifeline* has a reference to a list of *InteractionFragments* or specializations of *InteractionFragment* (including MOSs), which cover the lifeline, through the

The events that we are interested in are specializations of the *MessageEvent* abstract class mainly the *SendOperationEvent* (SOE) and *ReceiveOperationEvent* (ROE) classes. These types of events occur during the sending or receiving of a request for an operation invocation

Fig. 3.5. Aspect Composition as an ATL model transformation.

**4. Model composition semantics and definitions** 

the metaclasses for these model elements.

*coveredBy* association.

(OMG, 2009).


Table 1. Example of Security Aspect Binding Rules.


(a) Pointcut(b) Advice

Fig. 3.4. Context Specific Aspect Model.

## **3.2 Model composition**

After instantiating a context specific aspect model, a complete integrated system is obtained by composing the primary model with the context specific aspect model. We view composition as a form of model transformation as shown in Figure 3.5. Therefore, our aim is to transform the input models (Primary and Context Specific Aspect) to a target composed model. As shown in Figure 3.5, both the input and output models conform to an EMF implementation of the UML metamodel specification while our ATL program or model conforms to the ATL metamodel. All the metamodels conform to the EMF's Ecore metametamodel. As in other aspect composition approaches composition has to be performed on different views, that is, structural or behavioral views. Our main focus is on the behavioral view. Composition inevitably results in some model elements been replaced, removed, added or merged (Morin et al., 2008; Gong, 2008). Similarly in our approach, all model elements from the context specific aspect model that are not already in the primary model, will be added to the composed model but elements common to both models will be merged. All join point model elements (from primary model) are replaced by advice elements. The rest of the elements from the primary model will be added to the composed model. A formal definition of our models and the proposed algorithm (for matching and composing) are based on UML metamodel classes. The specification for the UML metamodel (OMG, 2009) is enormous and also includes metaclasses for other UML diagrams that we are not interested in. Therefore, it makes sense to look only at some of the important classes whose objects are used in creating sequence diagrams (SDs).


**Parameter Binding value Comment**  |client customer Lifeline object name. |server server Lifeline object name.

(a) Pointcut(b) Advice

important classes whose objects are used in creating sequence diagrams (SDs).

After instantiating a context specific aspect model, a complete integrated system is obtained by composing the primary model with the context specific aspect model. We view composition as a form of model transformation as shown in Figure 3.5. Therefore, our aim is to transform the input models (Primary and Context Specific Aspect) to a target composed model. As shown in Figure 3.5, both the input and output models conform to an EMF implementation of the UML metamodel specification while our ATL program or model conforms to the ATL metamodel. All the metamodels conform to the EMF's Ecore metametamodel. As in other aspect composition approaches composition has to be performed on different views, that is, structural or behavioral views. Our main focus is on the behavioral view. Composition inevitably results in some model elements been replaced, removed, added or merged (Morin et al., 2008; Gong, 2008). Similarly in our approach, all model elements from the context specific aspect model that are not already in the primary model, will be added to the composed model but elements common to both models will be merged. All join point model elements (from primary model) are replaced by advice elements. The rest of the elements from the primary model will be added to the composed model. A formal definition of our models and the proposed algorithm (for matching and composing) are based on UML metamodel classes. The specification for the UML metamodel (OMG, 2009) is enormous and also includes metaclasses for other UML diagrams that we are not interested in. Therefore, it makes sense to look only at some of the


Fig. 3.4. Context Specific Aspect Model.

**3.2 Model composition** 

Table 1. Example of Security Aspect Binding Rules.

Fig. 3.5. Aspect Composition as an ATL model transformation.

## **4. Model composition semantics and definitions**

A sequence diagram shows the order in which messages are exchanged among participants; hence, order is crucial [Hamilton & Miles, 2006; Pilone & Pitman, 2005). The messages or interactions, to be precise, on a specific lifeline are totally ordered but interactions between two lifelines are partially ordered. The most important model elements in a SD are probably lifelines (participants), messages, message ends, and the enclosing interaction. Figure 4.1 is a simplified class diagram of the Interactions Metamodel showing the relationships among the metaclasses for these model elements.

A complete description of each metaclass can be obtained from the UML specification (OMG, 2009). The *InteractionFragment* abstract class represents a general concept of an interaction (OMG, 2009). An *Interaction* is a sub class of *InteractionFragment* that represents the modeled behavior or interactions (exchange of messages) between participants (lifelines)[OMG09]. An *Interaction* essentially encloses *Messages*, *Lifelines* and other *InteractionFragments*. The enclosed *InteractionFragments* are stored in an ordered list referenced by the *fragment* role. This ordering is exploited in our algorithms for matching and composing SDs. A *Message* models the kind of communication between participants [OMG09]. There are five main types of messages; namely, synchronous, asynchronous, delete, create, and reply messages [Hamilton+06]. Each message is accompanied by a pair of MessageOccurrenceSpecifications (MOSs). The *sendEvent* MOS represents the sending of the message while *receiveEvent* MOS models the reception of the message. Each MOS also has a reference to the lifeline for which the message is received or sent from through the *covered* association. In return, each *Lifeline* has a reference to a list of *InteractionFragments* or specializations of *InteractionFragment* (including MOSs), which cover the lifeline, through the *coveredBy* association.

The events that we are interested in are specializations of the *MessageEvent* abstract class mainly the *SendOperationEvent* (SOE) and *ReceiveOperationEvent* (ROE) classes. These types of events occur during the sending or receiving of a request for an operation invocation (OMG, 2009).

Using Model Transformation Language Semantics for Aspects Composition 13

points with the advice SDs. Before instantiating a generic aspect, we first have to ensure that the aspect can be applied to the primary model; that is, whether its pointcut matches. A formal definition of matching will be given later. Also during composition we have to find where to weave. This makes pointcut detection or finding join points a core operation. The algorithm designed for pointcut detection manipulates the SD metaclasses by exploiting the relationship between *InteractionFragments* and their ordered list of fragments in an interaction. It also makes use of the fact that a sequence of messages (and indeed a SD) is

**P** be a sequence of fragments, that is, *InteractionFragments* (CombinedFragments and

Then, **P** = Sequence{f1, ..., fψ} where ψ = number of fragments in **P** and fi is either a

**CF**(O, Λ) CF **O** is a sequence of operands in the **CF** and each operand is

Ei is an event associated with fi. Mi is a message associated with fi.

**C** = Sequence{c1, ..., cμ} where μ = number of fragments in **C** and ci is also either a CF or a

**CF**(O, Λ) CF **O** is a sequence of operands in the **CF** and each operand is

Ei is an event associated with ci. Mi is a message associated with ci.

Using the above notation, we will define an aspect model as a pair of fragment sequences, that is, **Aspect** = (**P**, **A**) where **P** and **A** are the sequences defined earlier. This definition is adapted from Klein et al. in (Klein et al., 2006; Klein et al., 2007); However, Klein et al. define

**Λ** is a set of lifelines covered by the **CF**.

**Λ** is a list of lifelines covered by the **CF**.

also a sequence of fragments just like **P**. This is the case with

also a sequence of fragments just like **C**. This is the case with

*CombinedFragment* (CF) or a MessageOccurrenceSpecification (MOS), such that,

nested CFs.

nested CFs.

**MOS**(Li,Ei,Mi) MOS Li is a lifeline covered by fi.

**MOS**(Li,Ei,Mi) MOS Li is a lifeline covered by ci,

**4.1.2 Aspect and primary models definition** 

essentially a list of ordered fragments.

fi = If instance of where

ci = If instance of where

MOSs), from the aspect's pointcut SD.

Note that a sequence is an ordered collection/list.

Let,

and,

MOS, such that,

**4.1.1 Formal notations for defining aspects and primary models** 

 **A** be a sequence of fragments from the aspect's Advice SD. **C** be a sequence of fragments from the primary model SD.

Fig. 4.1. Simplify Metamodel for Sequence Diagrams.

#### **4.1 Sequence Diagram (SD) Composition**

As previously described, our AOM approach has a primary model, one or more generic aspect models and a mark model. The primary model describes the core system functionality (behavior) without cross-cutting concerns. The generic aspect models describes (encapsulate) cross-cutting concerns which could otherwise be scattered across core functionality; for example, new features (in software product lines), security, persistence, etc. Before composing the primary model with an aspect model we first instantiate the generic aspect model in the context of the application with the help of a mark model. We employ an ATL transformation model that takes the primary, generic aspect, and mark models as input, and produces a context specific aspect model as output. We would like to point out that the mark model does not necessarily have to specify all the bindings for the template parameters in cases where some of the bindings can be matched or implied from the primary model. A second ATL transformation model then takes as input the primary and context specific models to produce the composed model. Defining a generic aspect improves re-usability since the same aspect can be instantiated and then composed with the primary model multiple times until a complete integrated system model is obtained. Since we are mainly interested in the behavioral view (of our primary and aspect models), our work is mainly focused on the composition of interactions diagrams in the form of SDs. As described earlier, the aspect model consists of a pointcut and an advice defined as SDs where the pointcut is the behavior to detect and the advice is the new behavior to compose or weave at the join points [Klein et al., 2007]. Before composing, we first have to identify all our join points by matching the pointcut SD with the primary model. The pointcut SD consists of message or a sequence of messages between lifelines; therefore, we want to find the occurrence of these sequences of messages in the primary model and then insert the defined cross-cutting behavior (defined in the advice SD) at every join point. Composition is essentially inserting this new behavior; that is, composition is achieved by replacing the join points with the advice SDs. Before instantiating a generic aspect, we first have to ensure that the aspect can be applied to the primary model; that is, whether its pointcut matches. A formal definition of matching will be given later. Also during composition we have to find where to weave. This makes pointcut detection or finding join points a core operation. The algorithm designed for pointcut detection manipulates the SD metaclasses by exploiting the relationship between *InteractionFragments* and their ordered list of fragments in an interaction. It also makes use of the fact that a sequence of messages (and indeed a SD) is essentially a list of ordered fragments.

## **4.1.1 Formal notations for defining aspects and primary models**

Let,

12 Semantics in Action – Applications and Scenarios

As previously described, our AOM approach has a primary model, one or more generic aspect models and a mark model. The primary model describes the core system functionality (behavior) without cross-cutting concerns. The generic aspect models describes (encapsulate) cross-cutting concerns which could otherwise be scattered across core functionality; for example, new features (in software product lines), security, persistence, etc. Before composing the primary model with an aspect model we first instantiate the generic aspect model in the context of the application with the help of a mark model. We employ an ATL transformation model that takes the primary, generic aspect, and mark models as input, and produces a context specific aspect model as output. We would like to point out that the mark model does not necessarily have to specify all the bindings for the template parameters in cases where some of the bindings can be matched or implied from the primary model. A second ATL transformation model then takes as input the primary and context specific models to produce the composed model. Defining a generic aspect improves re-usability since the same aspect can be instantiated and then composed with the primary model multiple times until a complete integrated system model is obtained. Since we are mainly interested in the behavioral view (of our primary and aspect models), our work is mainly focused on the composition of interactions diagrams in the form of SDs. As described earlier, the aspect model consists of a pointcut and an advice defined as SDs where the pointcut is the behavior to detect and the advice is the new behavior to compose or weave at the join points [Klein et al., 2007]. Before composing, we first have to identify all our join points by matching the pointcut SD with the primary model. The pointcut SD consists of message or a sequence of messages between lifelines; therefore, we want to find the occurrence of these sequences of messages in the primary model and then insert the defined cross-cutting behavior (defined in the advice SD) at every join point. Composition is essentially inserting this new behavior; that is, composition is achieved by replacing the join

Fig. 4.1. Simplify Metamodel for Sequence Diagrams.

**4.1 Sequence Diagram (SD) Composition** 


Note that a sequence is an ordered collection/list.

Then, **P** = Sequence{f1, ..., fψ} where ψ = number of fragments in **P** and fi is either a *CombinedFragment* (CF) or a MessageOccurrenceSpecification (MOS), such that,


and,

**C** = Sequence{c1, ..., cμ} where μ = number of fragments in **C** and ci is also either a CF or a MOS, such that,


### **4.1.2 Aspect and primary models definition**

Using the above notation, we will define an aspect model as a pair of fragment sequences, that is, **Aspect** = (**P**, **A**) where **P** and **A** are the sequences defined earlier. This definition is adapted from Klein et al. in (Klein et al., 2006; Klein et al., 2007); However, Klein et al. define

Using Model Transformation Language Semantics for Aspects Composition 15

The pseudo code for the algorithm that detects or matches pointcuts and returns **S** is given below. The algorithm begins by creating an empty sequence **S** on line 2. It then iterates over all fragments ci in **C** checking if ci is equal to f1, the first element in our pointcut **P** on line 9. If the elements are **not** equal, the algorithm moves to the next ci. However, if the fragments (ci and f1) are equal, it obtains **Ji**, a sub sequence of **C** starting from ci and with the same size

where ψ = number of fragments in **P**, and μ = number of fragments in **C** 

*//compute the location of the end of the potential join point* 

Ji = C->*subSequence*(i,k) // Potential join point */\* check if fragment at the same location in P is equal to the* 

*corresponding element in the join point \*/* 

**if** k > μ **then** *//make sure we have a valid location* 

**if** *pairWiseMatch*(P, Ji) **then**  S->enqueue(ci)

On line 13 the algorithm then compares **P** and **Ji**, side-by-side by checking if each pair of fragments at index *j* on both sequences is equal for 1 ≤ j ≥ ψ. If this is true, then indeed **Ji** is a join point. So the algorithm inserts the first element (ci) of the join point into **S** and loops back to line 3. It continues looping until it has checked all the elements of **C** or the condition on line 6 is true to ensure we do not fall off the edge. More details on the implementation of this algorithm and its functions, like *pairWiseMatch*, will be discussed in the next section.

If the algorithm-1 has to visit all fragments in **C** (when ψ = 1) then both functions on lines 10 and 13 will take constant time, that is, O(1) which makes the algorithm linear or O(n). If **P** is the same size as C (ψ = μ), then the algorithm has to loop only once but both *subSequence* and *pairWiseMatch* functions are O(n); hence, the algorithm is again linear. However, if ψ < μ then again both functions (i.e., *Sequence* and *pairWiseMatch* ) are, in the worst case, linear and the algorithm will have to loop several times each time invoking the two functions making

the algorithm quadratic, that is O(n2); therefore, in general the algorithm is O(n2).

**4.2.1 Pointcut detection algorithm** 

**Algorithm-1** Pointcut Detection Algorithm

**Output** : **S** = Sequence{s1, ... ,sn}

**begin** 

**end** 

**4.2.1.1 Algorithm-1 complexity** 

**Input** : **P** = Sequence{f1, ..., fψ}, **C** = Sequence{c1, ..., cμ}

k = i + ψ -1

**end if if** ci = f1 **then** 

**end if** 

 **end loop return** S

**break** 

**end if** 

S = Sequence{} **foreach** ci in C do

as **P**, on line 10.

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 20.

a simple SD as a tuple that consists of a set of lifelines, a set of events, a set of actions and partial ordering between the messages (Klein et al., 2007). This is different from our definition of a sequence of fragments. Using our definition, the primary model = **C**, a sequence of fragments from the primary model SD. Then our pointcut **P** matches **C** if and only if there exists two sub-sequences M1 and M2 in C such that, C = M1 P M2, where denotes a union of sequences. A B returns a sequence composed of all elements of A followed by the elements of B. If the **P** matches **C** several times, say *n* times, then we can say, C = M1 P M2 … Mn P Mn+1. This definition is an adaptation of the definition given by Klein et al. in (Kleinet al., 2006).

## **4.1.3 Join point definition**

Part of the sequence **C** that corresponds or matches **P** is the join point. In other words, a join point is a sub sequence of **C** that is equal to the sequence **P**. Equal here means that fragments at the same corresponding location in **P** and join point (same index on either sequences) are equal. For example, if elements at position 1 in **P** and in the join point are both MOSs, they can only be equal if and only if they cover similar lifelines (same name and type), have the same message, and have other features that are similar. More details for checking for equality will be given in the design and implementation section. Since the size (number of fragments) of **P**, hence the size of a join point, is fixed we can afford to keep track of only fragments at the beginning of each join point. With this assumption we can define; **S** = Sequence{s1, ... ,sn}, a sequence of fragments at the beginning of each join point where n > 1 is number of join points matched by **P**. A join point, Ji is then given by a sub sequence of **C** from index of si in **S** to the index of si plus ψ minus 1. That is, if; xi = **indexOf**(si) and yi = xi + ψ - 1, where ψ = number of elements in **P**, then, Ji = C- >**subSequence**(xi,yi) for 1 ≤ i ≥ n.

#### **4.2 Composition algorithms - assumptions and requirements**

The below algorithm and indeed the other algorithms to be introduced later, make the following assumptions:


## **4.2.1 Pointcut detection algorithm**

14 Semantics in Action – Applications and Scenarios

a simple SD as a tuple that consists of a set of lifelines, a set of events, a set of actions and partial ordering between the messages (Klein et al., 2007). This is different from our definition of a sequence of fragments. Using our definition, the primary model = **C**, a sequence of fragments from the primary model SD. Then our pointcut **P** matches **C** if and only if there exists two sub-sequences M1 and M2 in C such that, C = M1 P M2, where denotes a union of sequences. A B returns a sequence composed of all elements of A followed by the elements of B. If the **P** matches **C** several times, say *n* times, then we can say, C = M1 P M2 … Mn P Mn+1. This definition is an adaptation of the definition

Part of the sequence **C** that corresponds or matches **P** is the join point. In other words, a join point is a sub sequence of **C** that is equal to the sequence **P**. Equal here means that fragments at the same corresponding location in **P** and join point (same index on either sequences) are equal. For example, if elements at position 1 in **P** and in the join point are both MOSs, they can only be equal if and only if they cover similar lifelines (same name and type), have the same message, and have other features that are similar. More details for checking for equality will be given in the design and implementation section. Since the size (number of fragments) of **P**, hence the size of a join point, is fixed we can afford to keep track of only fragments at the beginning of each join point. With this assumption we can define; **S** = Sequence{s1, ... ,sn}, a sequence of fragments at the beginning of each join point where n > 1 is number of join points matched by **P**. A join point, Ji is then given by a sub sequence of **C** from index of si in **S** to the index of si plus ψ minus 1. That is, if; xi = **indexOf**(si) and yi = xi + ψ - 1, where ψ = number of elements in **P**, then, Ji = C-

The below algorithm and indeed the other algorithms to be introduced later, make the

 The input models are well formed and valid; hence, the sequences **S**, **P**, and **A** are valid. For example, we do not have empty sequences. We also assume that the aspect models (generic and context specific) consists of two interactions (SDs) named Advice and Pointcut, and that the primary model represents one interaction or scenario; therefore, consists of one instance of *Model*, one instance of *Collaboration*, and one instance of

We can correctly compare any two fragments regardless of their specialization, for

 We can ignore other fragments like *BehaviorExecutionSpecifications* (BESs) and *ExecutionOccurrenceSpecifications* (EOSs) focusing only on MOSs and CFs (and their

given by Klein et al. in (Kleinet al., 2006).

**4.1.3 Join point definition** 

>**subSequence**(xi,yi) for 1 ≤ i ≥ n.

example, comparing a MOS with a CF.

Nested CFs have been properly and consistently unrolled.

operands), and still achieve accurate pointcut detection.

 A lifeline's name is the same as that of the represented object (property). Message have arguments with primitive UML types (strings and integers).

following assumptions:

*Interaction*.

**4.2 Composition algorithms - assumptions and requirements** 

The pseudo code for the algorithm that detects or matches pointcuts and returns **S** is given below. The algorithm begins by creating an empty sequence **S** on line 2. It then iterates over all fragments ci in **C** checking if ci is equal to f1, the first element in our pointcut **P** on line 9. If the elements are **not** equal, the algorithm moves to the next ci. However, if the fragments (ci and f1) are equal, it obtains **Ji**, a sub sequence of **C** starting from ci and with the same size as **P**, on line 10.



On line 13 the algorithm then compares **P** and **Ji**, side-by-side by checking if each pair of fragments at index *j* on both sequences is equal for 1 ≤ j ≥ ψ. If this is true, then indeed **Ji** is a join point. So the algorithm inserts the first element (ci) of the join point into **S** and loops back to line 3. It continues looping until it has checked all the elements of **C** or the condition on line 6 is true to ensure we do not fall off the edge. More details on the implementation of this algorithm and its functions, like *pairWiseMatch*, will be discussed in the next section.

### **4.2.1.1 Algorithm-1 complexity**

If the algorithm-1 has to visit all fragments in **C** (when ψ = 1) then both functions on lines 10 and 13 will take constant time, that is, O(1) which makes the algorithm linear or O(n). If **P** is the same size as C (ψ = μ), then the algorithm has to loop only once but both *subSequence* and *pairWiseMatch* functions are O(n); hence, the algorithm is again linear. However, if ψ < μ then again both functions (i.e., *Sequence* and *pairWiseMatch* ) are, in the worst case, linear and the algorithm will have to loop several times each time invoking the two functions making the algorithm quadratic, that is O(n2); therefore, in general the algorithm is O(n2).

Using Model Transformation Language Semantics for Aspects Composition 17

The algorithm then checks the CM for more join points on line 10. If more are found, it returns to line 6 to instantiate the GAM using CM (new primary model). It then creates another CM and checks for more join points. The algorithm continues looping until no join

As it is, this algorithm has a potential nasty flaw in the form of positive feedback, which, if left unattended, can cause the algorithm to loop indefinitely in some cases! The problem is rooted on the fact that composing an aspect in most cases results in the addition of new model elements (fragments, messages and lifelines) which in turn can produce more join points. This means that after composition on line 8, the algorithm may find more join points on line 10 causing the algorithm to iterate again and again. For example, if the pointcut is defined as a single message MSG1, and the primary model has two invocations of this message, then we have two join points. If the advice adds three instances of the same message MSG1, then after composition (1st iteration) we will have four join points. After the second iteration well have six, then eight, etc. With the number of join points increasing all the time the algorithm will never terminate. This problem is easily solved by tagging model elements from advice during instantiation on line 6. To be precise we only have to tag MOSs (fragments). Then when pointcut matching during the invocation of *JoinPointsCount*  (implementing algorithm-1), we check for that tagging. If a potential join point has at least one tagged fragment, then we know that this join point emerged only after composition;

The complexity of algorithm-2 is difficult to analyze because on the surface the algorithm appears to be linear on the number of join points. However, the algorithm is not necessarily linear on the number fragments. We have already seen that detecting the number of join points is quadratic. Therefore, if that is nested within a loop, we could say that (in general)

At the core of the *Compose* function, used by the Complete Composition Algorithm described above, is the Advice Composition algorithm that weaves the advice at the join point. Recall the definition of an **Aspect** = (**P**, **A**). We will use definition again where by "**Aspect**" we are referring to a context specific aspect model. Our main interest is mainly on

**A** = Sequence{a1, ..., aω}, a sequence of fragments from the aspect model advice SD,

**C** = Sequence{c1, ..., cμ}, a sequence of fragments from the primary model, where μ =

**S** = Sequence{s1, ... ,sn}, a sequence of fragments at the beginning of each join point,

 A join point, **Ji** is then given by a sub sequence of C from index of si in **S** to the index of si + ψ-1; That is, If, xi = indexOf(si) and yi = xi + ψ - 1, then, **Ji** = C->subSequence(xi,yi)

points are found.

therefore, it is immediately disqualified.

**4.2.2.1 Algorithm-2 complexity** 

the algorithm is cubic, that is, O(n3)

the advice sequence **A**. Recall that,

number of fragments in **C**.

for 0 ≤ i ≥ n

where ω = number of fragments in **A.**

where n > 1 is number of join points matched by **P**.

**P** is a sequence of fragments from the pointcut SD.

**4.2.3 Advice composition algorithm** 

## **4.2.2 Complete composition algorithm**

After detecting the pointcut and obtaining our join points, the next step is to weave the advice at the join points. Since the advice has already been bound to the context of the application during aspect instantiation, weaving the advice is simplified to replacing a join point with the advice. This is trivial with only one join point. Challenges arise when we have multiple join points because we have only one advice from the aspect model. We can either duplicate the advice or work with one join point at a time. Both options were explored but duplicating the advice (without duplicating the aspect model) proved to be complex due to the inability to navigate target models in ATL, and the nested relationships between *InteractionFragments*. Focusing one join point at a time is easier and more elegant. The complete composition algorithm presented in this section achieves this. Let us first introduce abbreviations that we will use in the algorithm.


The pseudo code of the Complete Composition Algorithm is given below. The three functions defined in this algorithm represent the ATL transformations used to implement this algorithm as we shall see in the next chapter. The algorithm begins by retrieving *n* the number of join points matched in the primary model (PM) using the *JoinPointsCount* function on line 2. This function implements algorithm-1 (Pointcut detection Algorithm) to return a sequence of fragments at the beginning of each join point, and then finds the size of that sequence. The details of the implementation of this function will be given in next chapter. The number of join points determines if the algorithm will execute lines 6 to 10, and not necessarily the number of times the loop will iterate. If n > 0, that is, we have at least one join point, the algorithm instantiates the GAM to create a CSAM on line 6. This corresponds to the instantiate process shown in Figure 5.1. It then composes PM with CSAM by weaving the advice at the first join point using the *Compose* function on line 8 to produce our composed model.

## **Algorithm-2** Complete Composition Algorithm


The algorithm then checks the CM for more join points on line 10. If more are found, it returns to line 6 to instantiate the GAM using CM (new primary model). It then creates another CM and checks for more join points. The algorithm continues looping until no join points are found.

As it is, this algorithm has a potential nasty flaw in the form of positive feedback, which, if left unattended, can cause the algorithm to loop indefinitely in some cases! The problem is rooted on the fact that composing an aspect in most cases results in the addition of new model elements (fragments, messages and lifelines) which in turn can produce more join points. This means that after composition on line 8, the algorithm may find more join points on line 10 causing the algorithm to iterate again and again. For example, if the pointcut is defined as a single message MSG1, and the primary model has two invocations of this message, then we have two join points. If the advice adds three instances of the same message MSG1, then after composition (1st iteration) we will have four join points. After the second iteration well have six, then eight, etc. With the number of join points increasing all the time the algorithm will never terminate. This problem is easily solved by tagging model elements from advice during instantiation on line 6. To be precise we only have to tag MOSs (fragments). Then when pointcut matching during the invocation of *JoinPointsCount*  (implementing algorithm-1), we check for that tagging. If a potential join point has at least one tagged fragment, then we know that this join point emerged only after composition; therefore, it is immediately disqualified.

### **4.2.2.1 Algorithm-2 complexity**

16 Semantics in Action – Applications and Scenarios

After detecting the pointcut and obtaining our join points, the next step is to weave the advice at the join points. Since the advice has already been bound to the context of the application during aspect instantiation, weaving the advice is simplified to replacing a join point with the advice. This is trivial with only one join point. Challenges arise when we have multiple join points because we have only one advice from the aspect model. We can either duplicate the advice or work with one join point at a time. Both options were explored but duplicating the advice (without duplicating the aspect model) proved to be complex due to the inability to navigate target models in ATL, and the nested relationships between *InteractionFragments*. Focusing one join point at a time is easier and more elegant. The complete composition algorithm presented in this section achieves this. Let us first

**4.2.2 Complete composition algorithm** 

**GAM** = Generic Aspect Model

 **MM** = Mark Model **PM** = Primary Model **CM** = Composed Model

introduce abbreviations that we will use in the algorithm.

**CSAM** = Context Specific Aspect Model (Instantiated generic aspect model)

point using the *Compose* function on line 8 to produce our composed model.

n = *JoinPointsCount*(GAM, MM, PM)

CSAM = *Instantiate*(GAM, MM, temp)

n = *JoinPointsCount*(GAM, MM,temp)

**Algorithm-2** Complete Composition Algorithm

temp = PM

**while** n > 0

temp = CM

**end while return** CM

**Input** :GAM, MM, PM

**begin** 

**end** 

**Output** :CM

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.

The pseudo code of the Complete Composition Algorithm is given below. The three functions defined in this algorithm represent the ATL transformations used to implement this algorithm as we shall see in the next chapter. The algorithm begins by retrieving *n* the number of join points matched in the primary model (PM) using the *JoinPointsCount* function on line 2. This function implements algorithm-1 (Pointcut detection Algorithm) to return a sequence of fragments at the beginning of each join point, and then finds the size of that sequence. The details of the implementation of this function will be given in next chapter. The number of join points determines if the algorithm will execute lines 6 to 10, and not necessarily the number of times the loop will iterate. If n > 0, that is, we have at least one join point, the algorithm instantiates the GAM to create a CSAM on line 6. This corresponds to the instantiate process shown in Figure 5.1. It then composes PM with CSAM by weaving the advice at the first join

*// instantiate our generic aspect model* 

*// compose advice and first join point* CM = *Compose* (temp,CSAM)

The complexity of algorithm-2 is difficult to analyze because on the surface the algorithm appears to be linear on the number of join points. However, the algorithm is not necessarily linear on the number fragments. We have already seen that detecting the number of join points is quadratic. Therefore, if that is nested within a loop, we could say that (in general) the algorithm is cubic, that is, O(n3)

## **4.2.3 Advice composition algorithm**

At the core of the *Compose* function, used by the Complete Composition Algorithm described above, is the Advice Composition algorithm that weaves the advice at the join point. Recall the definition of an **Aspect** = (**P**, **A**). We will use definition again where by "**Aspect**" we are referring to a context specific aspect model. Our main interest is mainly on the advice sequence **A**. Recall that,


Using Model Transformation Language Semantics for Aspects Composition 19

In the previous section, we introduced our definition of primary, aspect and mark models. We also introduced our approach to AOM composition, and discussed our Complete Composition Algorithm that uses two other algorithms to detect join points, and compose the primary and aspect models. In this chapter we describe how the Complete Composition Algorithm was implemented using ATL transformations to realize the functions *JoinPointsCount(...)*, *Instantiate(...)* and *Compose(...)* employed by the algorithm. These functions were implemented as ATL transformation models and used to transform several input models to desired target models to achieve composition of SDs. Before giving the implementation details of these transformation models, we would like to first justify some of

 The use of ATL transformation models for composition instead of, say, graph transformations or general programming languages (e.g., Java). Aspect composition or weaving can be considered a form of model transformation because we take at least two input (primary and aspect) models and produce at least one target model (composed). Therefore, model transformation approaches can be used for aspect composition. ATL was chosen because it is mature and has a rich set of development tools that are built on top of flexible and versatile Eclipse platform. ATL is based on OCL; therefore, it is not difficult for a developer with some OCL experience to learn. ATL was also chosen because no work on behavioral aspect composition, that we are

 The use of RSA 7.5 as a modeling tool of choice. RSA 7.5 is not free but we already have a license for it. It is a great UML modeling tool. It has excellent support for SDs. It is easy and intuitive to use. It allows for easy model migration. We can export or import UML models as .uml or XMI files. It allows for model validation (not available in some of the tools) which we found very useful. RSA can also generate a sequence diagram from an imported model. This makes verification of our composed model easy and less

 The use of a mark model. ATL Transformations only work with models; therefore, our binding rules have to be in the form of a model that has a metamodel. Mark models are a convenient way to work with parameterized transformations. MDA, certainly, allows for use of mark models in model transformations (Happe et al., 2009). Happe et al. use mark models to annotate performance models in their work on performance

 Ignoring BehaviorExecutionSpecifications (BESs) and Execution Occurrence Specifications (EOSs) model elements during pointcut detection and composition. As stated in the previous section, we are convinced that we can ignore these two and still achieve accurate pointcut detection. This is because BESs and EOSs are used to define the duration of execution of behavior (OMG, 2009) of say, a message invocation. Our work is focused on detecting the occurrence of a sequence of messages (interactions between participants) and doing something when we find the sequence. We are not concerned about how long the participant will execute after a message invocation.

our design decisions and also describe how we designed our mark model.

Several key decisions were taken in this work. These include:

aware of, has been attempted using ATL.

completions (Happe et al., 2009).

**5. Design and Implementation** 

**5.1 Design decisions** 

error prone.

Since our Complete Composition Algorithm is concerned with one join point at a time, our Advice Composition algorithm needs to work with only one join point; that is, the join point that begins with s1 (the first element in **S**). Then, let **CCM** be a sequence of fragments from the composed model. Recall again that;

With the notation defined, we can now describe our Advice Composition algorithm. Its pseudo code is given on the next page. In a nut shell, the algorithm simply replaces the join point with the advice. The algorithm first checks if we have a join point. If so, it obtains the first element of **S**, on line 5. Using that element, the algorithm finds the location (x1) at the beginning and at the end (y1) of the join point, as shown on lines 6 and 7. The algorithm then obtains a sub sequence of fragments from **C** (primary model) before the start of the join point, on line 12. Note that indexing for our sequence data structure starts at 1 instead of zero as in Java lists or arrays. On line 16, the algorithm returns a sequence of fragments after the last element of the join point to the end of **C**. The composed model is then given by **CCM** = *sub\_before*  **A** *sub\_after*, that is, the union of *sub\_before*, **A** and *sub\_after*.


#### **4.2.3.1 Algorithm-3 complexity**

Creating *sub\_before* and *sub\_after* is linear in the worse case. Creating CCM is also O(n) in the worst case; hence, the above algorithm is clearly linear.

## **5. Design and Implementation**

18 Semantics in Action – Applications and Scenarios

Since our Complete Composition Algorithm is concerned with one join point at a time, our Advice Composition algorithm needs to work with only one join point; that is, the join point that begins with s1 (the first element in **S**). Then, let **CCM** be a sequence of fragments from the

With the notation defined, we can now describe our Advice Composition algorithm. Its pseudo code is given on the next page. In a nut shell, the algorithm simply replaces the join point with the advice. The algorithm first checks if we have a join point. If so, it obtains the first element of **S**, on line 5. Using that element, the algorithm finds the location (x1) at the beginning and at the end (y1) of the join point, as shown on lines 6 and 7. The algorithm then obtains a sub sequence of fragments from **C** (primary model) before the start of the join point, on line 12. Note that indexing for our sequence data structure starts at 1 instead of zero as in Java lists or arrays. On line 16, the algorithm returns a sequence of fragments after the last element of the join point to the end of **C**. The composed model is then given by **CCM**

s1 = S->first() *// fetch the tail of the 1st join point* 

y1 = x1 + ψ–1 *// find the location of the join point's head* 

*// get all fragments before the join point*  sub\_before = **C->**subSequence**(**1,x1-1**)** 

*// get all fragments after the join point*  sub\_after = C->subSequence(y1+1, μ)

x1 = C->indexOf(s1) *// find its location in C* 

*//Insert the advice in place of the join point*  **CCM = Sequence {**sub\_before**, A,** sub\_after**}** 

Creating *sub\_before* and *sub\_after* is linear in the worse case. Creating CCM is also O(n) in the

= *sub\_before*  **A** *sub\_after*, that is, the union of *sub\_before*, **A** and *sub\_after*.

**Input** : **C**, **A**, **P**, **S -** where ψ = number of fragments in **P**

**if** S->isEmpty() **then CCM = {}** 

> sub\_before = {} sub\_after = {} **if** x1 > 1 **then**

**end if if** y1 < μ **then** 

**end if** 

worst case; hence, the above algorithm is clearly linear.

composed model. Recall again that;

**Algorithm-3** Advice Composition

**else** 

**end if return CCM** 

**end** 

**4.2.3.1 Algorithm-3 complexity** 

**Output** : **CCM** 

**begin** 

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. In the previous section, we introduced our definition of primary, aspect and mark models. We also introduced our approach to AOM composition, and discussed our Complete Composition Algorithm that uses two other algorithms to detect join points, and compose the primary and aspect models. In this chapter we describe how the Complete Composition Algorithm was implemented using ATL transformations to realize the functions *JoinPointsCount(...)*, *Instantiate(...)* and *Compose(...)* employed by the algorithm. These functions were implemented as ATL transformation models and used to transform several input models to desired target models to achieve composition of SDs. Before giving the implementation details of these transformation models, we would like to first justify some of our design decisions and also describe how we designed our mark model.

## **5.1 Design decisions**

Several key decisions were taken in this work. These include:


Using Model Transformation Language Semantics for Aspects Composition 21

*JoinPointsCount* transformation is implemented in an ATL module creatively named

... The header declares that the transformation takes as input two UML2 models (bound to variables PRIMARY and ASPECT), a model that conforms to the BD (Binding Directive) metamodel, that is the mark model bound to the variable BIND. The transformation then produces a UML2 target model bound to the variable NUMJOINPOINT. The header also declares that the transformation uses the *PointcutMatchHelpers* ATL library. This is where common or general purpose helpers such as the ones used for pointcut detection (and used also by other transformations) are defined. This helps reduce code duplication and allows

**create** NUMJOINPOINT:UML2 **from** PRIMARY:UML2, ASPECT:UML2, BIND:BD;

**JoinPointsCount** as shown below by its header definition.

Fig. 5.1. An Overview of the *JoinPointsCount* ATL Transformation.

The transformation employs several helpers listed in Appendix A. It also uses some of the helpers defined in the *PointcutMatchHelpers* library listed in Appendix B. Please note that aspect model here refers to the generic aspect model (not context specific) which is one of

Rules are used to generate the target model in ATL. Our *JoinPointsCount* transformation has two simple declarative rules (one matched rule and one lazy rule) that generate a UML model to store the number of join points found. A proper UML model should have a *Model* container element that packages all the other modeling elements. The list of contained objects is then referenced by the *packagedElement* attribute or association. The *Model* matched

**module** JoinPointsCount;

**uses** PointcutMatchHelpers;

for a better code maintenance.

**5.3.2 JoinPointsCount helpers** 

**5.3.3 JoinPointsCount rules** 

the input models to the transformation.

During early stages of system modeling or at high levels of system abstraction, BESs and EOSs are not really applicable or useful; therefore, our decision to ignore them is reasonable.

## **5.2 Designing mark model metamodel**

A mark model helps define binding rules for instantiating generic aspect models. These rules are merely template parameter and value pairs stored in mark model instances. In MDE, a model must have a metamodel that it conforms to, and our mark model is no exception. Since the mark model is to be used in ATL transformations (with an EMF model handler), its metamodel must conform to a metametamodel that is the same as the ATL metamodel, that is, Ecore as shown in Figure 4.1 and Figure 4.2. ATL development tools include KM3 (Kernel MetaMetaModel) which is textual notation for defining metamodels (ATL\_Manual, 2009). The code snippet below shows a KM3 definition of the metamodel for our mark model which we named *BindingDirectives*. The metamodel has one class with a *parameter* and *binding* value attributes of type *String*. This essentially means that the instances of the mark model will be a collection of objects with initialized parameter and binding attributes.

```
package BindingDirectives {
```

```
 class BindingDirective { 
 attribute parameter : String; 
 attribute binding : String; 
 } 
 package PrimitiveTypes { 
 datatype String; 
 } 
}
```
The metamodel is defined in a *.km3* file which is then converted (injected) to an Ecore format encoded in XMI 2.0 using injectors in the ATL IDE (ATL\_Manual, 2009). Once the metamodel has been defined, we can begin creating mark models in an XMI format.

## **5.3 Implementation of the complete composition algorithm**

As mentioned earlier, the Complete Composition Algorithm uses the *JoinPointsCount*, *Instantiate*, and *Compose* transformations to produce a composed model (SD). These transformations in return implement the other two algorithms (Pointcut detection and Advice Composition algorithm) to achieve their objectives.

### **5.3.1 Getting the Number of Join Points**

The *JoinPointsCount* transformation is implemented by the ATL transformation model shown in Figure 5.1. It returns the number of join points found in the primary model given a pointcut defined in a generic aspect, and binding rules defined in a mark model. The transformation produces a simple UML target model that contains the number of join points found. The number of join points must be returned in a model because an ATL transformation (module) has to produce a model but not a string or integer. The

A mark model helps define binding rules for instantiating generic aspect models. These rules are merely template parameter and value pairs stored in mark model instances. In MDE, a model must have a metamodel that it conforms to, and our mark model is no exception. Since the mark model is to be used in ATL transformations (with an EMF model handler), its metamodel must conform to a metametamodel that is the same as the ATL metamodel, that is, Ecore as shown in Figure 4.1 and Figure 4.2. ATL development tools include KM3 (Kernel MetaMetaModel) which is textual notation for defining metamodels (ATL\_Manual, 2009). The code snippet below shows a KM3 definition of the metamodel for our mark model which we named *BindingDirectives*. The metamodel has one class with a *parameter* and *binding* value attributes of type *String*. This essentially means that the instances of the mark model will be a collection of objects with initialized parameter and

The metamodel is defined in a *.km3* file which is then converted (injected) to an Ecore format encoded in XMI 2.0 using injectors in the ATL IDE (ATL\_Manual, 2009). Once the

As mentioned earlier, the Complete Composition Algorithm uses the *JoinPointsCount*, *Instantiate*, and *Compose* transformations to produce a composed model (SD). These transformations in return implement the other two algorithms (Pointcut detection and

The *JoinPointsCount* transformation is implemented by the ATL transformation model shown in Figure 5.1. It returns the number of join points found in the primary model given a pointcut defined in a generic aspect, and binding rules defined in a mark model. The transformation produces a simple UML target model that contains the number of join points found. The number of join points must be returned in a model because an ATL transformation (module) has to produce a model but not a string or integer. The

metamodel has been defined, we can begin creating mark models in an XMI format.

**5.3 Implementation of the complete composition algorithm** 

Advice Composition algorithm) to achieve their objectives.

**5.3.1 Getting the Number of Join Points** 

reasonable.

binding attributes.

}

}

}

package BindingDirectives { class BindingDirective {

 package PrimitiveTypes { datatype String;

 attribute parameter : String; attribute binding : String;

**5.2 Designing mark model metamodel** 

During early stages of system modeling or at high levels of system abstraction, BESs and EOSs are not really applicable or useful; therefore, our decision to ignore them is *JoinPointsCount* transformation is implemented in an ATL module creatively named **JoinPointsCount** as shown below by its header definition.

**module** JoinPointsCount; **create** NUMJOINPOINT:UML2 **from** PRIMARY:UML2, ASPECT:UML2, BIND:BD; **uses** PointcutMatchHelpers;

The header declares that the transformation takes as input two UML2 models (bound to variables PRIMARY and ASPECT), a model that conforms to the BD (Binding Directive) metamodel, that is the mark model bound to the variable BIND. The transformation then produces a UML2 target model bound to the variable NUMJOINPOINT. The header also declares that the transformation uses the *PointcutMatchHelpers* ATL library. This is where common or general purpose helpers such as the ones used for pointcut detection (and used also by other transformations) are defined. This helps reduce code duplication and allows for a better code maintenance.

...

Fig. 5.1. An Overview of the *JoinPointsCount* ATL Transformation.

## **5.3.2 JoinPointsCount helpers**

The transformation employs several helpers listed in Appendix A. It also uses some of the helpers defined in the *PointcutMatchHelpers* library listed in Appendix B. Please note that aspect model here refers to the generic aspect model (not context specific) which is one of the input models to the transformation.

## **5.3.3 JoinPointsCount rules**

Rules are used to generate the target model in ATL. Our *JoinPointsCount* transformation has two simple declarative rules (one matched rule and one lazy rule) that generate a UML model to store the number of join points found. A proper UML model should have a *Model* container element that packages all the other modeling elements. The list of contained objects is then referenced by the *packagedElement* attribute or association. The *Model* matched

Using Model Transformation Language Semantics for Aspects Composition 23

metamodel (BD); that is, a mark model. The header also declares that the module imports

Just like the *JoinPointsCount*, this module also uses some of the helpers defined in the *PointcutMatchHelpers* library listed in Appendix B. This transformation also uses helpers defined within its module. Before we can generate the context specific aspect model, we have to ensure that we have a join point where we can weave. The *pointcutMatched* attribute helper returns true if we have at least one join point. It is used as a guard condition for all the rules that generate the target model elements as we shall see later. This ensures that no model element will be generated if there are no join points. The details of this helper are

The helper returns true if the sequence that contains all the fragments at the beginning of each join point (returned by *joinPointsFragments()*) is not empty. Since *pointcutMatched* is defined as an attribute helper, it is evaluated once and the result cached. This means that successive calls to the helper will be faster which improves performance especially in our

Several rules are required to generate a complete context specific aspect model. In fact, we have a rule for every model element type required for a well formed UML sequence diagram. These rules include several matched rules and a handful of lazy rules. Just like in the previous transformation, our target UML model should have a *Model* container element that packages all the other modeling elements. The rule that generates the target *Model*

Fig. 5.2. An overview of the *Instantiate* ATL Transformation.

the *PointcutMatchHelpers* library.

**helper def**: pointcutMatched : Boolean =

thisModule.joinPointsFragments()->notEmpty();

case where the helper is called many times by all the rules.

**5.5 Instantiate helpers** 

shown below.

**5.5.1 Instantiate rules** 

element is shown below.

rule is the main rule that generates a *Model* element. We want the rule to match only one element. Therefore, its source pattern defines that the rule should match an element of type **UML2 Model** from the input aspect model as it can be seen on line 2 in the code snippet for the rule below. The rule's target pattern defines that a **UML2 Model** element will be generated.


The attributes of the target elements will be initialized as defined from line 4. A *Model* element has a name and a collection of packaged elements. The *name* attribute is set to 'NumberOfJoinpoints'. The *packagedElement* attribute will be set to a sequence containing a **UML2 LiteralInteger** element generated by the invoked *CreateLiteralInteger* lazy rule. This lazy rule is passed the number of join points and a string (name) as parameters. The number of join points is, therefore, returned in a **UML2 LiteralInteger** model element packaged in a **UML2 Model** element. The code snippet for the *CreateLiteralInteger* lazy rule is shown below. The rule creates a *LiteralInteger* model element and initializes its name and value attributes with a string (desired name) and an integer (number of join points found by our transformation) respectively.

...


## **5.4 Instantiating A generic aspect model**

The *Instantiate* transformation is implemented by the ATL transformation shown in Figure 5.2. This transformation instantiates a generic aspect model and produce a context specific aspect model. It inputs a primary model, generic aspect model and a mark model, and outputs a context specific aspect model.

...

The *Instantiate* ATL module, whose header is shown below, implements the *Instantiate* transformation. The header declares that the module creates a target **UML2 model** bound to the CONTEXTSPECIFIC variable.

**module** Instantiate; **create** CONTEXTSPECIFIC : UML2 **from** PRIMARY : UML2, ASPECT : UML2, BIND : BD; **uses** PointcutMatchHelpers;

The module has two UML2 source models (bound to variables PRIMARY and ASPECT) and one source model bound to the variable BIND that conforms to our *Binding Directives*

...

Fig. 5.2. An overview of the *Instantiate* ATL Transformation.

metamodel (BD); that is, a mark model. The header also declares that the module imports the *PointcutMatchHelpers* library.

#### **5.5 Instantiate helpers**

22 Semantics in Action – Applications and Scenarios

rule is the main rule that generates a *Model* element. We want the rule to match only one element. Therefore, its source pattern defines that the rule should match an element of type **UML2 Model** from the input aspect model as it can be seen on line 2 in the code snippet for the rule below. The rule's target pattern defines that a **UML2 Model** element will be

... The attributes of the target elements will be initialized as defined from line 4. A *Model* element has a name and a collection of packaged elements. The *name* attribute is set to 'NumberOfJoinpoints'. The *packagedElement* attribute will be set to a sequence containing a **UML2 LiteralInteger** element generated by the invoked *CreateLiteralInteger* lazy rule. This lazy rule is passed the number of join points and a string (name) as parameters. The number of join points is, therefore, returned in a **UML2 LiteralInteger** model element packaged in a **UML2 Model** element. The code snippet for the *CreateLiteralInteger* lazy rule is shown below. The rule creates a *LiteralInteger* model element and initializes its name and value attributes with a string (desired name) and an integer (number of join points found by our

...

The *Instantiate* transformation is implemented by the ATL transformation shown in Figure 5.2. This transformation instantiates a generic aspect model and produce a context specific aspect model. It inputs a primary model, generic aspect model and a mark model, and

The *Instantiate* ATL module, whose header is shown below, implements the *Instantiate* transformation. The header declares that the module creates a target **UML2 model** bound to

**create** CONTEXTSPECIFIC : UML2 **from** PRIMARY : UML2, ASPECT : UML2, BIND : BD;

... The module has two UML2 source models (bound to variables PRIMARY and ASPECT) and one source model bound to the variable BIND that conforms to our *Binding Directives*

generated.

**rule** Model {

**to** t : UML2!Model (

transformation) respectively.

**lazy rule** CreateLiteralInteger {  **from** count : Integer, name :String **to** t: UML2!LiteralInteger ( name <- name, value <- count

**5.4 Instantiating A generic aspect model** 

outputs a context specific aspect model.

the CONTEXTSPECIFIC variable.

**uses** PointcutMatchHelpers;

**module** Instantiate;

name <- 'NumberOfJoinpoints',

'NumberOfJoinpoints'),

**from** s : UML2!Model(s.fromAspectModel() )

packagedElement <- Sequence {

thisModule.CreateLiteralInteger(thisModule.numJoinPoints,

1. 2. 3. 4. 5. 6. 7. 8.

1. 2. 3. 4. 5. 6. Just like the *JoinPointsCount*, this module also uses some of the helpers defined in the *PointcutMatchHelpers* library listed in Appendix B. This transformation also uses helpers defined within its module. Before we can generate the context specific aspect model, we have to ensure that we have a join point where we can weave. The *pointcutMatched* attribute helper returns true if we have at least one join point. It is used as a guard condition for all the rules that generate the target model elements as we shall see later. This ensures that no model element will be generated if there are no join points. The details of this helper are shown below.

**helper def**: pointcutMatched : Boolean = thisModule.joinPointsFragments()->notEmpty();

The helper returns true if the sequence that contains all the fragments at the beginning of each join point (returned by *joinPointsFragments()*) is not empty. Since *pointcutMatched* is defined as an attribute helper, it is evaluated once and the result cached. This means that successive calls to the helper will be faster which improves performance especially in our case where the helper is called many times by all the rules.

#### **5.5.1 Instantiate rules**

Several rules are required to generate a complete context specific aspect model. In fact, we have a rule for every model element type required for a well formed UML sequence diagram. These rules include several matched rules and a handful of lazy rules. Just like in the previous transformation, our target UML model should have a *Model* container element that packages all the other modeling elements. The rule that generates the target *Model* element is shown below.

Using Model Transformation Language Semantics for Aspects Composition 25

rule ensures that the rule matches the interaction from the aspect's advice which has the

The guard for the *pInteractions* rule matches the interaction from the aspect's pointcut which has the name "Pointcut". Both rules then initialize the attributes of the generated interactions using the values from the attributes of the matched source elements as it can be

The *Lifelines* rule generates lifelines for both the advice and pointcut SDs. The rule matches all lifelines from the advice model as shown on line 3 of rule's code snippet on the next page. The *aLifelines* helper returns all lifelines from the generic aspect model (advice and pointcut SDs). The generated lifeline's attributes are then initialized as shown from lines 6 to 8. This rule probably best shows how helpers are used to assist rules in creating the target models other than been used as guard conditions in the source pattern. We can see on line 6 that binding is achieved by using the *bind()* helper to initialize the name of the generated lifeline.

thisModule.aLifelines->includes(s)**and** thisModule.pointcutMatched

After obtaining a context specific aspect model from the previous transformation (*Instantiate*), the next step is to compose the context specific aspect model with the primary model. This is achieved by the *Compose* ATL transformation whose overview is shown in

coveredBy <- thisModule.getMOSByLifeline(s),

s.name = 'Advice' **and** thisModule.pointcutMatched

name "Advice".

)

)

seen from lines 7 to line 15.

**rule** Lifelines {

)

}

)

name <- s.bind(),

**5.6 Composing aspect models** 

**from** s : UML2!Lifeline (

**to** targetLifeline : UML2!Lifeline (

represents <- s.represents

}

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. **rule** aInteractions {

**from** s : UML2!Interaction (

 **to** t : UML2!Interaction ( name <- s.name,

covered <- s.covered

 lifeline <- s.lifeline, fragment <- s.fragment, message <- s.message,

 ownedAttribute <- s.ownedAttribute, ownedConnector <- s.ownedConnector, generalOrdering <- s.generalOrdering,

ownedBehavior <- s.ownedBehavior,

1. 2. 3. 4. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.

```
1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 
  rule Model { 
   from s : UML2!Model ( 
   s.fromAspectModel() and thisModule.pointcutMatched 
   ) 
   to t : UML2!Model ( 
   name <- s.createAspectName(), 
   packagedElement <- s.packagedElement 
   ) 
  }
```
The source pattern specifies that the rule matches elements of type **UML2 Model**. It also has a condition that the matched element should come from the aspect model (using *fromAspectModel()* helper), and also that *pointcutMatched* must be true, as mentioned earlier. There is only one *Model* element from the aspect model. If at least one join point was found, then only one **UML2 Model** element will be created on the target model since the target pattern declares that the rule creates an instance of **UML2 Model**. Its packaged elements will be initialized to the list from the matched element as defined on line 7 above. The name will be initialized with the string returned by the *createAspectName()* helper on line 6 above. The UML specification describes that an *Interaction* can be contained in a *Collaboration.*  Collaborations are used to show the structure of cooperating elements with a particular purpose (OMG, 2009). Indeed, the primary and aspect models created using RSA have interactions contained within collaborations. The *Collaborations* matched rule has the task of generating *Collaboration* objects that enclose the interactions for the advice and pointcut. Recall that the aspect model consists of the advice and pointcut SDs (interactions). The rule is described by the code snippet shown below.

```
1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 
10. 
11. 
12. 
    rule Collaborations { 
     from s : UML2!Collaboration ( 
     thisModule.aCollaborations->includes(s) and 
                   thisModule.pointcutMatched 
     ) 
     to t : UML2!Collaboration ( 
     name <- s.name, 
     ownedBehavior <- s.ownedBehavior, 
     ownedAttribute <- s.ownedAttribute, 
     ownedConnector <- s.ownedConnector 
     ) 
    }
```
The guard condition for this rule's source pattern ensures that only collaborations from the aspect model (and not from primary model) are matched. It checks if a collaboration is included in the collection of collaborations from the aspect model returned by the *aCollaborations* attribute helper. The attributes of the generated collaboration, including the enclosed interactions (*ownedBehavior*), are initialized from those of the matched collaboration as shown on lines 7 to 10 above. The *aInteractions* and *pInteractions* rules are used to create *Interaction* target elements for the advice and pointcut respectively. The rules are almost identical with slight differences in the source pattern guard. The code below gives details of the *aInteractions* rule. The difference between the rules is in line 3. The guard for *aInteractions*

The source pattern specifies that the rule matches elements of type **UML2 Model**. It also has a condition that the matched element should come from the aspect model (using *fromAspectModel()* helper), and also that *pointcutMatched* must be true, as mentioned earlier. There is only one *Model* element from the aspect model. If at least one join point was found, then only one **UML2 Model** element will be created on the target model since the target pattern declares that the rule creates an instance of **UML2 Model**. Its packaged elements will be initialized to the list from the matched element as defined on line 7 above. The name will be initialized with the string returned by the *createAspectName()* helper on line 6 above. The UML specification describes that an *Interaction* can be contained in a *Collaboration.*  Collaborations are used to show the structure of cooperating elements with a particular purpose (OMG, 2009). Indeed, the primary and aspect models created using RSA have interactions contained within collaborations. The *Collaborations* matched rule has the task of generating *Collaboration* objects that enclose the interactions for the advice and pointcut. Recall that the aspect model consists of the advice and pointcut SDs (interactions). The rule

s.fromAspectModel() **and** thisModule.pointcutMatched

1. 2. 3. 4. 5. 6. 7. 8. 9. **rule** Model {

)

)

}

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.

**from** s : UML2!Model (

**to** t : UML2!Model (

name <- s.createAspectName(),

is described by the code snippet shown below.

 thisModule.aCollaborations->includes(s) **and** thisModule.pointcutMatched

The guard condition for this rule's source pattern ensures that only collaborations from the aspect model (and not from primary model) are matched. It checks if a collaboration is included in the collection of collaborations from the aspect model returned by the *aCollaborations* attribute helper. The attributes of the generated collaboration, including the enclosed interactions (*ownedBehavior*), are initialized from those of the matched collaboration as shown on lines 7 to 10 above. The *aInteractions* and *pInteractions* rules are used to create *Interaction* target elements for the advice and pointcut respectively. The rules are almost identical with slight differences in the source pattern guard. The code below gives details of the *aInteractions* rule. The difference between the rules is in line 3. The guard for *aInteractions*

ownedBehavior <- s.ownedBehavior,

 ownedAttribute <- s.ownedAttribute, ownedConnector <- s.ownedConnector

**from** s : UML2!Collaboration (

 **to** t : UML2!Collaboration ( name <- s.name,

**rule** Collaborations {

)

)

}

packagedElement <- s.packagedElement

rule ensures that the rule matches the interaction from the aspect's advice which has the name "Advice".

```
1. 
2. 
3. 
4. 
6. 
7. 
8. 
9. 
10. 
11. 
12. 
13. 
14. 
15. 
16. 
17. 
   rule aInteractions { 
    from s : UML2!Interaction ( 
    s.name = 'Advice' and thisModule.pointcutMatched 
    ) 
    to t : UML2!Interaction ( 
    name <- s.name, 
    lifeline <- s.lifeline, 
    fragment <- s.fragment, 
    message <- s.message, 
    ownedAttribute <- s.ownedAttribute, 
    ownedConnector <- s.ownedConnector, 
    generalOrdering <- s.generalOrdering, 
    ownedBehavior <- s.ownedBehavior, 
    covered <- s.covered 
    ) 
   }
```
The guard for the *pInteractions* rule matches the interaction from the aspect's pointcut which has the name "Pointcut". Both rules then initialize the attributes of the generated interactions using the values from the attributes of the matched source elements as it can be seen from lines 7 to line 15.

The *Lifelines* rule generates lifelines for both the advice and pointcut SDs. The rule matches all lifelines from the advice model as shown on line 3 of rule's code snippet on the next page. The *aLifelines* helper returns all lifelines from the generic aspect model (advice and pointcut SDs). The generated lifeline's attributes are then initialized as shown from lines 6 to 8. This rule probably best shows how helpers are used to assist rules in creating the target models other than been used as guard conditions in the source pattern. We can see on line 6 that binding is achieved by using the *bind()* helper to initialize the name of the generated lifeline.

```
1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 
10. 
   rule Lifelines { 
    from s : UML2!Lifeline ( 
    thisModule.aLifelines->includes(s)and thisModule.pointcutMatched 
           ) 
    to targetLifeline : UML2!Lifeline ( 
    name <- s.bind(), 
    coveredBy <- thisModule.getMOSByLifeline(s), 
    represents <- s.represents 
    ) 
   }
```
### **5.6 Composing aspect models**

After obtaining a context specific aspect model from the previous transformation (*Instantiate*), the next step is to compose the context specific aspect model with the primary model. This is achieved by the *Compose* ATL transformation whose overview is shown in

Using Model Transformation Language Semantics for Aspects Composition 27

Several rules are defined for creating the composed target model. Rules in the *Compose* transformation probably use more helpers compared to the two previous transformations, mainly because in this transformation more elements are removed or added. This requires modifications to many associations between model elements. The code below is that for the

name <- thisModule.getModelName(s.name, thisModule.aModel),

The rule matches elements of type **UML2 Model** from the primary model, and provided the pointcut matches as defined by the source pattern on lines 2 and 3. The rule creates instances of *Model* as declared on line 5. Since the primary model consists of one instance of the *Model* class, this rule will generate only one instance. It then initializes the created instance with the use of several helpers as defined on lines 6 to 11. The *getModelName* helper generates a string used to initialize the name attribute. The *packageElement* list attribute is initialized to a sequence of classes returned by *targetClasses*, a collaboration from the primary returned by *pCollaborations*, and a sequence of events returned by the *getTargetEvents()* operation helper. These three helpers are defined in the context of the module; hence, the use of the keyword

*Model* matched rule which is used to create the **UML2 Model** element.

s.fromPrimaryModel() **and** thisModule.pointcutMatched

thisModule.getTargetEvents()

 packagedElement <- Sequence { thisModule.targetClasses, thisModule.pCollaborations,

thisModule.targetMessages->includes(s) **and**

**if** e.oclIsTypeOf(UML2!LiteralString) **then** 

**if** e.oclIsTypeOf(UML2!LiteralInteger) **then** 

thisModule.CreateLI(e)

thisModule.CreateLS(e)

OclUndefined

**else** 

**endif** 

thisModule.pointcutMatched

sendEvent <- s.sendEvent, receiveEvent <- s.receiveEvent, messageSort <- s.messageSort, argument <- s.argument->collect (e |

**5.6.2 Compose rules** 

**rule** Model {

)

...

*thisModule*.

**rule** Messages{

)

...

**from** s : **UML2!Message** (

**to** t : **UML2!Message** (

name <- s.name,

**else** 

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. **from** s : UML2!Model (

**to** t : UML2!Model (

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.

Figure 3.5. The transformation inputs the primary and context specific source models, and produces a composed target model. Both the source models and the output model conform to the UML2 metamodel. This transformation is implemented by the *Compose* ATL module. The code snippet below shows a description of the module's header.

**module** Compose; **create** COMPOSED : UML2 from PRIMARY : UML2, ASPECT : UML2; **uses** PointcutMatchHelpers;

As expected, the header declares that the module creates a UML2 Model bound to the variable COMPOSED from two UML2 source models bound to the variables PRIMARY and ASPECT for the primary and aspect models respectively. The module also uses some helpers from the *PointcutMatchHelpers* library.

...

## **5.6.1 Compose helpers**

Our *Compose* transformation has a number of helpers. All the helpers have a necessary role to play but some roles are, certainly, more important than others. For example, the *getTargetFragments* attribute helper has the privilege of returning the composed sequence of fragments, that is, *sequence {sub\_before, A, sub\_after}* from algorithm-3. The code definition of this helper is shown below. The helper begins by ensuring that there is, at least, one join point by calling the *pointcutMatched* helper on line 2, which we have described earlier. If there exists a join point, *getTargetFragments* then obtains a sequence of fragments before the join point by invoking the *lowerFragSub* helper on line 4, a sequence of fragments from the advice (by invoking *getAspectFragments* ) on line 5, and a sequence fragments after the join point on line 6. It returns a flattened sequence consisting of those sequences. The fragments returned by *getTargetFragments* are used to initialize the fragment attribute of the interaction generated by our transformation.

1. **helper def** : getTargetFragments : Sequence(UML2!InteractionFragment) =


The *getTargetFragments* helper also serves as the base of our composition process. Almost all the other elements to be used in generating the target model are rooted from this helper. The *targetCFs* helper, which returns all combined fragments to be used for generating combined fragments in the target model, iterates through fragments returned by *getTargetFragments* returning all instances of *CombinedFragment*. The *targetOperands* helper, in return, iterates through the sequence of combined fragments returned by *targetCFs* to obtain all instances of *InteractionOperand*.

## **5.6.2 Compose rules**

26 Semantics in Action – Applications and Scenarios

Figure 3.5. The transformation inputs the primary and context specific source models, and produces a composed target model. Both the source models and the output model conform to the UML2 metamodel. This transformation is implemented by the *Compose* ATL module.

... As expected, the header declares that the module creates a UML2 Model bound to the variable COMPOSED from two UML2 source models bound to the variables PRIMARY and ASPECT for the primary and aspect models respectively. The module also uses some

Our *Compose* transformation has a number of helpers. All the helpers have a necessary role to play but some roles are, certainly, more important than others. For example, the *getTargetFragments* attribute helper has the privilege of returning the composed sequence of fragments, that is, *sequence {sub\_before, A, sub\_after}* from algorithm-3. The code definition of this helper is shown below. The helper begins by ensuring that there is, at least, one join point by calling the *pointcutMatched* helper on line 2, which we have described earlier. If there exists a join point, *getTargetFragments* then obtains a sequence of fragments before the join point by invoking the *lowerFragSub* helper on line 4, a sequence of fragments from the advice (by invoking *getAspectFragments* ) on line 5, and a sequence fragments after the join point on line 6. It returns a flattened sequence consisting of those sequences. The fragments returned by *getTargetFragments* are used to initialize the fragment attribute of the interaction

**helper def** : getTargetFragments : Sequence(UML2!InteractionFragment) =

thisModule.getAspectFragments('Advice'),

thisModule.numPCTFs-1)

thisModule.lowerFragSub(thisModule.firstJPIndex),

thisModule.upperFragSub(thisModule.firstJPIndex +

The *getTargetFragments* helper also serves as the base of our composition process. Almost all the other elements to be used in generating the target model are rooted from this helper. The *targetCFs* helper, which returns all combined fragments to be used for generating combined fragments in the target model, iterates through fragments returned by *getTargetFragments* returning all instances of *CombinedFragment*. The *targetOperands* helper, in return, iterates through the sequence of combined fragments returned by *targetCFs* to obtain all instances of

 **if** thisModule.pointcutMatched **then** 

The code snippet below shows a description of the module's header.

**create** COMPOSED : UML2 from PRIMARY : UML2, ASPECT : UML2;

**module** Compose;

**uses** PointcutMatchHelpers;

**5.6.1 Compose helpers** 

generated by our transformation.

**else**  Sequence{} **endif**;

*InteractionOperand*.

Sequence {

}->flatten()->asSequence()

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.

helpers from the *PointcutMatchHelpers* library.

Several rules are defined for creating the composed target model. Rules in the *Compose* transformation probably use more helpers compared to the two previous transformations, mainly because in this transformation more elements are removed or added. This requires modifications to many associations between model elements. The code below is that for the *Model* matched rule which is used to create the **UML2 Model** element.

```
1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 
10. 
11. 
   rule Model { 
    from s : UML2!Model ( 
    s.fromPrimaryModel() and thisModule.pointcutMatched 
    ) 
    to t : UML2!Model ( 
             name <- thisModule.getModelName(s.name, thisModule.aModel), 
             packagedElement <- Sequence { 
             thisModule.targetClasses, 
             thisModule.pCollaborations, 
             thisModule.getTargetEvents() 
   ...
```
The rule matches elements of type **UML2 Model** from the primary model, and provided the pointcut matches as defined by the source pattern on lines 2 and 3. The rule creates instances of *Model* as declared on line 5. Since the primary model consists of one instance of the *Model* class, this rule will generate only one instance. It then initializes the created instance with the use of several helpers as defined on lines 6 to 11. The *getModelName* helper generates a string used to initialize the name attribute. The *packageElement* list attribute is initialized to a sequence of classes returned by *targetClasses*, a collaboration from the primary returned by *pCollaborations*, and a sequence of events returned by the *getTargetEvents()* operation helper. These three helpers are defined in the context of the module; hence, the use of the keyword *thisModule*.

```
1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 
10. 
11. 
12. 
13. 
14. 
15. 
16. 
17. 
18. 
19. 
20. 
  rule Messages{ 
   from s : UML2!Message ( 
         thisModule.targetMessages->includes(s) and 
         thisModule.pointcutMatched 
   ) 
   to t : UML2!Message ( 
         name <- s.name, 
         sendEvent <- s.sendEvent, 
         receiveEvent <- s.receiveEvent, 
         messageSort <- s.messageSort, 
         argument <- s.argument->collect (e | 
   if e.oclIsTypeOf(UML2!LiteralString) then 
   thisModule.CreateLS(e) 
   else 
          if e.oclIsTypeOf(UML2!LiteralInteger) then 
          thisModule.CreateLI(e) 
          else 
   OclUndefined 
   endif 
  ...
```
Using Model Transformation Language Semantics for Aspects Composition 29

Fig. 6.1. Receive a Call Primary Model Adapted from (Whittle et al., 2007).

 <BindingDirective parameter="|receiver" binding="user"/> <BindingDirective parameter="|sender" binding="caller"/>

<xmi:XMI xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI"

 <BindingDirective parameter="|notify" binding="displayCallInfo"/> <BindingDirective parameter="|acknowledge" binding="ok"/> <BindingDirective parameter="|accept" binding="pickUp"/>

 <BindingDirective parameter="|end" binding="hangUp"/> <BindingDirective parameter="|suspend" binding="putOnHold"/> <BindingDirective parameter="|Client" binding="User"/> <BindingDirective parameter="|notifier" binding="display"/>

<BindingDirective parameter="|Transducer" binding="Display"/>

<BindingDirective parameter="|anotherRequest" binding="incomingCall"/>

<?xml version="1.0" encoding="ISO-8859-1"?>

xmlns="BindingDirectives">

</xmi:XMI>

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.

The *Messages* rule shown below is used to generate messages for the target model. This rule is more interesting since it calls a few lazy rules to help initialize some of the attributes of the target messages to be created. The rule matches all messages included in *targetMessages* and creates messages for the target model. The generated message is initialized as defined from line 7. On line 12, the *argument* attribute is initialized by calling a suitable lazy rule. We are only interested in primitive type message arguments (integers and strings); therefore, we have two lazy rules for creating an instance of *LiteralInteger* or *LiteralString* depending on the argument type for the matched message.

The rule uses ATL's built-in *oclIsTypeOf(t: oclType)* operation to check the type of the argument for the matched message. If it is a *LiteralString* then the *CreateLS* lazy rule is called but if it is a *LiteralInteger* the *CreateLI* lazy rule is called instead. If the argument is neither an integer nor a string, the message's argument attribute is initialized to *OclUndefined*, ATL's equivalent of null. All the rules that are used by the Compose transformation to generate the composed model are listed in Appendix C.

## **6. Case studies - phone call features as aspects**

This case study of a cell phone application was adapted from Whittle and Jayaraman in (Whittle et al., 2007). The application has three use cases but we are only interested in two; namely, *Receive a Call* and *Notify Call Waiting* (Whittle et al., 2007]. The *Receive a Call* use case is considered to be the base model and the *Notify Call Waiting* is considered the aspect. Figure 6.1 shows a dynamic view of the *Receive a Call* use case modeled as a sequence diagram. This will be our primary model. When the user's phone receives a call (*incomingCall* message), it alerts the user by displaying the appropriate information about the caller on the phone's display (Whittle et al., 2007) by sending a *displayCallInfo* message to the display. The phone also sends a *ring* message to the ringer. The user then has several options captured by an **alt** combined fragment. The user can accept the call by sending a *pickUp* message to the phone and later end the call by sending a *hangUp* message. Alternatively, if the user chooses not to accept the call, the user can send a *disconnect* message to the phone. If the user elects to ignore the call, the phone will ring for a specified amount of time and then time out ending the scenario. As mentioned, the *Notify Call Waiting* scenario or feature is considered an aspect. The approach (graph transformations) taken by Whittle and Jayaraman (Whittle & Jayaraman, 2006) does not have the notion of generic or context specific models like our approach. Therefore, Figure 6.2 shows our representation of the behavioral model of the *Notify Call Waiting* scenario as a generic aspect model.

The pointcut is defined as a sequence of parameterized |accept and |end messages from the receiver lifeline to the phone lifeline. This will match a sequence of two messages that will be bound to |accept and |end from the lifeline bound to |receiver. The advice shown in Figure 5.2b is slightly more complex. It introduces messages that if bound properly, will place the current call on hold (Whittle et al., 2007). The behavior defined by the advice is only applicable when the user is currently on call; therefore, we must ensure that the advice is weaved between the *pickUp* and *hangUp* messages on the primary model (Whittle et al., 2007). To achieve this, we will bind |accept and |end to *pickUp* and *hangUp* respectively, as shown on lines 9 and 10 of the mark model below. The |receiver parameter is bound to *user* so the pointcut matches *pickUp* and *hangUp* from the user to the phone.

The *Messages* rule shown below is used to generate messages for the target model. This rule is more interesting since it calls a few lazy rules to help initialize some of the attributes of the target messages to be created. The rule matches all messages included in *targetMessages* and creates messages for the target model. The generated message is initialized as defined from line 7. On line 12, the *argument* attribute is initialized by calling a suitable lazy rule. We are only interested in primitive type message arguments (integers and strings); therefore, we have two lazy rules for creating an instance of *LiteralInteger* or *LiteralString* depending on the

The rule uses ATL's built-in *oclIsTypeOf(t: oclType)* operation to check the type of the argument for the matched message. If it is a *LiteralString* then the *CreateLS* lazy rule is called but if it is a *LiteralInteger* the *CreateLI* lazy rule is called instead. If the argument is neither an integer nor a string, the message's argument attribute is initialized to *OclUndefined*, ATL's equivalent of null. All the rules that are used by the Compose transformation to generate the

This case study of a cell phone application was adapted from Whittle and Jayaraman in (Whittle et al., 2007). The application has three use cases but we are only interested in two; namely, *Receive a Call* and *Notify Call Waiting* (Whittle et al., 2007]. The *Receive a Call* use case is considered to be the base model and the *Notify Call Waiting* is considered the aspect. Figure 6.1 shows a dynamic view of the *Receive a Call* use case modeled as a sequence diagram. This will be our primary model. When the user's phone receives a call (*incomingCall* message), it alerts the user by displaying the appropriate information about the caller on the phone's display (Whittle et al., 2007) by sending a *displayCallInfo* message to the display. The phone also sends a *ring* message to the ringer. The user then has several options captured by an **alt** combined fragment. The user can accept the call by sending a *pickUp* message to the phone and later end the call by sending a *hangUp* message. Alternatively, if the user chooses not to accept the call, the user can send a *disconnect* message to the phone. If the user elects to ignore the call, the phone will ring for a specified amount of time and then time out ending the scenario. As mentioned, the *Notify Call Waiting* scenario or feature is considered an aspect. The approach (graph transformations) taken by Whittle and Jayaraman (Whittle & Jayaraman, 2006) does not have the notion of generic or context specific models like our approach. Therefore, Figure 6.2 shows our representation of

the behavioral model of the *Notify Call Waiting* scenario as a generic aspect model.

so the pointcut matches *pickUp* and *hangUp* from the user to the phone.

The pointcut is defined as a sequence of parameterized |accept and |end messages from the receiver lifeline to the phone lifeline. This will match a sequence of two messages that will be bound to |accept and |end from the lifeline bound to |receiver. The advice shown in Figure 5.2b is slightly more complex. It introduces messages that if bound properly, will place the current call on hold (Whittle et al., 2007). The behavior defined by the advice is only applicable when the user is currently on call; therefore, we must ensure that the advice is weaved between the *pickUp* and *hangUp* messages on the primary model (Whittle et al., 2007). To achieve this, we will bind |accept and |end to *pickUp* and *hangUp* respectively, as shown on lines 9 and 10 of the mark model below. The |receiver parameter is bound to *user*

argument type for the matched message.

composed model are listed in Appendix C.

**6. Case studies - phone call features as aspects** 

Fig. 6.1. Receive a Call Primary Model Adapted from (Whittle et al., 2007).

```
1. 
   <?xml version="1.0" encoding="ISO-8859-1"?>
```

```
2. 
   <xmi:XMI xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI"
```

Using Model Transformation Language Semantics for Aspects Composition 31

With only one join point, our composition algorithm only has to loop once; hence, no loop unrolling is required. The next step is to validate our models using both RSA and our custom *validator* package. Running the *ValidateComposedModel* class from our custom

package to validate the composed model (CM) produces the output shown below.

Reading model CM from disk ...Validating CM

Collaboration model element ReceiveCall is valid :) All Owned attributes in ReceiveCall are valid :) Interaction model element ReceiveCall is valid :)

All Message Occurrence Specifications in ReceiveCall are valid :)

Our model and all its model elements meet our validation requirements

(a) Pointcut (b) Advice

Model container is valid All Classes are valid :)

All Class Operations are valid :) All Message events are valid :)

All Messages in ReceiveCall are valid :) All lifelines in ReceiveCall are valid :)

Fig. 6.4. Context Specific Aspect Model.

Fig. 6.2. Notify Call Waiting Generic Aspect Model.

The rest of the parameters are bound as defined by the mark model. Recall that by binding these parameters from the generic aspect model, we are actually instantiating it to produce a context specific aspect model. This is done by the *Instantiate* ATL transformation as discussed in earlier sections. However, before going into the trouble of instantiating a context specific aspect (and composing it with the primary model), we must first determine if the pointcut matches, and if so, how much join points were found. Getting the number of join points is performed by the *JoinPointsCount* transformation which takes the generic aspect, the primary and mark models as input, and produces a target model that contains the number of join points. Executing this transformation produces the model shown in Figure 6.3 (when viewed on Eclipse's uml editor). The model contains a *LiteralInteger* object with the name *NumberOfJoinpoints* and which has a value of one, that is, our primary model has one join point.

Fig. 6.3. JoinPointsCount Output Model.

The rest of the parameters are bound as defined by the mark model. Recall that by binding these parameters from the generic aspect model, we are actually instantiating it to produce a context specific aspect model. This is done by the *Instantiate* ATL transformation as discussed in earlier sections. However, before going into the trouble of instantiating a context specific aspect (and composing it with the primary model), we must first determine if the pointcut matches, and if so, how much join points were found. Getting the number of join points is performed by the *JoinPointsCount* transformation which takes the generic aspect, the primary and mark models as input, and produces a target model that contains the number of join points. Executing this transformation produces the model shown in Figure 6.3 (when viewed on Eclipse's uml editor). The model contains a *LiteralInteger* object with the name *NumberOfJoinpoints* and which has a value of one, that is, our primary model

(a) Pointcut (b) Advice

Fig. 6.2. Notify Call Waiting Generic Aspect Model.

has one join point.

Fig. 6.3. JoinPointsCount Output Model.

With only one join point, our composition algorithm only has to loop once; hence, no loop unrolling is required. The next step is to validate our models using both RSA and our custom *validator* package. Running the *ValidateComposedModel* class from our custom package to validate the composed model (CM) produces the output shown below.

Reading model CM from disk ...Validating CM Model container is valid All Classes are valid :) All Class Operations are valid :) All Message events are valid :) Collaboration model element ReceiveCall is valid :) All Owned attributes in ReceiveCall are valid :) Interaction model element ReceiveCall is valid :) All Message Occurrence Specifications in ReceiveCall are valid :) All Messages in ReceiveCall are valid :) All lifelines in ReceiveCall are valid :) Our model and all its model elements meet our validation requirements

Fig. 6.4. Context Specific Aspect Model.

Using Model Transformation Language Semantics for Aspects Composition 33

composed model. We can see that the advice has been properly weaved into the **alt** combined fragment's first operand. This way, if the user chooses to accept a call and another *incomingCall* message is received by the phone the phone's display will show the new caller's information. The user can then send an *ok* message to phone to put the caller on hold.

The main objective of this work is to compose aspect models represented as UML sequence diagrams (SDs) using the Atlas Transformation Language (ATL). Toward this end, we proposed a formal definition of SDs in terms of an ordered list of interaction fragments, and in the process defined three algorithms for pointcut detection, advice composition and complete composition. We designed and implemented the Complete Composition algorithm to achieve composition of the primary model and generic aspect models. We consider aspect composition as a form of model transformation; therefore, the algorithm is implemented using model transformations written in the ATL model transformation language. We also designed a simple metamodel in Ecore for mark models used to define binding rules which are used to instantiate generic aspect models. We finally designed and implemented a custom Java package to help validate the composed model. The Java classes check the composed model elements against a list of defined constraints designed to ensure that essential model elements are present in the composed model and are properly

The Complete Composition Algorithm proposed and implemented composes behavioral views of both primary and aspect models represented as UML sequence diagrams. The primary model defines the core system behavior without cross-cutting concerns while the aspect models represent behavior that cross-cuts the primary model. The models are described in UML Sequence diagrams created using an eclipse-based modeling tool (RSA)

Using ATL, a mature model transformation language, the Complete Composition Algorithm is implemented using three transformation models; namely JoinPointsCount, Instantiate, and Compose. The JoinPointsCount transformation determines the number of join points in the primary model given the pointcut from the aspect model. The aspect models are made generic so that they can be more reusable; therefore, they must first be instantiated before they can be composed with the primary model. The Instantiate transformation is used to instantiate generic aspect models in the context of the application using a set of binding rules defined in mark models to produce context specific aspect models. The Compose transformation then takes the primary model and context specific aspect model as inputs, and produces a composed model. This process is repeated as many times as there are join

To test our design and implementation, several test cases and case studies were successfully conducted. Validation was achieved by using custom Java classes to check the model against a set of defined constraints. The composed model was also validated using RSA's built-in validation feature. To verify composition, a sequence diagram was generated from the model's UML2.1 file using RSA. The generated sequence diagram was then visually

and then exported to a UML2.1 file for composition and validation.

points and aspect models until a complete integrated system is obtained.

inspected to see if the composition was performed properly.

**7. Conclusion** 

initialized.

Fig. 6.5. Composed Model.

Running the *ValidateComposedModel* to validate the context specific aspect model (CSAM) also shows that the model is valid. Both models are valid according to the checklist defined in our *validator* package. Both models are then imported into RSA for validation and visual inspection.

Figure 6.4 and 6.5 show the context specific aspect and composed model SDs created on RSA after importing the models. The pointcut was properly bound. We can see in Figure 6.4a that |accept and |end are bound to *pickUp* and *hangUp* while |receiver and |Client have been bound to *user* and *User* respectively. Figure 6.4b shows that the advice has also been bound as specified by the mark model. Figure 6.5 shows a sequence diagram for our composed model. We can see that the advice has been properly weaved into the **alt** combined fragment's first operand. This way, if the user chooses to accept a call and another *incomingCall* message is received by the phone the phone's display will show the new caller's information. The user can then send an *ok* message to phone to put the caller on hold.

## **7. Conclusion**

32 Semantics in Action – Applications and Scenarios

Running the *ValidateComposedModel* to validate the context specific aspect model (CSAM) also shows that the model is valid. Both models are valid according to the checklist defined in our *validator* package. Both models are then imported into RSA for validation and visual

Figure 6.4 and 6.5 show the context specific aspect and composed model SDs created on RSA after importing the models. The pointcut was properly bound. We can see in Figure 6.4a that |accept and |end are bound to *pickUp* and *hangUp* while |receiver and |Client have been bound to *user* and *User* respectively. Figure 6.4b shows that the advice has also been bound as specified by the mark model. Figure 6.5 shows a sequence diagram for our

Fig. 6.5. Composed Model.

inspection.

The main objective of this work is to compose aspect models represented as UML sequence diagrams (SDs) using the Atlas Transformation Language (ATL). Toward this end, we proposed a formal definition of SDs in terms of an ordered list of interaction fragments, and in the process defined three algorithms for pointcut detection, advice composition and complete composition. We designed and implemented the Complete Composition algorithm to achieve composition of the primary model and generic aspect models. We consider aspect composition as a form of model transformation; therefore, the algorithm is implemented using model transformations written in the ATL model transformation language. We also designed a simple metamodel in Ecore for mark models used to define binding rules which are used to instantiate generic aspect models. We finally designed and implemented a custom Java package to help validate the composed model. The Java classes check the composed model elements against a list of defined constraints designed to ensure that essential model elements are present in the composed model and are properly initialized.

The Complete Composition Algorithm proposed and implemented composes behavioral views of both primary and aspect models represented as UML sequence diagrams. The primary model defines the core system behavior without cross-cutting concerns while the aspect models represent behavior that cross-cuts the primary model. The models are described in UML Sequence diagrams created using an eclipse-based modeling tool (RSA) and then exported to a UML2.1 file for composition and validation.

Using ATL, a mature model transformation language, the Complete Composition Algorithm is implemented using three transformation models; namely JoinPointsCount, Instantiate, and Compose. The JoinPointsCount transformation determines the number of join points in the primary model given the pointcut from the aspect model. The aspect models are made generic so that they can be more reusable; therefore, they must first be instantiated before they can be composed with the primary model. The Instantiate transformation is used to instantiate generic aspect models in the context of the application using a set of binding rules defined in mark models to produce context specific aspect models. The Compose transformation then takes the primary model and context specific aspect model as inputs, and produces a composed model. This process is repeated as many times as there are join points and aspect models until a complete integrated system is obtained.

To test our design and implementation, several test cases and case studies were successfully conducted. Validation was achieved by using custom Java classes to check the model against a set of defined constraints. The composed model was also validated using RSA's built-in validation feature. To verify composition, a sequence diagram was generated from the model's UML2.1 file using RSA. The generated sequence diagram was then visually inspected to see if the composition was performed properly.

Using Model Transformation Language Semantics for Aspects Composition 35

Future research would also include composition of other behavioral views (Statechart and Activity diagrams) and structural views (class diagrams) of the primary model and aspect models. Our current approach does achieve some composition of structural model elements from the primary and aspect models but does ignores associations between the structural

This research work was partly sponsored by NSERC (Natural Sciences and Engineering

Research Council) of Canada through grant number EGP 401451-10.

aMessages Sequence Returns all messages from the aspect model.

aLifelines Sequence Returns all lifelines from the aspect model. aSendEvents Sequence Returns all SOEs from the aspect model. aRecvEvents Sequence Returns all ROEs from the aspect model.

aProperties Sequence Returns all lifeline properties the aspect model. allBindings Sequence Returns all binding rules from the mark model.

adviceMOSEncoding() String A tagging string for advice MOSs getAspectFragments(sd) Sequence Returns MOSs and CFs from a

getPrimaryFragments() Sequence Returns MOSs and CFs from a

getFragments() Sequence Returns MOSs and CFs within the

getFragments() Sequence Returns MOSs and CFs within the

sameEventType(e1, e2) Boolean Returns true if the given

given SD from the aspect model.

given SD from the primary model.

*MessageEvents* are both ROE or

index from *src* and *tgt* sequences

context Interaction Operand.

context CF.

SOE

Boolean Checks if the fragments at the same

are "equal".

aOperations Sequence Returns all class operations from the aspect model.

**9. Appendix A - JointPointsCount helpers** 

**10. Appendix B - PointcutMatchHelpers library** 

**Helper name Return type Purpose** 

**Helper Name Return type Purpose** 

elements.

**8. Acknowledgement** 

PairwiseMatchFragments

(src, tgt)

Using ATL for composing models does have its challenges. The inability to navigate target models or modify input models makes intricate weaving of aspects a hard problem. However, the benefits of using a versatile language that allows for powerful expressions may outweigh the challenges.

## **7.1 Limitations and future work**

This work is part of an ambitious quest for a complete AOM composition framework and a set of tools that can allow software architects and developers to easily apply AO techniques to model driven software development. There are several limitations that must be addressed, and new features to be added before our composition approach can be more useful. These include:


Our approach currently supports the use of the wildcard "\*" for defining template parameters. This gives some flexibility when defining generic aspect models. However, to allow for powerful expressions, we need to use regular expressions. Currently ATL (version 2.0.x) does not support the use of regular expressions for comparing or matching strings. ATL only uses regular expressions for replacing and splitting strings. Future research may include development of a custom string ATL library that will provide helpers that implement regular expression matching operations.

Furthermore, we have assumed that messages in the primary and aspect models have simple arguments that are either strings or integers. However, messages can have arguments that are instances of classes defined in the structural view of the system. Therefore, the current use of lazy rules to create message arguments (and class operation parameters) may not be ideal. Future work would look at a more efficient and elegant way of creating message arguments in the target model. Interaction Occurrences provide a way to reuse and manage complex SDs. They are a notation for copying one SD (basic) into another one, which may be larger (Pilone et al., 2005). Our current composition approach has no support processing interaction occurrences; therefore, future research would look into including interaction occurrences in pointcut detection and composition. This will provide challenges because the use of interaction occurrences means that the primary model SD or the aspect model SDs may contain more than one instance of *Interaction* depending on the number of interaction occurrences.

Recall that BESs and EOSs were ignored in our composition approach. Future work could look into how these model elements can be processed with other interaction fragments. We would also add support for other model elements that are not currently supported; like, connectors, signals (and related events), gates, etc.

Future research would also include composition of other behavioral views (Statechart and Activity diagrams) and structural views (class diagrams) of the primary model and aspect models. Our current approach does achieve some composition of structural model elements from the primary and aspect models but does ignores associations between the structural elements.

## **8. Acknowledgement**

34 Semantics in Action – Applications and Scenarios

Using ATL for composing models does have its challenges. The inability to navigate target models or modify input models makes intricate weaving of aspects a hard problem. However, the benefits of using a versatile language that allows for powerful expressions

This work is part of an ambitious quest for a complete AOM composition framework and a set of tools that can allow software architects and developers to easily apply AO techniques to model driven software development. There are several limitations that must be addressed, and new features to be added before our composition approach can be more

Support for BehaviorExecutionSpecifications (BESs), ExecutionOccurrenceSpecifications

Our approach currently supports the use of the wildcard "\*" for defining template parameters. This gives some flexibility when defining generic aspect models. However, to allow for powerful expressions, we need to use regular expressions. Currently ATL (version 2.0.x) does not support the use of regular expressions for comparing or matching strings. ATL only uses regular expressions for replacing and splitting strings. Future research may include development of a custom string ATL library that will provide helpers that

Furthermore, we have assumed that messages in the primary and aspect models have simple arguments that are either strings or integers. However, messages can have arguments that are instances of classes defined in the structural view of the system. Therefore, the current use of lazy rules to create message arguments (and class operation parameters) may not be ideal. Future work would look at a more efficient and elegant way of creating message arguments in the target model. Interaction Occurrences provide a way to reuse and manage complex SDs. They are a notation for copying one SD (basic) into another one, which may be larger (Pilone et al., 2005). Our current composition approach has no support processing interaction occurrences; therefore, future research would look into including interaction occurrences in pointcut detection and composition. This will provide challenges because the use of interaction occurrences means that the primary model SD or the aspect model SDs may contain more than one instance of *Interaction* depending on

Recall that BESs and EOSs were ignored in our composition approach. Future work could look into how these model elements can be processed with other interaction fragments. We would also add support for other model elements that are not currently supported; like,

Improved string pattern definition and matching for template parameters.

Non primitive message and operation argument types.

An eclipse plug-in to help in the creation of the mark model.

may outweigh the challenges.

useful. These include:

**7.1 Limitations and future work** 

Support for Interaction Occurrences.

(EOSs) and other model elements.

implement regular expression matching operations.

the number of interaction occurrences.

connectors, signals (and related events), gates, etc.

 Structural view composition. Invoking ATL from Java.

This research work was partly sponsored by NSERC (Natural Sciences and Engineering Research Council) of Canada through grant number EGP 401451-10.


## **9. Appendix A - JointPointsCount helpers**

## **10. Appendix B - PointcutMatchHelpers library**


Using Model Transformation Language Semantics for Aspects Composition 37

joinPointsFragments() Sequence Returns fragments at the start of

getMaxInt() ValueSpecificationAction Returns the *maxInt* value of the

getMinInt() ValueSpecificationAction Returns the *minInt* value of the

Model Generates a *Model* element that contains all the other target model

Collaborations Creates a *Collaboration* that contains the composed model

Interactions Generates the *Interaction* that owns fragments, lifelines, messages,

CombinedFragments Generates all *CombinedFragments* including nested ones.

Boolean Returns true if the context

Boolean Returns true if self is the same as

Boolean Returns true if the context CF is the

Boolean Returns true if the context

Sequence Returns all MOSs enclosed by the

Sequence Returns all MOSs enclosed by the context CF.

Boolean Returns true if the given sequences

Boolean Checks if the elements at the same

are equal.

the join points.

supplied operation.

same operation.

operation is the same as the

the supplied event; that is, they are of the same type and have the

same as the given interaction fragment which also has to be CF.

interaction operand is equal to the

of operands have matching pair of operands, that is, the operands at

index from the *src* and *tgt* sequence

given interaction fragment .

context interaction operand.

the same index are equal.

given Interaction constraint.

given Interaction constraint.

equals(o) Context =

equals(e) Context =

equals(f) Context =

equals(f) Context =

getMOSs() Context =

getMOSs() Context =

PairwiseMatchMOSs (src,

**11. Appendix C – Compose rules** 

elements.

interaction.

InteractionOperands Generates all *InteractionOperands*.

etc.

**Rule Purpose** 

Operation

MessageEvent

CombinedFragment

InteractionOperand

InteractionOperand

CombinedFragment

(src,tgt)

tgt)

PairwiseMatchOperands


getBinding(name) String Retrieves the binding value from

bindingDefined(String) Boolean Returns true if a binding value

equals(mos) Context = MOS Boolean Checks if the context and supplied

invalidMOSName() Boolean Returns true if the context MOS

getLifelineMOS() Sequence Returns MOSs that cover the

getAspectSD (name) Interaction Returns an SD (Advice or Pointcut)

getMessageLifelines (m) Sequence Returns the sender and receiver

getSDLifelines (sd) Sequence Returns all lifelines in a given SD. getSDMessages (sd) Sequence Returns all messages in a given SD. createMOSName (code, idx) String Generates a name for the context

equals(c) Context = Class Boolean Returns true the context class is the

classMatch (c1 , c2) Returns true if the two classes are

samePropertyType(pct ,

fromAspectModel() Context

fromAspectModel() Context

fromPrimaryModel() Context

fromPrimaryModel() Context = Message

notInPrimaryModel() Context = Class

notInPrimaryModel() Context = MessageEvent

notInPrimaryModel() Context = Operation

= Model

= Model

= Interaction

core)

the mark model for the given

has been tagged meaning it was added from a previous

exists for the given parameter.

have the same type (class).

MOSs are equivalent.

composition iteration

from the Aspect Model.

from the aspect model.

from the primary model.

is from the primary model.

from the primary model.

lifelines for the given message.

MOS given our tagging string and MOS index in the sequence of

Interaction is from the aspect

MessageEvent is from the primary

operation is from the primary

same as the supplied class; that is,

if they have the same name.

context lifeline.

MOS.

Boolean Returns true if the context model is

Boolean Returns true if the context

Boolean Returns true if the context model is

Boolean Returns true if the context message

Boolean Returns true if the context class is

Boolean Returns true if the context

Boolean Returns true if the context

model.

model.

equal.

model.

template parameter.

Boolean Returns true if the given *Properties*


## **11. Appendix C – Compose rules**


Using Model Transformation Language Semantics for Aspects Composition 39

M. Didonet Del Fabro, J. Bézivin, and P. Valduriez, "Weaving Models with the Eclipse

F. Fleurey, B. Baudry, R. France, and S. Ghosh, "A Generic Approach for Automatic Model

R. France, I. Ray, G. Georg and S. Ghosh, "Aspect-Oriented Approach to Design Modeling,"

I. Groher, and M. Voelter. "XWeave: models and aspects in concert," In Proceedings of the 10th international workshop on Aspect-oriented modeling, March 2007. K. Hamilton, and R. Miles, "Learning UML 2.0", O'Reilly, April 2006, ISBN-10: 0-596-00982-8

J. Happe, S. Becker, C. Rathfelder, H. Friedrich, and R. H. Reussner, "Parametric

C. Jeanneret, R. France, and B. Baudry, "A reference process for model composition," In

J.M. Jézéquel, "Model Transformation Techniques Model Techniques," Online resource

F. Jouault, and I. Kurtev, "Transforming Models with ATL," In proceedings of the Model

J. Kienzle, W. A. Abed, and J. Klein, "Aspect-oriented multi-view modeling," In Proceedings

J. Klein, L. Hélouët, and J.M. Jézéquel, "Semantic-based Weaving of Scenarios," AOSD 06,

J. Klein, F. Fleurey, and J.M. Jézéquel, "Weaving Multiple Aspects in Sequence Diagrams,"

Kompose website, http://www.kermeta.org/kompose/ last accessed on September 23,

M. Milanovic, "Complete ATL Bundle for launching ATL transformations

http://milan.milanovic.org/download/atl.zip, last accessed on September 10,

B. Morin, J. Klein, O. Barais, and J.M. Jézéquel, "A generic weaver for supporting product

lines," In proceedings of the 13th international workshop on Software architectures

Wesley Professional, December 30, 2004, ISBN-10: 0321268881.

Transformation in Practice Workshop, October 3rd 2005.

Transactions on AOSD III, LNCS 4620, pp. 167–199, 2007

programmatically." Online resource available at:

and mobility, Leipzig, Germany, May 2008.

Performance Completions for Model-Driven Performance Prediction," Journal of Systems and Software, Volume 82 Issue 1, January 2009, Elsevier Science Inc. I. Jacobson, and P. Ng, "Aspect-oriented software development with use case," Addison-

Proceedings of the 2008 AOSD workshop on Aspect-oriented modeling, April 2008.

available at: http://modelware.inria.fr/static\_pages/slides/ModelTransfo.pdf, last

of the 8th ACM international conference on Aspect-oriented software development,

Esslingen, Germany, 2006.

thesis, Carleton University, 2008.

accessed in August 30, 2009

March 20-24, Bonn, Germany, 2006

March 2009.

2009.

2009.

Heidelberg.

.

AMW plug-in," In: Eclipse Modeling Symposium, Eclipse Summit Europe 2006,

Composition," Lecture Notes In Computer Science archive Models in Software Engineering: Workshops and Symposia at MoDELS 2007, Nashville, TN, USA, September 30 - October 5, 2007,pages 7–15, 2008, Springer-Verlag Berlin,

IEEE Proceedings – Software, Special Issue on Early Aspects: Aspect –Oriented Requirements Engineering and Architecture Design, 151(4):173-185, August 2004. H. Gong, "Composition of Aspects Represented as UML Activity Diagrams.", Master's


InteractionConstraints Creates all the constraints.

### **12. References**

http://wiki.eclipse.org/ATL/User\_Guide, last accessed on September 22, 2009.


.

38 Semantics in Action – Applications and Scenarios

OpaqueExpressions Generates *OpaqueExpressions* for *InteractionConstraints*.

MOSs Creates all *MessageOccurrenceSpecifications* (MOSs).

SOEs Produces *SendOperationEvents* for *sendEvent* MOSs.

ROEs Generates *ReceiveOperationEvents* for *receiveEvent* MOSs.

lazy rule CreateLUN Creates *LiteralUnlimitedNaturals* that are used for guard conditions.

lazy rule CreateLS Creates strings that are used for guard conditions and arguments for

lazy rule CreateLI Creates integers that are used for guard conditions and arguments

Samuel A. Ajila, Dorina Petriu, and Pantanowitz Motshegwa, *Using Model Transformation* 

ATLAS group LINA & INRIA Nantes, "ATL User Manual version 0.7," Online resource

http://www.eclipse.org/m2m/atl/doc/ATL\_User\_Manual[v0.7].pdf, last

O. Barais, J. Klein, B. Baudry, A. Jackson, and S. Clarke, "Composing Multi-View Aspect

A. Colyer, A. Clement, G. Harley, M. Webster, "Eclipse AspectJ. Aspect-Oriented

Wesley Professional, December 14, 2004, ISBN : 0-321-24587-3

Models," In Proceedings of the Seventh International Conference on Composition-

Programming with AspectJ and the Eclipse AspectJ Development Tools,"Addison

*Semantics for Aspects Composition*, 2010 IEEE 4th International Conference on Semantic Computing (IEEE-ICSC 2010), pp 325-332, Carnegie Mellon University,

lazy rule CreateParam Creates parameters for operations in the target model.

http://wiki.eclipse.org/ATL/User\_Guide, last accessed on September 22, 2009.

Messages Produces messages for the target model.

Operations Generates operations for classes.

Properties Generates all *Properties* for lifelines.

messages.

for messages.

Pittsburgh, PA, USA, September 22 - 24, 2010

Based Software Systems, pages 43-52, 2008.

accessed on September 22, 2009.

Classes Generates classes.

**12. References** 

available at:

InteractionConstraints Creates all the constraints.

Lifelines Generates all lifelines.


**2** 

*China* 

Yingzhou Zhang1,2,3

**Program Slicing Based on Monadic Semantics** 

A program slice consists of those statements of a program that may directly or indirectly affect the variables computed at a given program point (Weiser, 1984). The program point p and the variable set V, denoted by <p, V>, is called a slicing criterion. Program slicing has applications in software testing and debugging, measurement, re-engineering, program comprehension and so on (Kamkar, 1995; Tip, 1995; Harman, 2001; Binkley, 1996; Gallagher,

Program slicing algorithms can be roughly classified as static slicing and dynamic slicing methods, according to whether they only use statically available information or compute those statements that influence the value of a variable occurrence for a specific program input. Most of the existing slicing algorithms rely on relation graphs such as system dependence graphs (SDG) or program dependence graphs (PDG). These slicing methods are incremental, sequential, not combinatorial or not parallelizable easily for multi-core systems. However modern programming languages support modularized programming and programs might consist of a set of modules. So the program analysis should reflect this design technology, and their methods (including program slicing) should be flexible,

As the behavior of a program is determined by the semantics of the language, it is reasonable to expect an approach for program slicing based on formal semantics of a program. On the basis of this view, this paper proposes an approach for program slicing based on modular monadic semantics, called *modular monadic slicing*. It can compute slices directly on abstract syntax, without explicit construction of intermediate structures such as

The program slicing methods focused on the semantics of programs can be found in ref. (Hausler, 1989; Ouarbya, 2002; Venkatesh, 1991). These methods are based on the standard denotational semantics of a program language. As mentioned in ref. (Moggi, 1991; Liang & Hudak, 1996; Wansbrough, 1997; Mosses, 1998; Zhang & Xu, 2004), traditional denotational semantics lack modularity and reusability. A better solution was to use *monads* (Moggi, 1991) to structure denotational semantics, with the help of *monad transformers* (Moggi, 1991; Wadler, 1992; Espinosa, 1995) which can transform a given monad into a new one with new

combinable, and parallelizable for improving the efficiency.

**1. Introduction** 

dependence graphs.

1991).

*1College of Computer, Nanjing University of Posts and Telecomm., Nanjing 2Jiangsu High Technology Research Key Lab. for Wireless Sensor Networks, 3Key Lab of Broadband Wireless Communication and Sensor Network Technology,* 

*Ministry of Education Jiangsu Province, Nanjing,* 


## **Program Slicing Based on Monadic Semantics**

## Yingzhou Zhang1,2,3

*1College of Computer, Nanjing University of Posts and Telecomm., Nanjing 2Jiangsu High Technology Research Key Lab. for Wireless Sensor Networks, 3Key Lab of Broadband Wireless Communication and Sensor Network Technology, Ministry of Education Jiangsu Province, Nanjing, China* 

## **1. Introduction**

40 Semantics in Action – Applications and Scenarios

Object Management Group, "UML Superstructure Specification, v2.1.1," Online resource

Object Management Group, "UML Superstructure Specification, v2.2," Online resource

OpenArchitectureware website, http://www.openarchitectureware.org/ last accessed on

D.C. Petriu, H. Shen, and A. Sabetta, "Performance Analysis of Aspect- Oriented UML

D. Pilone, and N. Pitman, "UML 2.0 in a Nutshell", O'Reilly, (June 2005 ) ISBN: 0-596-00795-

G. Straw, G. Georg, E. Song, S. Ghosh, R. France, and J. M. Bieman, "Model Composition Directives," LNCS 3273, pages 84-97, 2004, Springer Berlin / Heidelberg J. Whittle, J. Araújo, and A. Moreira, "Composing aspect models with graph

J. Whittle, and P. Jayaraman, "MATA: A Tool for Aspect-Oriented Modeling Based on

M. Woodside, D.C. Petriu, D.B. Petriu, J. Xu, T. Israr, G. Georg, R. France, J.M. Bieman, S.H.

accessed on July 20, 2009.

August 22, 2009.

7

September 23, 2009.

2007, Springer Berlin / Heidelberg

aspects at ICSE, pages 59-65, Shanghai, China, 2006

Oriented Modeling, Nashville TN USA, Oct 2007.

82 , Issue 1 (January 2009), pp 56-74, 2009

available at: http://www.omg.org/docs/formal/07-02-05.pdf, last accessed last

available at: http://www.omg.org/docs/formal/09-02-02.pdf, last accessed on

Models", Software and Systems Modeling (SoSyM), Vol. 6, No. 4., pages. 453-471,

transformations," In Proceedings of the 2006 international workshop on Early

Graph Transformation," At MoDELS'07: 11th International Workshop on Aspect-

Houmb, and J. Jürjens, "Performance analysis of security aspects by weaving scenarios extracted from UML models," Journal of Systems and Software, Volume A program slice consists of those statements of a program that may directly or indirectly affect the variables computed at a given program point (Weiser, 1984). The program point p and the variable set V, denoted by <p, V>, is called a slicing criterion. Program slicing has applications in software testing and debugging, measurement, re-engineering, program comprehension and so on (Kamkar, 1995; Tip, 1995; Harman, 2001; Binkley, 1996; Gallagher, 1991).

Program slicing algorithms can be roughly classified as static slicing and dynamic slicing methods, according to whether they only use statically available information or compute those statements that influence the value of a variable occurrence for a specific program input. Most of the existing slicing algorithms rely on relation graphs such as system dependence graphs (SDG) or program dependence graphs (PDG). These slicing methods are incremental, sequential, not combinatorial or not parallelizable easily for multi-core systems. However modern programming languages support modularized programming and programs might consist of a set of modules. So the program analysis should reflect this design technology, and their methods (including program slicing) should be flexible, combinable, and parallelizable for improving the efficiency.

As the behavior of a program is determined by the semantics of the language, it is reasonable to expect an approach for program slicing based on formal semantics of a program. On the basis of this view, this paper proposes an approach for program slicing based on modular monadic semantics, called *modular monadic slicing*. It can compute slices directly on abstract syntax, without explicit construction of intermediate structures such as dependence graphs.

The program slicing methods focused on the semantics of programs can be found in ref. (Hausler, 1989; Ouarbya, 2002; Venkatesh, 1991). These methods are based on the standard denotational semantics of a program language. As mentioned in ref. (Moggi, 1991; Liang & Hudak, 1996; Wansbrough, 1997; Mosses, 1998; Zhang & Xu, 2004), traditional denotational semantics lack modularity and reusability. A better solution was to use *monads* (Moggi, 1991) to structure denotational semantics, with the help of *monad transformers* (Moggi, 1991; Wadler, 1992; Espinosa, 1995) which can transform a given monad into a new one with new

Program Slicing Based on Monadic Semantics 43

(>>=) :: *m a* (*a m b*) *m b*

Here, *return* is the Haskell name for the unit and >>= (pronounced "bind") is the extension operation of the monad. The above definition of the monad class means: a parameterized type *m* (which may think of as a function from types to types) is a monad if it supports the two operations *return* and >>= with the types given. Using the combinator analogy, a monad *m* is a combinator that can apply to different values. *m a* is a combinator applying to a value of type *a*. The *return* operation puts a value into a monadic combinator. The >>= operation takes the value from a monadic combinator and passes it to a function to produce a monadic combinator containing a new value, possibly of a different type. The >>= operation is known as "bind" because it binds the value in a monadic combinator to the first

To be a proper monadic combinators, the *return* and >>= operations must work together according to some simple laws. Monads laws state in essence that >>= operation (sequential composition) is associative, and *return* is its unit/identity. Failure to satisfy these laws will result in monads that do not behave properly and may cause subtle problems when using

A monad (call it *m*) therefore defines a type of computation. The nature of the computation is captured by the choice of the type *m*. The *return* operation constructs a trivial computation that just renders its argument as its result. The >>= operation combines two computations

To make the use of monads more convenient, we adopt the following syntactic sugar (which

{*m*; *e*}*m m* >>= \ \_ {*e*} {*x m*; *e*}*<sup>m</sup> m* >>= \ *x*  {*e*} {**let** *exp*; *e*} **let** *exp* **in** {*e*}

In practice, the computations can't be performed in isolation. In this case, we need a monad that combines the features of the two monads into a single computation. It is impossible in general to combine two monads to form a new monad. Moreover, it is inefficient and poor practice to write a new monad instance with the required characteristics each time a new combination is desired. Instead, there is the technique, called monad transformers (Liang, 1998), which can transform a given monad into a new one that has both the new operations and maintains those of the former monad. The concept of monad transformers was rediscovered by D.Espinosa in Moggi's original work. He developed a system, Semantic Lego, which implemented Moggi's original *monad constructors* to give a modular semantics for

2 Do notation is an expressive shorthand for building up monadic computations. In short, the do notation allows us to write monadic computations using a pseudo-imperative style with named variables. The result of a monadic computation can be "assigned" to a variable using a left arrow

together to make more complex computations of that type.

is similar to S. Liang's notation and the do-notation in Haskell as well):

{*e*} *e*

**class** Monad *m* **where** *return* :: *a m a*

argument of an operation.

the do-notation2.

languages.

operator.

operations. S.Liang et al. used monads and monad transformers to specify the semantics of programming language; called it *modular monadic semantics*. In this paper, we will employ it in our program slicing algorithm.

In our previous work (Zhang et al, 2004, 2005, 2006; Zhang & Xu, 2005; Wu et al. 2006), we abstracted the computation of program slicing as a simple slice monad transformer, which took only the label set *L* as its parameter, without reflecting explicitly the change of the slice table Slices. In ref. (Zhang, 2007), we presented theoretical foundation for our previous work. Based on these theories of the monadic slicing and from the view of the practical implementation, this paper will redesign the static slice monad transformer. The extensibility and reusability of our monadic method will be showed by easily introducing a new program feature (such as pointers) to slicing analysis.

The rest of the paper is organized as follows: In Section 2, we briefly introduce and illustrate the concepts of modular monadic semantics through a simple example language. The computation of program slicing is abstracted as *slice monad transformer* in Section 3. In Section 4, we discuss and illustrate our static slicing algorithm in detail. In Section 5, we show how our slicing algorithm can be readily adapted to an extension for the example language with pointers. In Section 6 and 7, we address the implementation, the time and space complexity analysis. We conclude this paper with directions for future work in Section 8.

Along the paper, the presentation follows the monadic semantics style. We use Haskell1 notation with some freedom in the use of mathematical symbols and declarations. For brevity and convenience, we will omit the type constructors in some definitions.

## **2. Modular monadic semantics**

In this section, we briefly review the theory of monads (Wadler & Thiemann, 2003; Moggi, 1989), monad transformers and modular monadic semantics (Liang, 1998). Readers familiar with these topics may skip the section, except for the last three paragraphs (about the syntax and monadic semantics of an example language).

## **2.1 Monads and monad transformers**

Monads, originally coming from philosophy, were discovered in category theory in the 1950s and introduced to the semantics community by Moggi in 1990s (Moggi, 1989). After this work, Wadler popularized Moggi's ideas in the functional programming community (esp. in Haskell) (Wadler, 1995). In the monad-based view of computation, a monad is a way to structure computations in terms of values and sequences of computations using those values (Newbern, 2002). The monad determines how combined computations form a new computation and frees the programmer from having to code the combination manually each time it is required. From this view, a monad can be thought as a strategy for combining computations into more complex computations.

In Haskell, monads are implemented as a type constructor class with two member operations/functions.

 1 Haskell is an advanced purely functional programming language. Please visit its official website (http://www.haskell.org or http://haskell.org) for more information.

42 Semantics in Action – Applications and Scenarios

operations. S.Liang et al. used monads and monad transformers to specify the semantics of programming language; called it *modular monadic semantics*. In this paper, we will employ it

In our previous work (Zhang et al, 2004, 2005, 2006; Zhang & Xu, 2005; Wu et al. 2006), we abstracted the computation of program slicing as a simple slice monad transformer, which took only the label set *L* as its parameter, without reflecting explicitly the change of the slice table Slices. In ref. (Zhang, 2007), we presented theoretical foundation for our previous work. Based on these theories of the monadic slicing and from the view of the practical implementation, this paper will redesign the static slice monad transformer. The extensibility and reusability of our monadic method will be showed by easily introducing a

The rest of the paper is organized as follows: In Section 2, we briefly introduce and illustrate the concepts of modular monadic semantics through a simple example language. The computation of program slicing is abstracted as *slice monad transformer* in Section 3. In Section 4, we discuss and illustrate our static slicing algorithm in detail. In Section 5, we show how our slicing algorithm can be readily adapted to an extension for the example language with pointers. In Section 6 and 7, we address the implementation, the time and space complexity analysis. We conclude this paper with directions for future work in

Along the paper, the presentation follows the monadic semantics style. We use Haskell1 notation with some freedom in the use of mathematical symbols and declarations. For

In this section, we briefly review the theory of monads (Wadler & Thiemann, 2003; Moggi, 1989), monad transformers and modular monadic semantics (Liang, 1998). Readers familiar with these topics may skip the section, except for the last three paragraphs (about the syntax

Monads, originally coming from philosophy, were discovered in category theory in the 1950s and introduced to the semantics community by Moggi in 1990s (Moggi, 1989). After this work, Wadler popularized Moggi's ideas in the functional programming community (esp. in Haskell) (Wadler, 1995). In the monad-based view of computation, a monad is a way to structure computations in terms of values and sequences of computations using those values (Newbern, 2002). The monad determines how combined computations form a new computation and frees the programmer from having to code the combination manually each time it is required. From this view, a monad can be thought as a strategy for combining

In Haskell, monads are implemented as a type constructor class with two member

1 Haskell is an advanced purely functional programming language. Please visit its official website

brevity and convenience, we will omit the type constructors in some definitions.

in our program slicing algorithm.

**2. Modular monadic semantics** 

**2.1 Monads and monad transformers** 

and monadic semantics of an example language).

computations into more complex computations.

(http://www.haskell.org or http://haskell.org) for more information.

operations/functions.

Section 8.

new program feature (such as pointers) to slicing analysis.

Here, *return* is the Haskell name for the unit and >>= (pronounced "bind") is the extension operation of the monad. The above definition of the monad class means: a parameterized type *m* (which may think of as a function from types to types) is a monad if it supports the two operations *return* and >>= with the types given. Using the combinator analogy, a monad *m* is a combinator that can apply to different values. *m a* is a combinator applying to a value of type *a*. The *return* operation puts a value into a monadic combinator. The >>= operation takes the value from a monadic combinator and passes it to a function to produce a monadic combinator containing a new value, possibly of a different type. The >>= operation is known as "bind" because it binds the value in a monadic combinator to the first argument of an operation.

To be a proper monadic combinators, the *return* and >>= operations must work together according to some simple laws. Monads laws state in essence that >>= operation (sequential composition) is associative, and *return* is its unit/identity. Failure to satisfy these laws will result in monads that do not behave properly and may cause subtle problems when using the do-notation2.

A monad (call it *m*) therefore defines a type of computation. The nature of the computation is captured by the choice of the type *m*. The *return* operation constructs a trivial computation that just renders its argument as its result. The >>= operation combines two computations together to make more complex computations of that type.

To make the use of monads more convenient, we adopt the following syntactic sugar (which is similar to S. Liang's notation and the do-notation in Haskell as well):

$$\begin{array}{ll} \{e\} & \equiv & e\\ \{m; e\}\_m & \equiv & m \gg = \{e\}\\ \{x \leftarrow m; e\}\_m & \equiv & m \gg = \{x \rightarrow \{e\}\\ \{\textbf{let} \, exp; e\} & \equiv & \textbf{let} \, exp \text{ in } \{e\} \end{array}$$

In practice, the computations can't be performed in isolation. In this case, we need a monad that combines the features of the two monads into a single computation. It is impossible in general to combine two monads to form a new monad. Moreover, it is inefficient and poor practice to write a new monad instance with the required characteristics each time a new combination is desired. Instead, there is the technique, called monad transformers (Liang, 1998), which can transform a given monad into a new one that has both the new operations and maintains those of the former monad. The concept of monad transformers was rediscovered by D.Espinosa in Moggi's original work. He developed a system, Semantic Lego, which implemented Moggi's original *monad constructors* to give a modular semantics for languages.

<sup>2</sup> Do notation is an expressive shorthand for building up monadic computations. In short, the do notation allows us to write monadic computations using a pseudo-imperative style with named variables. The result of a monadic computation can be "assigned" to a variable using a left arrow operator.

Program Slicing Based on Monadic Semantics 45

building blocks. The crucial property of modular monadic semantics is the division of the monad *m* into a series of monad transformers, each representing a computation. As mentioned in the previous section, monad transformers provide the power to represent the abstract notion of programming language features, but still allow us to access low-level semantic details. The concept of lifting allows us to consider the interactions between various features. In some sense, *monad transformers can be designed once and for all* (Liang, 1998), since they are entirely independent of the language being described. From this view, we can draw the computation of program slicing as an entity that is independent of the language being analyzed. This will be discussed in the next section. Before doing it, we illustrate the modular monadic semantic description of a very simple imperative

The **W** language considered in this paper is very similar to the language described in ref. (Slonneger & Kurtz, 1995).The abstract syntax and semantic building blocks of the **W** language are provided in Figure 2 and 3 respectively. In Figure 3, the identifier *Fix* denotes a fixpoint operator; *xtdEnv* and *lkpEnv* are the updating and lookup operators of environments Env, respectively; *updSto* and *alloc* are the updating and allocation functions of stores Loc, respectively; *rdEnv* and *inEnv* are the basic operators of the enviornment monad EnvMonad (given in Figure 1); *putValue* and *getValue* are the writting and reading functions of I/O actions, respectively. Following Venkatesh's assumption for expressions in ref. (Venkatesh, 1990), we also assume that the labeled expressions have no side-effects. The expressions, whose syntax is left unspecified for the sake of generality, consist of operations over identifiers and are uniquely labeled. The label is for the entire

arg: Arg (Arguments); b: Blk (Blocks); c: Cmd (Commands);

d: Dec (Declarations); e: Exp (Expressions); ide: Ide (Identifiers);

l: Label (Labels); p: Prg (Programs); t: Type (Types)

In modular monadic semantics, the monad definition is simply a composition of the corresponding monad transformers, applied to a base monad. In this paper, we use the


programming language **W**.

Domains:

Abstract Syntax:

p :: = **program** ide **is** b

d :: = **const** ide = l.e | **var** ide : t | d1; d2

c :: = ide := l.e | c1; c2| skip | **read** ide | **write** l.e

b :: = d **begin** c **end**

Fig. 2. Abstract syntax of the **W** language.

expression.

In Haskell, a monad transformer can be defined as any type constructor *t* such that if *m* is a monad, so is "*t m*", by using the two parameter constructor class MonadTrans:

**class** (Monad *m*, Monad *t m*) MonadTrans *t m* **where** *lift* :: *m a t m a*

The member function *lift* lifts a monadic computation in the inner monad *m* into the combined monad "*t m*". Furthermore, we expect a monad transformer to add features, without changing the nature of an existing computation. This can be obtained by the properties of *lift* function above (also called *monad transformer laws*). The monad transformer laws guarantee the basic lifting property that any program, which does not use the added features, should behave in the same way after a monad transformer is applied. Intuitively, these laws say that lifting a null computation brings about a null computation, and that lifting a sequence of computations is equivalent to first lifting them individually, and then combining them in the lifted monad.

For example, Figure 1 gives the environment monad transformer, EnvT, which can be used to add environment reading functionality to other monads. In Figure 1, the functions *rdEnv* and *inEnv*, return the current environment and perform a computation in a given environment, respecitively.

```
newtype EnvT r m a = EnvT {runEnvT :: r  m a} 
instance (Monad m)  Monad (EnvT r m) where
 return a = EnvT (\_ return a) 
 m >>= k = EnvT (\r {a  runEnvT m r; runEnvT (k a) r} ) 
instance MonadTrans (EnvT r) where
    lift m = EnvT (\_ m) 
class (Monad m)  EnvMonad r m where
    inEnv :: r  m a  m a
    rdEnv :: m r
instance (Monad m)  EnvMonad r (EnvT r m) where
    inEnv r m = EnvT (\_ runEnvT m r) 
    rdEnv = EnvT (\r return r)
```
Fig. 1. Environment monad transformer EnvT(Liang,1998).

### **2.2 Modular monadic semantics**

Modular monadic semantics specifies the semantics of a programming language by mapping terms to computations, where the details of the environment, store, etc. are hidden within a monad. This is difference from traditional denotational semantics, which maps a term (an environment or a continuation) to an answer. The modular monadic semantics is composed of two parts: *modular semantic building blocks* and *monad transformers*. Semantic building blocks define the monadic semantics of individual source language features. They are independent of each other.

Monad transformers define the kernel-level operations in a modular way. Multiple monad transformers can be composed to form the underlying monad used by all the semantic

In Haskell, a monad transformer can be defined as any type constructor *t* such that if *m* is a

**class** (Monad *m*, Monad *t m*) MonadTrans *t m* **where**

The member function *lift* lifts a monadic computation in the inner monad *m* into the combined monad "*t m*". Furthermore, we expect a monad transformer to add features, without changing the nature of an existing computation. This can be obtained by the properties of *lift* function above (also called *monad transformer laws*). The monad transformer laws guarantee the basic lifting property that any program, which does not use the added features, should behave in the same way after a monad transformer is applied. Intuitively, these laws say that lifting a null computation brings about a null computation, and that lifting a sequence of computations is equivalent to first lifting them individually, and then

For example, Figure 1 gives the environment monad transformer, EnvT, which can be used to add environment reading functionality to other monads. In Figure 1, the functions *rdEnv* and *inEnv*, return the current environment and perform a computation in a given

> **newtype** *EnvT r m a* = *EnvT* {*runEnvT* :: *r m a*} **instance** (Monad *m*) Monad (*EnvT r m*) **where**

**instance** (Monad *m*) EnvMonad *r* (*EnvT r m*) **where** *inEnv r m* = *EnvT* (\\_ *runEnvT m r*)

*m* >>= *k* = *EnvT* (\*r* {*a runEnvT m r*; *runEnvT* (*k a*) *r*} )

Modular monadic semantics specifies the semantics of a programming language by mapping terms to computations, where the details of the environment, store, etc. are hidden within a monad. This is difference from traditional denotational semantics, which maps a term (an environment or a continuation) to an answer. The modular monadic semantics is composed of two parts: *modular semantic building blocks* and *monad transformers*. Semantic building blocks define the monadic semantics of individual source language features. They

Monad transformers define the kernel-level operations in a modular way. Multiple monad transformers can be composed to form the underlying monad used by all the semantic

monad, so is "*t m*", by using the two parameter constructor class MonadTrans:

*lift* :: *m a t m a*

Fig. 1. Environment monad transformer EnvT(Liang,1998).

*rdEnv* = *EnvT* (\*r return r*)

*return a* = *EnvT* (\\_ *return a*)

*inEnv* :: *r m a m a*

*rdEnv* :: *m r*

**instance** MonadTrans (*EnvT r*) **where** *lift m* = *EnvT* (\\_ *m*)

**class** (Monad *m*) EnvMonad *r m* **where**

**2.2 Modular monadic semantics** 

are independent of each other.

combining them in the lifted monad.

environment, respecitively.

building blocks. The crucial property of modular monadic semantics is the division of the monad *m* into a series of monad transformers, each representing a computation. As mentioned in the previous section, monad transformers provide the power to represent the abstract notion of programming language features, but still allow us to access low-level semantic details. The concept of lifting allows us to consider the interactions between various features. In some sense, *monad transformers can be designed once and for all* (Liang, 1998), since they are entirely independent of the language being described. From this view, we can draw the computation of program slicing as an entity that is independent of the language being analyzed. This will be discussed in the next section. Before doing it, we illustrate the modular monadic semantic description of a very simple imperative programming language **W**.

The **W** language considered in this paper is very similar to the language described in ref. (Slonneger & Kurtz, 1995).The abstract syntax and semantic building blocks of the **W** language are provided in Figure 2 and 3 respectively. In Figure 3, the identifier *Fix* denotes a fixpoint operator; *xtdEnv* and *lkpEnv* are the updating and lookup operators of environments Env, respectively; *updSto* and *alloc* are the updating and allocation functions of stores Loc, respectively; *rdEnv* and *inEnv* are the basic operators of the enviornment monad EnvMonad (given in Figure 1); *putValue* and *getValue* are the writting and reading functions of I/O actions, respectively. Following Venkatesh's assumption for expressions in ref. (Venkatesh, 1990), we also assume that the labeled expressions have no side-effects. The expressions, whose syntax is left unspecified for the sake of generality, consist of operations over identifiers and are uniquely labeled. The label is for the entire expression.

Domains:

arg: Arg (Arguments); b: Blk (Blocks); c: Cmd (Commands); d: Dec (Declarations); e: Exp (Expressions); ide: Ide (Identifiers); l: Label (Labels); p: Prg (Programs); t: Type (Types)

Abstract Syntax:

p :: = **program** ide **is** b

b :: = d **begin** c **end**

d :: = **const** ide = l.e | **var** ide : t | d1; d2

c :: = ide := l.e | c1; c2| skip | **read** ide | **write** l.e


Fig. 2. Abstract syntax of the **W** language.

In modular monadic semantics, the monad definition is simply a composition of the corresponding monad transformers, applied to a base monad. In this paper, we use the

Program Slicing Based on Monadic Semantics 47

SliceT. Its definition is given in Figure 4, where *l* denotes a set of labels of expressions that were required to compute the current statement; *st* denotes a slice table Slices whose data

**type** Slices = [(Var, Labels)]

**type** Var = String **type** Labels = [Int]

*updSli* :: (Var, ComptM Labels) Slices ComptM ()

variable with its slice, and to merge two given slice tables into a new one, respectively.

**Theorem 2**. *SliceT l st* is a monad transformer.

Here, [] denotes a table data structure. The three operators, *lkpSli*, *updSli* and *mrgSli*, represent to lookup the slice of a variable in a given Slices table, to update a table Slices through a

In the similar way as ref. (Zhang, 2005, 2007), the following theorems are straightforward. These theorems guarantee the correctness of the definition of the slice monad transformer in

A static slice-monad transformer *SliceT l st*, taking an initial set of labels and a slice table, returns a computation of a pair of the resulting value and the new slice table. The operator *return* returns the given value with the unchanged slice table. The operator >>= takes a monad *m* and a function *k*. It passes the initial set of labels *l* and slice table *st* to the monad *m*; this yields a value *a* paired with an intermediate slice table *st*; function *k* is applied to the value *a*, yielding a monad (*k a*); this yields in *l* and *st* the final result paired with the final

The lifting function *lift* says that a computation in the monad *m* behaves identically in the monad *SliceT l st m* and makes no changes to the slice table. The operation *rdLabel*/*inLabel*  and *getSli*/*setSli* support reading/setting of the parameter *l* and *st* in the static slice-monad

With the use of the transformer *SliceT*, other monads can be easily transformed into the static slice-monad SliceMonad. For instance, we can lift respectively the basic operators of

The static slice for a variable in a program is the collection of all possible computations of values of that variable. In this section, we only consider end slicing for a single variable, i.e. the slicing criterion is <p, v>, where v the variable of interest, and p the end program point. One can easily generalize this to a set of points and a set of variables at each point by taking

The main idea of monadic static slicing algorithms can be briefly stated as follows: for obtaining a static slice, we firstly apply the slice transformer SliceT to semantic building blocks of the program analyzed. It makes the resulting semantic description include

monads StateMonad and EnvMonad through *SliceT* as shown in Figure 4.

*lkpSli* :: Var Slices ComptM Labels

*mrgSli* :: Slices Slices ComptM Slices

**Theorem 1**. *SliceT l st m* is a monad.

structure is defined as follows:

Figure 4.

slice table.

SliceMonad, respectively.

**4. A monadic static slicing algorithm** 

the union of the individual slices (Binkley, 1996).

input/output monad IO as the base monad. We then select some monad transformers, say StateT and EnvT, and apply them to the base monad IO, forming the combined monad ComptM:

$$\mathbf{ComptM} \equiv \mathbf{(EnvT \cdot StateT)} \text{ IO}$$

The environment monad transformer adds an environment to the given monad. The *return* function ignores the environment, while >>= passes the inherited environment to both subcomputations. Equipped with the monad transformers, the resulting monad ComptM can support all of the semantic building blocks in Figure 3, which gives the formal semantic description we expected.

Domains:

*r*: Env (Environments); *loc*: Loc (Stores); *s*: State (States); *v*: Value (Values)

Semantics Functions:

*M* :: Prg ComptM () *M* **program** ide **is** b = *B* b *B* :: Blk ComptM () *B* d **begin** c **end** = {*r D* d ; *inEnv r C* c } *D* :: Dec ComptM Env ; *E* :: Exp ComptM Value *D* **const** ide = l.e = {*v E* l.e ; *r rdEnv*; *xtdEnv* (ide, *v*, *r*) } *D* **var** ide : t = {*loc alloc*; *r rdEnv*; *xtdEnv* (ide, *loc*, *r*)} *D* d1; d2 = {*r rdEnv*; *rinEnv r D* d1 ; *inEnv r D* d2 )} *C* :: Cmd ComptM () *C* ide := l.e = {*v E* l.e ; *r rdEnv*; *loc lkpEnv*(ide, *r*); *updSto*(*loc*, *v*)} *C* c1; c2 = {*C* c1 ; *C* c2 } *C* skip = *return* () *C* **if** l.e **then** c1 **else** c2 **endif** = {*v E* l.e ; *case v of* **TRUE** *C* c1 **FALSE** *C* c2} *C* **while** l.e **do** c **endwhile** = *Fix* (\ *f*  {*v E* l.e ; *case v of* **TRUE** *f C* c **FALSE** *return* () } ) *C* **read** ide = {*loc lkpEnv*(ide, *rdEnv*); *v getValue*; *updSto*(*loc*, *return v*)} *C* **write** l.e = { *v E* l.e ; *putValue v* }

Fig. 3. Semantic building blocks of the **W** language.

#### **3. Static slice monad transformer**

As mentioned above, each monad transformer represents a single notion of computation. Since static program slicing can be viewed as a computation, we can abstract it as a language-independent notion of a computation by using a *static slice-monad transformer* 

input/output monad IO as the base monad. We then select some monad transformers, say StateT and EnvT, and apply them to the base monad IO, forming the combined monad

The environment monad transformer adds an environment to the given monad. The *return* function ignores the environment, while >>= passes the inherited environment to both subcomputations. Equipped with the monad transformers, the resulting monad ComptM can support all of the semantic building blocks in Figure 3, which gives the formal semantic

*M* :: Prg ComptM ()

*B* :: Blk ComptM ()

*D* **const** ide = l.e = {*v E* l.e ; *r rdEnv*; *xtdEnv* (ide, *v*, *r*) }

*D* **var** ide : t = {*loc alloc*; *r rdEnv*; *xtdEnv* (ide, *loc*, *r*)}

*D* d1; d2 = {*r rdEnv*; *rinEnv r D* d1 ; *inEnv r D* d2 )}

*C* **if** l.e **then** c1 **else** c2 **endif** = {*v E* l.e ; *case v of* **TRUE** *C* c1

*C* **while** l.e **do** c **endwhile** = *Fix* (\ *f*  {*v E* l.e ;

*C* ide := l.e = {*v E* l.e ; *r rdEnv*; *loc lkpEnv*(ide, *r*); *updSto*(*loc*, *v*)}

*C* :: Cmd ComptM ()

*B* d **begin** c **end** = {*r D* d ; *inEnv r C* c }

*M* **program** ide **is** b = *B* b

As mentioned above, each monad transformer represents a single notion of computation. Since static program slicing can be viewed as a computation, we can abstract it as a language-independent notion of a computation by using a *static slice-monad transformer* 

*C* **read** ide = {*loc lkpEnv*(ide, *rdEnv*); *v getValue*; *updSto*(*loc*, *return v*)}

ComptM (EnvT StateT) IO

*r*: Env (Environments); *loc*: Loc (Stores); *s*: State (States); *v*: Value (Values)

*D* :: Dec ComptM Env ; *E* :: Exp ComptM Value

**FALSE** *C* c2}

**FALSE** *return* () } )

*case v of* **TRUE** *f C* c

ComptM:

description we expected.

Semantics Functions:

Domains:

Fig. 3. Semantic building blocks of the **W** language.

*C* **write** l.e = { *v E* l.e ; *putValue v* }

*C* c1; c2 = {*C* c1 ; *C* c2 }

*C* skip = *return* ()

**3. Static slice monad transformer** 

SliceT. Its definition is given in Figure 4, where *l* denotes a set of labels of expressions that were required to compute the current statement; *st* denotes a slice table Slices whose data structure is defined as follows:

```
type Var = String 
                 type Labels = [Int] 
                 type Slices = [(Var, Labels)] 
lkpSli :: Var  Slices  ComptM Labels 
updSli :: (Var, ComptM Labels) Slices ComptM () 
mrgSli :: Slices  Slices  ComptM Slices
```
Here, [] denotes a table data structure. The three operators, *lkpSli*, *updSli* and *mrgSli*, represent to lookup the slice of a variable in a given Slices table, to update a table Slices through a variable with its slice, and to merge two given slice tables into a new one, respectively.

In the similar way as ref. (Zhang, 2005, 2007), the following theorems are straightforward. These theorems guarantee the correctness of the definition of the slice monad transformer in Figure 4.

> **Theorem 1**. *SliceT l st m* is a monad. **Theorem 2**. *SliceT l st* is a monad transformer.

A static slice-monad transformer *SliceT l st*, taking an initial set of labels and a slice table, returns a computation of a pair of the resulting value and the new slice table. The operator *return* returns the given value with the unchanged slice table. The operator >>= takes a monad *m* and a function *k*. It passes the initial set of labels *l* and slice table *st* to the monad *m*; this yields a value *a* paired with an intermediate slice table *st*; function *k* is applied to the value *a*, yielding a monad (*k a*); this yields in *l* and *st* the final result paired with the final slice table.

The lifting function *lift* says that a computation in the monad *m* behaves identically in the monad *SliceT l st m* and makes no changes to the slice table. The operation *rdLabel*/*inLabel*  and *getSli*/*setSli* support reading/setting of the parameter *l* and *st* in the static slice-monad SliceMonad, respectively.

With the use of the transformer *SliceT*, other monads can be easily transformed into the static slice-monad SliceMonad. For instance, we can lift respectively the basic operators of monads StateMonad and EnvMonad through *SliceT* as shown in Figure 4.

## **4. A monadic static slicing algorithm**

The static slice for a variable in a program is the collection of all possible computations of values of that variable. In this section, we only consider end slicing for a single variable, i.e. the slicing criterion is <p, v>, where v the variable of interest, and p the end program point. One can easily generalize this to a set of points and a set of variables at each point by taking the union of the individual slices (Binkley, 1996).

The main idea of monadic static slicing algorithms can be briefly stated as follows: for obtaining a static slice, we firstly apply the slice transformer SliceT to semantic building blocks of the program analyzed. It makes the resulting semantic description include

Program Slicing Based on Monadic Semantics 49

expression l.e included in the current expression is computed, the initial set *L* should be transformered by adding the label of expression l.e, i.e., label l, and all sets of labels of

In addition, the *updSli* operation of the Slices type is applied to record the result of the static slicing in program analysis. In case of the language **W**, only when describe the semantics of assignment statement and initial assignment statement within a variable declaration, the

A static slice includes the statements that possibly affect the variable in the slicing criterion. Therefore, for capturing these possible statements, in Figure 5 we ought to add the operator

After the last statement of a program is analyzed, we could obtain, from the result Slices table, the static slice of each single variable (say *var*) of the program, which is the set of

For getting the final result of the static slices, i.e., a syntactically valid subprogram, we - following Venkatesh (Venkatesh, 1990) -- define *Syn*(s, *L*) for language **W** in Figure 6, where s is a W-program analyzed. It guides us how to construct a syntactically valid subprogram of s from the set *L*. It just gives a strategy for returning the final result/subprogram from a given set *L*, so it could be changed to cater to different people's need. For example, if one does not consider variable declaration as part of slices, then one might change the corresponding term

The correctness proofs of our monadic static slicing algorithms can refer to their termination theorem and their consistency with PDG-based slicing algorithms, given in ref. (Zhang,

For more about the algorithm, we now illustrate to use the rules in Figure 5 to compute the static slice w.r.t. <8, sum> of an example **W** program in Figure 7. Its each expression is uniquely labeled through the label (marked in source program) of the place where the

According to the rule/semantics of assignment statements in Figure 5, after the third expression (i.e. "sum := 0") is analyzed, its intermediate set *L* (whose initial value is

Where *T* is the current slice table, including the static slices of the "i" and "sum" variables, written briefly as L(i) and L(sum), respectively. Since this expression is an assignment one, the related data in Slices need to update through *updSli*, i.e. L(sum) = *L* = {3}. Similarly, after

*L* = {4} *L lkpSli* (i, *T*) = {4}; L(i) = {4}

in the above definition of *L* can accurately

 = {3} ) is

(,)

*lkpSli x T*

*L* = {3} *L lkpSli* (sum, *T*) = {3}

 *L* = *lkpSli* (*var*, *getSli*)

expressions influenced by the variables of the expression l.e.

labels of all expressions influenced on the *var* variable:

"**var** ide : t " : "skip"

expression presences. So the fourth expression is "i := 1".

(l.e)

capture control dependences and data dependences related, respectively.

*x Refs*

in Figure 6 as follows:

changed to *L*:

2007). In fact, the term *L* and

the 4th expression is analyzed, we have

corresponding operator *updSli* should be added in as shown in Figure 5.

*mrgSli* into semantic descriptions of conditional statement and loop statement.

```
newtype SliceT l st m a = SliceT {runSliceT :: (l, st)  m (a, st)} 
instance (Monad m)  Monad (SliceT l st m) where
 return a = SliceT (\(_, st) return (a, st) ) 
     m >>= k = SliceT (\(l, st) {(a, st)  runSliceT m (l, st); 
                                        runSliceT (k a) (l, st)} ) 
instance MonadTrans (SliceT l st) where
      lift m = SliceT (\(_, st) {a  m; return (a, st)} ) 
class (Monad m)  SliceMonad l st m where
     rdLabel :: m l 
     inLabel :: l  m a  m a
     getSli :: m st
     setSli :: st  m st
instance (Monad m)  SliceMonad l st (SliceT l st m) where
     rdLabel = SliceT return
     inLabel l m = SliceT (\(_, st) runSliceT m (l, st) ) 
     getSli = SliceT (\(_, st) return (st, st) ) 
     setSli st = SliceT (\(l, _) return (l, st) ) 
instance (EnvMonad r m, MonadTrans (SliceT l st) m) 
   EnvMonad r (SliceT l st m) where
     inEnv r (SliceT l st m) = SliceT (\(l,st) inEnv r (runSliceT m (l,st)) )
     rdEnv = lift rdEnv
instance (StateMonad s m, MonadTrans (SliceT l st) m) 
   StateMonad s (SliceT l st m) where
     update = lift . update
```
Fig. 4. Static slice-monad transformer SliceT.

program slice semantic feature. According to the semantic description, we then compute static slices of each statement in sequence. Finally we will obtain the static slices of all single variables in the program. In fact, with the process of analyzing a program, the Slices table, which includes the current individual program slices for all variables of this program, is modified steadily according to the monadic slicing algorithm.

Concretely, with respect to the example program **W** mentioned in Section 2, Figure 5 gives the main part of our monadic static algorithm. Figure 5 gives the rules of when and how to modify the current slice table. It adds the computation of static slicing into program analysis modularly, with the help of the monad transformers SliceT given in Section 3. SliceT can be composed with other transformers such as EnvT and StateT as follows, and apply them to monad IO, forming the underlying monad ComptM:

ComptM (SliceT StateT EnvT) IO

In Figure 5 (or the monadic slicing algorithm in a sense), for each computation of a labeled expression l.e, there is an intermediate set *L* as follows:

$$L' = \{ \mathbf{l} \} \cup L \cup \bigcup\_{\mathbf{x} \in \operatorname{Rcfs}(\mathbf{l}, \mathbf{e})} \operatorname{lkpSli}(\mathbf{x}, T)$$

where *T* represents the current slice table before analyzing the expression l.e; *Refs*(l.e) denotes the set of variables appeared in l.e. The relation above means that after an

*runSliceT* (*k a*) (*l*, *st*)} )

**newtype** *SliceT l st m a* = *SliceT* {*runSliceT* :: (*l*, *st*)  *m* (*a*, *st*)}

*m* >>= *k* = *SliceT* (\(*l*, *st*) {(*a*, *st*) *runSliceT m* (*l*, *st*);

*lift m* = *SliceT* (\(\_, *st*) {*a m*; *return* (*a*, *st*)} )

**instance** (Monad *m*) SliceMonad *l st* (*SliceT l st m*) **where**

*inLabel l m* = *SliceT* (\(\_, *st*) *runSliceT m* (*l, st*) ) *getSli* = *SliceT* (\(\_, *st*) *return* (*st*, *st*) ) *setSli st* = *SliceT* (\(*l*, \_) *return* (*l*, *st*) ) **instance** (EnvMonad *r m*, MonadTrans (*SliceT l st*) *m*)

**instance** (StateMonad *s m*, MonadTrans (*SliceT l st*) *m*)

*inEnv r* (*SliceT l st m*) *= SliceT* (\(*l*,*st*) *inEnv r* (*runSliceT m* (*l,st*)) )

**instance** (Monad *m*) Monad (*SliceT l st m*) **where** *return a* = *SliceT* (\(\_, *st*) *return* (*a*, *st*) )

**instance** MonadTrans (*SliceT l st*) **where**

*inLabel* :: *l m a m a*

*rdLabel* = *SliceT return*

*rdEnv = lift rdEnv*

*update* = *lift . update* 

*rdLabel* :: *m l* 

*getSli* :: *m st setSli* :: *st m st*

**class** (Monad *m*) SliceMonad *l st m* **where**

program slice semantic feature. According to the semantic description, we then compute static slices of each statement in sequence. Finally we will obtain the static slices of all single variables in the program. In fact, with the process of analyzing a program, the Slices table, which includes the current individual program slices for all variables of this program, is

Concretely, with respect to the example program **W** mentioned in Section 2, Figure 5 gives the main part of our monadic static algorithm. Figure 5 gives the rules of when and how to modify the current slice table. It adds the computation of static slicing into program analysis modularly, with the help of the monad transformers SliceT given in Section 3. SliceT can be composed with other transformers such as EnvT and StateT as follows, and apply them to

In Figure 5 (or the monadic slicing algorithm in a sense), for each computation of a labeled

where *T* represents the current slice table before analyzing the expression l.e; *Refs*(l.e) denotes the set of variables appeared in l.e. The relation above means that after an

*L* = {l} *L*

ComptM (SliceT StateT EnvT) IO

(l.e) (,) *x Refs lkpSli x T*

Fig. 4. Static slice-monad transformer SliceT.

StateMonad *s* (*SliceT l st m*) **where**

EnvMonad *r* (*SliceT l st m*) **where**

modified steadily according to the monadic slicing algorithm.

monad IO, forming the underlying monad ComptM:

expression l.e, there is an intermediate set *L* as follows:

expression l.e included in the current expression is computed, the initial set *L* should be transformered by adding the label of expression l.e, i.e., label l, and all sets of labels of expressions influenced by the variables of the expression l.e.

In addition, the *updSli* operation of the Slices type is applied to record the result of the static slicing in program analysis. In case of the language **W**, only when describe the semantics of assignment statement and initial assignment statement within a variable declaration, the corresponding operator *updSli* should be added in as shown in Figure 5.

A static slice includes the statements that possibly affect the variable in the slicing criterion. Therefore, for capturing these possible statements, in Figure 5 we ought to add the operator *mrgSli* into semantic descriptions of conditional statement and loop statement.

After the last statement of a program is analyzed, we could obtain, from the result Slices table, the static slice of each single variable (say *var*) of the program, which is the set of labels of all expressions influenced on the *var* variable:

$$L = lkp\text{Sli} \text{ (var, getSli)}$$

For getting the final result of the static slices, i.e., a syntactically valid subprogram, we - following Venkatesh (Venkatesh, 1990) -- define *Syn*(s, *L*) for language **W** in Figure 6, where s is a W-program analyzed. It guides us how to construct a syntactically valid subprogram of s from the set *L*. It just gives a strategy for returning the final result/subprogram from a given set *L*, so it could be changed to cater to different people's need. For example, if one does not consider variable declaration as part of slices, then one might change the corresponding term in Figure 6 as follows:

$$\text{\textquotedblleft}\mathbf{var}\text{ ide}:\text{t}\text{\textquotedblright}:\text{\textquotedblleft}\text{skip\textquotedblright}$$

The correctness proofs of our monadic static slicing algorithms can refer to their termination theorem and their consistency with PDG-based slicing algorithms, given in ref. (Zhang, 2007). In fact, the term *L* and (l.e) (,) *x Refs lkpSli x T* in the above definition of *L* can accurately

capture control dependences and data dependences related, respectively.

For more about the algorithm, we now illustrate to use the rules in Figure 5 to compute the static slice w.r.t. <8, sum> of an example **W** program in Figure 7. Its each expression is uniquely labeled through the label (marked in source program) of the place where the expression presences. So the fourth expression is "i := 1".

According to the rule/semantics of assignment statements in Figure 5, after the third expression (i.e. "sum := 0") is analyzed, its intermediate set *L* (whose initial value is ) is changed to *L*:

$$L' = \{ \mathfrak{B} \} \cup L \cup lkpSli \text{ (sum, \, T)} = \{ \mathfrak{B} \} \cup \mathfrak{g} \cup \mathfrak{q} = \{ \mathfrak{B} \}$$

Where *T* is the current slice table, including the static slices of the "i" and "sum" variables, written briefly as L(i) and L(sum), respectively. Since this expression is an assignment one, the related data in Slices need to update through *updSli*, i.e. L(sum) = *L* = {3}. Similarly, after the 4th expression is analyzed, we have

$$L' = \{4\} \cup L \cup lkpSli \text{ (i, } T \text{)} = \{4\}; \text{ L(i) = \{4\}}$$

Program Slicing Based on Monadic Semantics 51

**program** Example **is**

1 **var** sum 2 **var** i **begin**  3 sum := 0 ; 4 i := 1;

5 **while** i < 11 **do** 6 sum := sum + i;

**end** 

7 i := i + 1

8 **write** sum

Because of the loop of While statement, the 5th-7th expressions are analyzed again:

statement (8h), we obtain the final table Slices as follows:

5th: *L* = {5} *L* L(i) = {4, 5, 7}

**endwhile;** 

6th: *L* = {6} *L* (L(sum) L(i)) = {3, 4, 5, 6, 7};

L(sum) = {3, 4, 5, 6, 7}

7th: *L* = {7} *L* L(i) = {4, 5, 7}; L(i) = {4, 5, 7} Now, if the whole loop body is analyzed over again in this way, the slice L(sum) and L(i) will not be changed again, reaching their fixpoint. So after finishing the analysis of the last

L(i) = {4, 5, 7}; L(sum) = {3, 4, 5, 6, 7} According the rules of *Syn*(s, *L*) in Figure 6, we can obtain the final result of static slice w.r.t.

The modular monadic approach mentioned previously is flexible enough that we can easily introduce a new program feature to analysis. In this section, we will illustrate this power by considering an extension of the language **W** with pointers. We shall show how to adapt the implementation to this extension, with a small change in our existing monadic slice

The introduction of a pointer will lead to aliasing problems (Horwitz, 1989; Hind, 1999) (i.e. multiple variables access the same memory location), so we need pointer analysis to obtain the corresponding data dependency information. In order to represent the unbounded data

Fig. 7. An example program.

<8, sum> of the example program.

algorithm.

**5. Extending W language with pointers** 

And keep going for the 5th-7th expressions, we obtain

$$\text{5th: } L' = \{5\} \cup \phi \cup L(i) = \{4, 5\}$$

$$\text{6th: } L'' = \{6\} \cup L' \cup \{\text{L(sum)} \cup L(i)\} = \{3, 4, 5, 6\}; \text{ L(sum)} = \{3, 4, 5, 6\}$$

$$\text{7th: } L'' = \{7\} \cup L' \cup L(i) = \{4, 5, 7\}; \text{ L(i) = \{4, 5, 7\}}$$

```
M :: Prg  ComptM () 
    M program ide is b = return () 
                     B :: Blk  ComptM () 
    B d begin c end = return () 
                     D :: Dec  ComptM Env E :: Exp  ComptM Value 
    D const ide = l.e = { L  rdLabels; T  getSli; 
                         L  {l}  L 
                                            ( l.e )
                                                      (, )
                                       x Refs
                                               lkpSli x T 
                                           ; updSli(ide, L, T ) } 
    D var ide : t = return () 
    D d1; d2 = return () 
                     C :: Cmd  ComptM () 
C ide := l.e = { L  rdLabels; T  getSli; 
               L  {l}  L 
                                  (l.e)
                                            (, )
                             x Refs
                                      lkpSli x T 
                                 ; updSli(ide, L, T ) } 
C c1; c2 = {C c1 ; C c2 } 
C skip = return () 
C if l.e then c1 else c2 endif = { L  rdLabels; T  getSli; 
                             L  {l}  L 
                                                ( l.e )
                                                          (, )
                                           x Refs
                                                   lkpSli x T 
                                               ; 
                             inLabels L C c1 ; T1  getSli; setSli(T); 
                             inLabels L C c2 ; T2  getSli; mrgSli(T1, T2) } 
C while l.e do c endwhile = Fix (f. {L rdLabels; T  getSli; 
                                    L{l}L 
                                                    (l.e)
                                                              (, )
                                               x Refs
                                                        lkpSli x T 
                                                   ; 
                                    f  {inLabels L C c ; T getSli; mrgSli(T, T )}}) 
C read l.ide = {L  rdLabels; T  getSli; L  {l}  L; updSli(ide, L, T)} 
C write l.e = return ()
```
Fig. 5. Monadic semantics of static slicing for **W** programs.

6th: *L* = {6} *L* (L(sum) L(i)) = {3, 4, 5, 6}; L(sum) = {3, 4, 5, 6}

7th: *L* = {7} *L* L(i) = {4, 5, 7}; L(i) = {4, 5, 7}

*D* :: Dec ComptM Env *E* :: Exp ComptM Value

( l.e )

(, )

; *updSli*(ide, *L*, *T* ) }

( l.e )

*x Refs*

*f* {*inLabels L C c* ; *T*

*inLabels L C c*<sup>1</sup> ; *T*1 *getSli*; *setSli*(*T*); *inLabels L C c*<sup>2</sup> ; *T*2 *getSli*; *mrgSli*(*T*1, *T*2) }

*x Refs*

*L*{l}*L*

*lkpSli x T*

*x Refs*

(, )

; *updSli*(ide, *L*, *T* ) }

(, )

*lkpSli x T*

;

(, )

*getSli*; *mrgSli*(*T*, *T*

)}})

*lkpSli x T*

;

(l.e)

*lkpSli x T*

L(i) = {4, 5}

5th: *L* = {5}

*M* :: Prg ComptM ()

*B* :: Blk ComptM ()

*L* {l} *L*

*C* :: Cmd ComptM ()

*x Refs*

*C* **if** l.e **then** c1 **else** c2 **endif** = { *L rdLabels*; *T getSli*;

*C* **while** l.e **do** c **endwhile** = *Fix* (*f*. {*L rdLabels*; *T getSli*;

*C* **read** l.ide = {*L rdLabels*; *T getSli*; *L* {l} *L*; *updSli*(ide, *L*, *T*)}

(l.e)

*L* {l} *L*

*D* **const** ide = l.e = { *L rdLabels*; *T getSli*;

And keep going for the 5th-7th expressions, we obtain

*M* **program** ide **is** b = *return* ()

*B* d **begin** c **end** = *return* ()

*D* **var** ide : t = *return* () *D* d1; d2 = *return* ()

*C* c1; c2 = {*C* c1 ; *C* c2 }

*C* skip = *return* ()

*C* **write** l.e = *return* ()

*C* ide := l.e = { *L rdLabels*; *T getSli*; *L* {l} *L*

Fig. 5. Monadic semantics of static slicing for **W** programs.

#### **program** Example **is**


Fig. 7. An example program.

Because of the loop of While statement, the 5th-7th expressions are analyzed again:

$$\text{5th: } L^{\prime\prime} = \{\text{5}\} \cup L^{\prime} \cup L(\text{i}) = \{\text{4, 5, 7}\}$$

$$\text{6th: } L^{\prime\prime\prime} = \{\text{6}\} \cup L^{\prime\prime} \cup \{\text{L}(\text{sum}) \cup L(\text{i})\} = \{\text{3, 4, 5, 6, 7}\};$$

$$\text{L(sum)} = \{\text{3, 4, 5, 6, 7}\}$$

$$\text{7th: } L^{\prime\prime\prime} = \{\text{7}\} \cup L^{\prime\prime} \cup L(\text{i}) = \{\text{4, 5, 7}\}; \text{L(i)} = \{\text{4, 5, 7}\}$$

Now, if the whole loop body is analyzed over again in this way, the slice L(sum) and L(i) will not be changed again, reaching their fixpoint. So after finishing the analysis of the last statement (8h), we obtain the final table Slices as follows:

$$\mathcal{L}(\mathbf{i}) = \{ \mathbf{4}, \mathbf{5}, \mathbf{7} \}; \ \mathcal{L}(\text{sum}) = \{ \mathbf{3}, \mathbf{4}, \mathbf{5}, \mathbf{6}, \mathbf{7} \}$$

According the rules of *Syn*(s, *L*) in Figure 6, we can obtain the final result of static slice w.r.t. <8, sum> of the example program.

### **5. Extending W language with pointers**

The modular monadic approach mentioned previously is flexible enough that we can easily introduce a new program feature to analysis. In this section, we will illustrate this power by considering an extension of the language **W** with pointers. We shall show how to adapt the implementation to this extension, with a small change in our existing monadic slice algorithm.

The introduction of a pointer will lead to aliasing problems (Horwitz, 1989; Hind, 1999) (i.e. multiple variables access the same memory location), so we need pointer analysis to obtain the corresponding data dependency information. In order to represent the unbounded data

Program Slicing Based on Monadic Semantics 53

The point-to table, PT, is a table of pairs of a single variable and its associated point-to set (a set of variables). It has five operators *getPT*, *setPT*, *lkpPT*, *updPT* and *mrgPT*, which return and set the current table of point-to sets, lookup a point-to set corresponding to a variable in a given table of point-to sets, update some point-to sets corresponding to a list of variables in a given table of point-to sets, and merge two table of point-to sets into one table,

With the pointers, we sometimes need to update the slices or the point-to sets of some variables at the same time, so we extend the operator *xtdSli* for Slices datatype, and the

*xtdSli* :: [Var] Labels Slices ComptM ()

*xtdPT* :: [Var] PtSet PT ComptM ()

Now we can study in depth the assignment statements with pointers. For simplicity, we only consider single dereference of a pointer (e.g. \*x), since multi-dereference (e.g. \*\*x) can be divided into multiple single dereferences. We decompose an assignment into a left-value expression (such as x or \*x), and a right-value expression (also notated as l.e, but may contain \*y or &y). So we need to expand the definition of *Refs*(l.e) in Section 4. The variables appeared in a right-value expression can be divided into three categories: reference

*Refs*(l.e) = {x | x is a reference variable} {y | y is a dereference variable}

 {z | z *lkpPT*(y, *getPT*), where y is a dereference variable} The detail algorithm of *Refs*(l.e) is shown in Figure 8, where the *PtInfo*(l.e) function can

The algorithm in Figure 8 addresses the issues of the reference and point-to information of a right-value expression, and hence facilitates the expansion of the existing slicing algorithm in Figure 5. The final expansion for the static slices of a **W** program with pointers is shown

By introducing point-to analysis to our previous monadic slicing, we presented (in Figure 9) an approach of monadic slicing for a program with pointers. This approach obtained the point-to information through the data-flow iteration. Being different from the traditional methods where the point-to information and slicing are analyzed in two different phases, they are computed in the same phase in our method, by combining the forward monad slicing with data-flow iteration. Instead of recording point-to information for every statement, we only need to record the information for current analysis statements. So our method saves space without losing the precision. In addition, our approach also reserves the excellent properties of compositionality and language-flexibility from the original monadic

in Figure 9, which is generated by adding the bold and blue terms in Figure 5.

variables, dereference variables and address variables. So we have

obtain the point-to information generated by l.e.

respectively.

slicing method.

operator *xtdPT* for PT datatype as follows:

*updPT* :: Var PtSet PT ComptM ()

*mrgPT* :: PT PT ComptM PT

*Syn*(s, *L*) = case s of "**program** ide **is** b " : *if Syn*(b, *L*) = "skip" *then* "skip" *else* "**program** ide **is** b" "d **begin** c **end** " : *if Syn*(d, *L*)=*Syn*(c, *L*)="skip" *then* "skip" *else* "d **begin** c **end**" "**const** ide = l.e " : *if* l *L then* "**const** ide = l.e" *else* "skip" "**var** ide : t " : *if* ide {*var*} *L Re fs* l (l.e) *then* "**var** ide : t " *else* "skip" "d1; d2 " : *Syn*(d1, *L*); *Syn*(d2, *L*) "ide := l.e " : *if* l *L then* "ide := l.e" *else* "skip" "c1; c2 " : *Syn*(c1, *L*); *Syn*(c2, *L*) "skip " : "skip" "**read** ide " : *if* ide {*var*} *L Re fs* l (l.e) *then* "**read** ide" *else* "skip" "**write** l.e " : "skip" "**if** l.e **then** c1 **else** c2 **endif** ": *if* (*Syn*(c1, *L*) = *Syn*(c2, *L*) = "skip") (l *L*) *then* "skip" *else* "**if** l.e **then** *Syn*(c1, *L*) **else** *Syn*(c2, *L*) **endif** " "**while** l.e **do** c **endwhile** " : *if* (*Syn*(c, *L*) = "skip") (l *L*) *then* "skip" *else* "**while** l.e **do** *Syn*(c, *L*) **endwhile**"

Fig. 6. The definition of *Syn*(s, *L*), where *L* = *lkpSli*(*var*, *getDSli*).

structures in a finite way for the presence of pointers, we consider that all point in the same procedure applied to form an array of heap space, and will deal with this array as a whole. The extended algorithm combines the point-to analysis by data-flow iteration with forward monad slicing. With the illumination of this idea, the key issue to be addressed is the solution to assignments. The other statements (such as conditional statements, loop statements, etc.) could be resolved by adding point-to computation to the existing slicing methods. Before going on, we introduce a data structure for point-to analysis.

Similar to the Slices datatype in Section 3, we design an abstract datatype PT for point-to analysis:

> **type** Var = String **type** PtSet = [Var] **type** PT = [(Var, PtSet)] *getPT* :: ComptM PT *setPT* :: PT ComptM PT *lkpPT* :: Var PT ComptM PtSet

"**program** ide **is** b " : *if Syn*(b, *L*) = "skip" *then* "skip" *else* "**program** ide **is** b" "d **begin** c **end** " : *if Syn*(d, *L*)=*Syn*(c, *L*)="skip" *then* "skip" *else* "d **begin** c

> *L Re fs* l

> *L Re fs* l

"**if** l.e **then** c1 **else** c2 **endif** ": *if* (*Syn*(c1, *L*) = *Syn*(c2, *L*) = "skip") (l *L*) *then*

"**while** l.e **do** c **endwhile** " : *if* (*Syn*(c, *L*) = "skip") (l *L*) *then* "skip"

(l.e) *then* "**var** ide : t " *else* "skip"

(l.e) *then* "**read** ide" *else* "skip"

*else* "**if** l.e **then** *Syn*(c1, *L*) **else** *Syn*(c2, *L*) **endif** "

*else* "**while** l.e **do** *Syn*(c, *L*) **endwhile**"

"**const** ide = l.e " : *if* l *L then* "**const** ide = l.e" *else* "skip"

"ide := l.e " : *if* l *L then* "ide := l.e" *else* "skip"

"**var** ide : t " : *if* ide {*var*}

"d1; d2 " : *Syn*(d1, *L*); *Syn*(d2, *L*)

"c1; c2 " : *Syn*(c1, *L*); *Syn*(c2, *L*)

"**read** ide " : *if* ide {*var*}

"skip " : "skip"

"**write** l.e " : "skip"

Fig. 6. The definition of *Syn*(s, *L*), where *L* = *lkpSli*(*var*, *getDSli*).

*Syn*(s, *L*) = case s of

**end**"

"skip"

analysis:

structures in a finite way for the presence of pointers, we consider that all point in the same procedure applied to form an array of heap space, and will deal with this array as a whole. The extended algorithm combines the point-to analysis by data-flow iteration with forward monad slicing. With the illumination of this idea, the key issue to be addressed is the solution to assignments. The other statements (such as conditional statements, loop statements, etc.) could be resolved by adding point-to computation to the existing slicing

Similar to the Slices datatype in Section 3, we design an abstract datatype PT for point-to

*getPT* :: ComptM PT *setPT* :: PT ComptM PT

*lkpPT* :: Var PT ComptM PtSet

methods. Before going on, we introduce a data structure for point-to analysis.

**type** Var = String **type** PtSet = [Var] **type** PT = [(Var, PtSet)] The point-to table, PT, is a table of pairs of a single variable and its associated point-to set (a set of variables). It has five operators *getPT*, *setPT*, *lkpPT*, *updPT* and *mrgPT*, which return and set the current table of point-to sets, lookup a point-to set corresponding to a variable in a given table of point-to sets, update some point-to sets corresponding to a list of variables in a given table of point-to sets, and merge two table of point-to sets into one table, respectively.

With the pointers, we sometimes need to update the slices or the point-to sets of some variables at the same time, so we extend the operator *xtdSli* for Slices datatype, and the operator *xtdPT* for PT datatype as follows:

> *xtdSli* :: [Var] Labels Slices ComptM () *xtdPT* :: [Var] PtSet PT ComptM ()

Now we can study in depth the assignment statements with pointers. For simplicity, we only consider single dereference of a pointer (e.g. \*x), since multi-dereference (e.g. \*\*x) can be divided into multiple single dereferences. We decompose an assignment into a left-value expression (such as x or \*x), and a right-value expression (also notated as l.e, but may contain \*y or &y). So we need to expand the definition of *Refs*(l.e) in Section 4. The variables appeared in a right-value expression can be divided into three categories: reference variables, dereference variables and address variables. So we have

*Refs*(l.e) = {x | x is a reference variable} {y | y is a dereference variable}

{z | z *lkpPT*(y, *getPT*), where y is a dereference variable}

The detail algorithm of *Refs*(l.e) is shown in Figure 8, where the *PtInfo*(l.e) function can obtain the point-to information generated by l.e.

The algorithm in Figure 8 addresses the issues of the reference and point-to information of a right-value expression, and hence facilitates the expansion of the existing slicing algorithm in Figure 5. The final expansion for the static slices of a **W** program with pointers is shown in Figure 9, which is generated by adding the bold and blue terms in Figure 5.

By introducing point-to analysis to our previous monadic slicing, we presented (in Figure 9) an approach of monadic slicing for a program with pointers. This approach obtained the point-to information through the data-flow iteration. Being different from the traditional methods where the point-to information and slicing are analyzed in two different phases, they are computed in the same phase in our method, by combining the forward monad slicing with data-flow iteration. Instead of recording point-to information for every statement, we only need to record the information for current analysis statements. So our method saves space without losing the precision. In addition, our approach also reserves the excellent properties of compositionality and language-flexibility from the original monadic slicing method.

Program Slicing Based on Monadic Semantics 55

*D* :: Dec ComptM Env *E* :: Exp ComptM Value

{ *L rdLabels*; *T getSli*; *P getPT***; ps =** *PtInfo***(l.e); xs =** *lkpPT* **(ide,** *P***);**

(, )

(, )

{ *L rdLabels*; *T getSli*; *P getPT***; ps =** *PtInfo***(l.e); xs =** *lkpPT* **(ide,** *P***);**

(, )

*lkpSli r T*

(, )

*inLabels L C c*<sup>2</sup> ; *T*2 *getSli*; *P***2**  *getPT*; *mrgSli*(*T*1,*T*2); *mrgPT***(***P***1,** *P***2)**}

*lkpSli x T*

*inLabels L C c*<sup>1</sup> ; *T*1 *getSli*; *P***1**  *getPT*; *setSli*(*T*); *setPT***(***P***)**;

; *xtdSli* **(xs,** *L***,** *T***)**; *xtdPT* **(xs, ps,** *P***)**;

; *updSli*(ide, *L*, *T* ); *updPT* **(ide, ps,**

; *xtdSli* **(xs,** *L***,** *T***)**; *xtdPT* **(xs, ps,** *P***)**;

(l.e)

*x Refs*

(l.e)

*x Refs*

*getSli*; *P getPT*; *mrgSli*(*T*, *T*

**if (ide is of the form \*x) then** *xtdSli***(***lkpPT***(x,** *P***),** *L***,** *T***)** 

(, )

*lkpSli x T*

(, )

); *mrgPT***(P, P )** }})

*lkpSli x T*

; *P*

;

; *updSli*(ide, *L*, *T* ); *updPT* **(ide, ps,** *P***)**}

*lkpSli r T*

*lkpSli x T*

*M* :: Prg ComptM ()

*B* :: Blk ComptM ()

(l.e ) {x}

( l.e )

*r Refs*

*x Refs*

*C* :: Cmd ComptM ()

(l.e ) {x}

*r Refs*

*x Refs*

( l.e )

{ *L rdLabels*; *T getSli*; *P getPT*; *L* {l} *L*

*Fix*(*f*. {*LrdLabels*; *TgetSli*; *L*{l}*L*

*C* **read** l.ide = { *L rdLabels*; *T getSli*; *L* {l} *L*; *P getPT*;

**else** *updSli*(ide, *L*, *T*) }

*M* **program** ide **is** b = *return* ()

**if (ide is of the form \*x) then**  *L*  **{l}**  *L*

*L* {l} *L*

**if (ide is of the form \*x) then**  *L*  **{l}**  *L*

*B* d **begin** c **end** = *return* ()

*D* **const** ide = l.e =

**else** 

*D* **var** ide : t = *return* () *D* d1; d2 = *return* ()

> **else**  *L* {l} *L*

*C* c1; c2 = {*C* c1 ; *C* c2 }

*C* **if** l.e **then** c1 **else** c2 **endif** =

*C* **while** l.e **do** c **endwhile** =

*C* **write** l.e = *return* ()

*C* skip = *return* ()

*getPT*;

*P***)**}

*C* ide := l.e =

Fig. 9. Monadic slicing semantics for the **W** language with pointers.

*f* {*inLabels L C c* ; *T*

*Input:* an expression (e.g. e) *Output:* the reference set, *Refs*, and the point-to set, *PtInfo*, of the expression *Algorithm:*  **case** e **of** e is the expression of the form &y *Refs*(e) = ; *PtInfo*(e) = {y} e is the expression of the form \*y Let ys = *lkpPT*(y, *getPT*) where ys is the current point-to set of y; *Refs*(e) = {y} ys; *PtInfo*(e) = v v ys (, ) *lkpPT getPT* e is a pure variable such as y Let ys = *lkpPT*(y, *getPT*); *Refs*(e) = {y}; *PtInfo*(e) = ys e is the compound expression with two sub-expressions e1 and e2 *Refs*(e) = *Refs*(e1) *Refs*(e2); *PtInfo*(e) = *PtInfo*(e1) *PtInfo*(e2) e is a constant of a value type *Refs*(e) = ; *PtInfo*(e) = **end case**;

Fig. 8. The algorithm for the reference set and the point-to set of an expression.

## **6. Implementation and complexity analysis**

In this section, we will implement the monadic slicing algorithms, and analyze its complexity as well.

Because of the use of monad transformers, modular denotational semantics achieves a high level of modularity and extensibility. Despite this, it is still executable: there is a clear operational interpretation of the semantics (Wadler, 1995). In ref. (Liang, 1995, 1998; Wadler, 1995), some modular compilers/interpreters using monad transformers were constructed. On the basis of these works, our monadic approach for static program slicing is feasible. Based on Labra's language prototyping system LPS (Labra et al, 2001), we developed a simple monadic slice prototype MSlicer (for more, see ref. (Zhang, 2007) or its website: https://sourceforge.net/projects/ lps). Its implementation language is Haskell, which is a purely functional language with lazy evaluation. The beauty of laziness allows Haskell to deal with infinite data, because Haskell will only load the data as it is needed, and because the garbage collector will throw out the data after it has been processed. Using higher-order

*Output:* the reference set, *Refs*, and the point-to set, *PtInfo*, of the expression

Let ys = *lkpPT*(y, *getPT*) where ys is the current point-to set of y;

v ys

*Refs*(e) = *Refs*(e1) *Refs*(e2); *PtInfo*(e) = *PtInfo*(e1) *PtInfo*(e2)

(, )

*lkpPT getPT*

; *PtInfo*(e) = {y}

*Refs*(e) = {y} ys; *PtInfo*(e) = v

e is the compound expression with two sub-expressions e1 and e2

Fig. 8. The algorithm for the reference set and the point-to set of an expression.

; *PtInfo*(e) =

In this section, we will implement the monadic slicing algorithms, and analyze its

Because of the use of monad transformers, modular denotational semantics achieves a high level of modularity and extensibility. Despite this, it is still executable: there is a clear operational interpretation of the semantics (Wadler, 1995). In ref. (Liang, 1995, 1998; Wadler, 1995), some modular compilers/interpreters using monad transformers were constructed. On the basis of these works, our monadic approach for static program slicing is feasible. Based on Labra's language prototyping system LPS (Labra et al, 2001), we developed a simple monadic slice prototype MSlicer (for more, see ref. (Zhang, 2007) or its website: https://sourceforge.net/projects/ lps). Its implementation language is Haskell, which is a purely functional language with lazy evaluation. The beauty of laziness allows Haskell to deal with infinite data, because Haskell will only load the data as it is needed, and because the garbage collector will throw out the data after it has been processed. Using higher-order

**6. Implementation and complexity analysis** 

**end case**;

*Input:* an expression (e.g. e)

e is the expression of the form &y

e is the expression of the form \*y

e is a pure variable such as y

e is a constant of a value type

*Refs*(e) =

Let ys = *lkpPT*(y, *getPT*);

*Refs*(e) = {y}; *PtInfo*(e) = ys

*Refs*(e) =

*Algorithm:* 

**case** e **of**

complexity as well.

*M* :: Prg ComptM () *M* **program** ide **is** b = *return* () *B* :: Blk ComptM () *B* d **begin** c **end** = *return* () *D* :: Dec ComptM Env *E* :: Exp ComptM Value *D* **const** ide = l.e = { *L rdLabels*; *T getSli*; *P getPT***; ps =** *PtInfo***(l.e); xs =** *lkpPT* **(ide,** *P***); if (ide is of the form \*x) then**  *L*  **{l}**  *L* (l.e ) {x} (, ) *r Refs lkpSli r T* ; *xtdSli* **(xs,** *L***,** *T***)**; *xtdPT* **(xs, ps,** *P***)**; **else**  *L* {l} *L* ( l.e ) (, ) *x Refs lkpSli x T* ; *updSli*(ide, *L*, *T* ); *updPT* **(ide, ps,** *P***)**} *D* **var** ide : t = *return* () *D* d1; d2 = *return* () *C* :: Cmd ComptM () *C* ide := l.e = { *L rdLabels*; *T getSli*; *P getPT***; ps =** *PtInfo***(l.e); xs =** *lkpPT* **(ide,** *P***); if (ide is of the form \*x) then**  *L*  **{l}**  *L* (l.e ) {x} (, ) *r Refs lkpSli r T* ; *xtdSli* **(xs,** *L***,** *T***)**; *xtdPT* **(xs, ps,** *P***)**; **else**  *L* {l} *L* ( l.e ) (, ) *x Refs lkpSli x T* ; *updSli*(ide, *L*, *T* ); *updPT* **(ide, ps,** *P***)**} *C* c1; c2 = {*C* c1 ; *C* c2 } *C* skip = *return* () *C* **if** l.e **then** c1 **else** c2 **endif** = { *L rdLabels*; *T getSli*; *P getPT*; *L* {l} *L* (l.e) (, ) *x Refs lkpSli x T* ; *inLabels L C c*<sup>1</sup> ; *T*1 *getSli*; *P***1**  *getPT*; *setSli*(*T*); *setPT***(***P***)**; *inLabels L C c*<sup>2</sup> ; *T*2 *getSli*; *P***2**  *getPT*; *mrgSli*(*T*1,*T*2); *mrgPT***(***P***1,** *P***2)**} *C* **while** l.e **do** c **endwhile** = *Fix*(*f*. {*LrdLabels*; *TgetSli*; *L*{l}*L* (l.e) (, ) *x Refs lkpSli x T* ; *P getPT*; *f* {*inLabels L C c* ; *TgetSli*; *P getPT*; *mrgSli*(*T*, *T* ); *mrgPT***(P, P )** }}) *C* **read** l.ide = { *L rdLabels*; *T getSli*; *L* {l} *L*; *P getPT*; **if (ide is of the form \*x) then** *xtdSli***(***lkpPT***(x,** *P***),** *L***,** *T***) else** *updSli*(ide, *L*, *T*) } *C* **write** l.e = *return* ()

Fig. 9. Monadic slicing semantics for the **W** language with pointers.

Program Slicing Based on Monadic Semantics 57

Figure 10 gives the framework of the monadic slicer MSlicer. Figure 11 gives the static slice results for the example program in Figure 7 from our current monadic slicer. The final

In practice, in order to obtain good performance, we choice the Haskell type IntSet instead of the original set type of Labels (i.e. [Int]) in Section 3. The implementation of IntSet is based on big-Endian Patricia trees (Okasaki & Gill, 1998; Morrison, 1968). This data structure performs especially well on binary operations like union and intersection. Many operations have a worst-case complexity of *O***(min(***n***,***W***))**, where *n* is the number of elements; *W* is the number of bits in an Int (32 or 64). This means that the operation can

The measures of system size used below are those associated with the data structure of

In a modular compiler/interpreter, our slice monad transformer could be modularly and safely combined into the semantic buildings, so the complexity analysis is restricted to *L* and *Syn*(s, *L*) of a concrete programming language. In the case of our example language **W**, the intermediate label set *L* can be determined in worst-case time *O*(*v m*), where *v* refers to the number of single variables in the program analyzed; and *m* is the number of labeled expressions in the program. To determine the *Syn*(s, *L*) shown in Figure 6 may cost *O*(min(*m*,*W*)). Therefore the time cost of the predominant part of program slicing in the monadic slicing algorithm is bounded by *O*(*v m n*), where *n* is the number of all labeled expressions appeared (perhaps repeatly) in the sequence of analyzing the program. In addition, an extra time cost *O*(*v* min(*m*,*W*)) needs to get the executable slices of all single variables. Now we can see that the worst-case time of the whole static slice is *O*(*v m n*). Since we finally obtain the static slices of all variables after the last statement is analyzed, the program slice of each variable, on the average, costs *O*(*m n*). In fact, *n* = *O*(*m*2) at worst, for more see the following loop statement shown in Figure 12. So all of its static slices will cost

To analyze the space complexity of the algorithms, we pay our attention to the constructions *Refs*(l.e), Slices, *L* and *L*. We need space *O*(*v v*) and *O*(*v m*) to save *Refs*(l.e) and Slices, respectively. According to the definition of slice monad transformer SliceT in Figure 4, we need more intermediate labels when SliceT is applied to loop statements (e.g. while statements). So it takes the space *O*(*k m*) to save intermediate labels, where *k* refers to the maximal times of analyzing the loop statements in the program (until the slice stabilizes) . The label set *L* will cost the space *O*(*m*). Therefore, the total space cost is *O*(*v v v m k*

By analyzing the complexity of algorithms in Figure 8 and 9, we find that the cost of point-to analysis is less than the cost of slicing. So our expansion algorithm to pointers has no

We have tested the complexity analysis of our monadic static algorithms by using the program with *l* while loop statements shown in Figure 12, which is similar to the while program in ref. (Binkley & Gallagher, 1996). From the results given in Figure 13, we can see that *n* = *O*(*m*2) at worst. This shows that the prototype monadic slicer MSlicer without optimization (such as BDD or SEQUITUR (Zhang et al, 2003) for the slice and trace tables)

results also include the output value, analysis trace, static slice table and CPU time.

become linear in the number of elements with a maximum of *W*.

program slice Slices (which is a Hash table).

the worst-case time *O*(*m*3).

additional complexity.

*m*).

 functions from the libraries, Haskell modules can be written to concisely describe each language feature (Peterson et al, 1997; Thompson, 1996). Features such as arbitrary precision integer arithmetic, list comprehensions, infinite lists, all come in handy for the effective monadic slicing of a large program.

Fig. 10**.** The framework of the prototype MSlicer.


Fig. 11. The slice results of the sample program in Figure 7 from our MSlicer.


Fig. 12. A loop sample with *l* while statements.

 functions from the libraries, Haskell modules can be written to concisely describe each language feature (Peterson et al, 1997; Thompson, 1996). Features such as arbitrary precision integer arithmetic, list comprehensions, infinite lists, all come in handy for the effective

monadic slicing of a large program.

Fig. 10**.** The framework of the prototype MSlicer.

Fig. 12. A loop sample with *l* while statements.

Fig. 11. The slice results of the sample program in Figure 7 from our MSlicer.

*m*  1

> 5 6

5+*l* 6+*l*

10+*l*

var v1 = 1; ⋮ var v5 = 5;

while (1 < 2) do

v1 := v2;

 v5 := v1 endwhile;

while (1 < 2) do

⋮

⋱

**statements** 

Figure 10 gives the framework of the monadic slicer MSlicer. Figure 11 gives the static slice results for the example program in Figure 7 from our current monadic slicer. The final results also include the output value, analysis trace, static slice table and CPU time.

In practice, in order to obtain good performance, we choice the Haskell type IntSet instead of the original set type of Labels (i.e. [Int]) in Section 3. The implementation of IntSet is based on big-Endian Patricia trees (Okasaki & Gill, 1998; Morrison, 1968). This data structure performs especially well on binary operations like union and intersection. Many operations have a worst-case complexity of *O***(min(***n***,***W***))**, where *n* is the number of elements; *W* is the number of bits in an Int (32 or 64). This means that the operation can become linear in the number of elements with a maximum of *W*.

The measures of system size used below are those associated with the data structure of program slice Slices (which is a Hash table).

In a modular compiler/interpreter, our slice monad transformer could be modularly and safely combined into the semantic buildings, so the complexity analysis is restricted to *L* and *Syn*(s, *L*) of a concrete programming language. In the case of our example language **W**, the intermediate label set *L* can be determined in worst-case time *O*(*v m*), where *v* refers to the number of single variables in the program analyzed; and *m* is the number of labeled expressions in the program. To determine the *Syn*(s, *L*) shown in Figure 6 may cost *O*(min(*m*,*W*)). Therefore the time cost of the predominant part of program slicing in the monadic slicing algorithm is bounded by *O*(*v m n*), where *n* is the number of all labeled expressions appeared (perhaps repeatly) in the sequence of analyzing the program. In addition, an extra time cost *O*(*v* min(*m*,*W*)) needs to get the executable slices of all single variables. Now we can see that the worst-case time of the whole static slice is *O*(*v m n*). Since we finally obtain the static slices of all variables after the last statement is analyzed, the program slice of each variable, on the average, costs *O*(*m n*). In fact, *n* = *O*(*m*2) at worst, for more see the following loop statement shown in Figure 12. So all of its static slices will cost the worst-case time *O*(*m*3).

To analyze the space complexity of the algorithms, we pay our attention to the constructions *Refs*(l.e), Slices, *L* and *L*. We need space *O*(*v v*) and *O*(*v m*) to save *Refs*(l.e) and Slices, respectively. According to the definition of slice monad transformer SliceT in Figure 4, we need more intermediate labels when SliceT is applied to loop statements (e.g. while statements). So it takes the space *O*(*k m*) to save intermediate labels, where *k* refers to the maximal times of analyzing the loop statements in the program (until the slice stabilizes) . The label set *L* will cost the space *O*(*m*). Therefore, the total space cost is *O*(*v v v m k m*).

By analyzing the complexity of algorithms in Figure 8 and 9, we find that the cost of point-to analysis is less than the cost of slicing. So our expansion algorithm to pointers has no additional complexity.

We have tested the complexity analysis of our monadic static algorithms by using the program with *l* while loop statements shown in Figure 12, which is similar to the while program in ref. (Binkley & Gallagher, 1996). From the results given in Figure 13, we can see that *n* = *O*(*m*2) at worst. This shows that the prototype monadic slicer MSlicer without optimization (such as BDD or SEQUITUR (Zhang et al, 2003) for the slice and trace tables)

Program Slicing Based on Monadic Semantics 59

dependences and data dependences respectively, which are the base of PDG-based

According to the complexity analysis (in Sections 6) for monadic slicing algorithms, their time complexity of each variable is averagely *O*(*m*3) time. While the intra-procedural slicing algorithms based on dataflow equations can compute a slice in *O*(*v n e*) time, or averagely in *O*(*n e*) time for each variable, where *n* is the number of vertices in the control flow graph (CFG) and *e* the number of edges in CFG (Weiser, 1984). Although the PDGbased algorithms extract slices in linear time (i.e. *O*(V + E), where V and E are the number of vertices and edges in the slice, respectively) after the PDG has been computed, a PDG can be constructed in *O*(*n e* + *n d*) time, where *d* is the number of definitions in the program (Tip, 1995). Here V, *n*, *e* and *d* are the same complexity level of *m*, so the whole time of PDG-

In this paper, we have proposed a new approach for program slicing. We have called it *modular monadic program slicing* as it is based on modular monadic semantics. We have abstracted the computation of program slicing as a language-independence object, *slice monad transformer*. Therefore, the modular monadic slicing has excellent flexibility and reusability properties comparing with the existing program slicing algorithms. The modular monadic slicing algorithm has allowed that program slices could be computed directly on abstract syntax, with no needs to explicitly construct intermediate structures such as data

As the behavior of a program is determined by the semantics of the language, it is reasonable to present the modular monadic program slicing. Furthermore, it is feasible, because modular monadic semantics is executable and some modular

For our future work, we will analyze slicing for programs with special features such as concurrent, object-oriented, exceptions and side-effects, by combining slice monad transformer with existing ones such as concurrent (Papaspyrou, 2001), object-oriented (Labra, 2002), non-determination, exceptions and side-effects (Moggi, 1991; Wadler, 1992, 2003). At the same time, we will improve our prototype of monadic slicers and give more

We thank Dr. J.Labra for his cooperating with us in the implementation of the slicer. This work was supported in part by the National Natural Science Foundation of China (60703086, 60903026, 60873049, 60973046), the Priority Academic Program Development of Jiangsu Higher Education Institutions(PAPD), Qing Lan Project of Jiangsu Province, NSF of Jiangsu Higher Education Institutions of China (10KJB520015),"Climbing"Program of Nanjing University of Posts and Telecommunications of China (NY210009), Open Foundation of Guangxi Key Laboratory of Trusted Software(Guilin University of Electronic Technology).

in the definition of *L* can accurately capture control

the term *L* and

algorithms.

(l.e )

*r Refs*

**8. Conclusions and future work** 

flow graphs or dependence graphs.

**9. Acknowledgments** 

compilers/interpreters have already been existed.

comparisons with other slicing methods in experiments.

( , )

based algorithms (including the PDG-construct time) is also *O*(*m*3) nearly.

*lkpSli r getSli*

can compute slices for large analysis histories in reasonable time and space. (The experiments run on a Pentium Willamette (0.18) Windows PC system with 1GB RAM).

Fig. 13. The relations between CPU time, *t*, and the number of all labeled expressions analyzed in practice, *n*, or the number of labeled expressions in the program, *m*.

## **7. Related work and comparisons**

The original program slicing method was expressed as a sequence of data flow analysis problems (Weiser, 1984). An alternative approach was relied on program dependence graphs (PDG) (Ottenstein & Ottenstein, 1984). Most of the existing slicing methods were evolved from the two approaches. A few program slicing methods focused on the semantics of programs.

G.Canfora et al.'s *conditioned slicing* (Canfora et al, 1998) adds a condition in a slicing criterion. Statements that do not satisfy the condition are deleted from the slice. M.Harman et al.'s *amorphous slicing* (Harman & Danicic, 1997) allows for any simplifying transformations which preserve this semantic projection. These two methods are not really based on formal semantics of a program. P.A.Hauser et.al 's *denotational slicing* (Hausler, 1989; Ouarbya et al, 2002) employs the functional semantics of a program language in the denotational (and static) program slicer. G.A.Venkatesh (Venkatesh, 1991) also took account of denotational slicing with formal slicing algorithms including dynamic and static. This approach is indeed based on the standard denotational semantics of a program language. The language Venkatesh considered is a very simply one without pointers. We have extended it in this paper to a more realistic programming language containing pointers, but take an entirely different approach called *modular monadic slicing*.

Compared with the existing static slicing algorithms, the monadic static-slice algorithm has excellent flexibility, combinability and parallelizability properties, because it has abstracted the computation of static slicing as an independent entity, *static slice-monad transformer*. Our algorithm has allowed that static slices could be computed directly on abstract syntax, with no needs to explicitly construct intermediate structures such as dependence graphs.

In respect of accuracy, in Section 4 or in ref. (Zhang, 2007) we have stated that the slice results of monadic static slicing algorithm are not less precise than PDG-based ones. This is because

can compute slices for large analysis histories in reasonable time and space. (The experiments run on a Pentium Willamette (0.18) Windows PC system with 1GB RAM).

(a) t vs. n (b) n vs. m

*n*

0 1,000,000 2,000,000 3,000,000 4,000,000 5,000,000 6,000,000 7,000,000

> 0 200 400 600 800 1,000 1,200 *m*

*n vs. m*

The original program slicing method was expressed as a sequence of data flow analysis problems (Weiser, 1984). An alternative approach was relied on program dependence graphs (PDG) (Ottenstein & Ottenstein, 1984). Most of the existing slicing methods were evolved from the two approaches. A few program slicing methods focused on the semantics

G.Canfora et al.'s *conditioned slicing* (Canfora et al, 1998) adds a condition in a slicing criterion. Statements that do not satisfy the condition are deleted from the slice. M.Harman et al.'s *amorphous slicing* (Harman & Danicic, 1997) allows for any simplifying transformations which preserve this semantic projection. These two methods are not really based on formal semantics of a program. P.A.Hauser et.al 's *denotational slicing* (Hausler, 1989; Ouarbya et al, 2002) employs the functional semantics of a program language in the denotational (and static) program slicer. G.A.Venkatesh (Venkatesh, 1991) also took account of denotational slicing with formal slicing algorithms including dynamic and static. This approach is indeed based on the standard denotational semantics of a program language. The language Venkatesh considered is a very simply one without pointers. We have extended it in this paper to a more realistic programming language containing pointers, but take an

Compared with the existing static slicing algorithms, the monadic static-slice algorithm has excellent flexibility, combinability and parallelizability properties, because it has abstracted the computation of static slicing as an independent entity, *static slice-monad transformer*. Our algorithm has allowed that static slices could be computed directly on abstract syntax, with

In respect of accuracy, in Section 4 or in ref. (Zhang, 2007) we have stated that the slice results of monadic static slicing algorithm are not less precise than PDG-based ones. This is because

no needs to explicitly construct intermediate structures such as dependence graphs.

Fig. 13. The relations between CPU time, *t*, and the number of all labeled expressions analyzed in practice, *n*, or the number of labeled expressions in the program, *m*.

**7. Related work and comparisons** 

*t vs. n*

0 500,000 1,000,000 1,500,000 2,000,000 2,500,000 *n*

entirely different approach called *modular monadic slicing*.

of programs.

*t* **(sec.)**

the term *L* and (l.e ) ( , ) *r Refs lkpSli r getSli* in the definition of *L* can accurately capture control

dependences and data dependences respectively, which are the base of PDG-based algorithms.

According to the complexity analysis (in Sections 6) for monadic slicing algorithms, their time complexity of each variable is averagely *O*(*m*3) time. While the intra-procedural slicing algorithms based on dataflow equations can compute a slice in *O*(*v n e*) time, or averagely in *O*(*n e*) time for each variable, where *n* is the number of vertices in the control flow graph (CFG) and *e* the number of edges in CFG (Weiser, 1984). Although the PDGbased algorithms extract slices in linear time (i.e. *O*(V + E), where V and E are the number of vertices and edges in the slice, respectively) after the PDG has been computed, a PDG can be constructed in *O*(*n e* + *n d*) time, where *d* is the number of definitions in the program (Tip, 1995). Here V, *n*, *e* and *d* are the same complexity level of *m*, so the whole time of PDGbased algorithms (including the PDG-construct time) is also *O*(*m*3) nearly.

## **8. Conclusions and future work**

In this paper, we have proposed a new approach for program slicing. We have called it *modular monadic program slicing* as it is based on modular monadic semantics. We have abstracted the computation of program slicing as a language-independence object, *slice monad transformer*. Therefore, the modular monadic slicing has excellent flexibility and reusability properties comparing with the existing program slicing algorithms. The modular monadic slicing algorithm has allowed that program slices could be computed directly on abstract syntax, with no needs to explicitly construct intermediate structures such as data flow graphs or dependence graphs.

As the behavior of a program is determined by the semantics of the language, it is reasonable to present the modular monadic program slicing. Furthermore, it is feasible, because modular monadic semantics is executable and some modular compilers/interpreters have already been existed.

For our future work, we will analyze slicing for programs with special features such as concurrent, object-oriented, exceptions and side-effects, by combining slice monad transformer with existing ones such as concurrent (Papaspyrou, 2001), object-oriented (Labra, 2002), non-determination, exceptions and side-effects (Moggi, 1991; Wadler, 1992, 2003). At the same time, we will improve our prototype of monadic slicers and give more comparisons with other slicing methods in experiments.

## **9. Acknowledgments**

We thank Dr. J.Labra for his cooperating with us in the implementation of the slicer. This work was supported in part by the National Natural Science Foundation of China (60703086, 60903026, 60873049, 60973046), the Priority Academic Program Development of Jiangsu Higher Education Institutions(PAPD), Qing Lan Project of Jiangsu Province, NSF of Jiangsu Higher Education Institutions of China (10KJB520015),"Climbing"Program of Nanjing University of Posts and Telecommunications of China (NY210009), Open Foundation of Guangxi Key Laboratory of Trusted Software(Guilin University of Electronic Technology).

Program Slicing Based on Monadic Semantics 61

Wu Z Q; Zhang Y Z & Xu B W (2006). Modular Monadic Slicing in the Presence of Pointers,

Zhang Y Z (2007). A Novel Formal Approach to Program Slicing, *Science in China F:* 

Wadler P & Thiemann P (2003). The Marriage of Effects and Monads, *ACM Transactions on* 

Moggi E (1989). An Abstract View of Programming Languages, *LFCS Report, ECS-LFCS-90-*

Liang S (1998). Modular Monadic Semantics and Compilation, PhD dissertation, University

Wadler P (1995). Monads for Functional Programming, *Lecture Notes on Advanced Functional Programming Techniques*, *LNCS 925*, Springer-Verlag, Berlin, pp. 24-52. Newbern J (2002). All About Monads, http://www.haskell.org/all about\_monads/html

Slonneger K & Kurtz B (1995). *Formal Syntax And Semantics of Programming Language: A Lab* 

Horwitz S; Pfeiffer P & Reps T (1989). Dependence Analysis For Pointer Variables, In:

Hind M; Burke M & Carini P, et al (1999). Interprocedural Pointer Alias Analysis, ACM Transactions on Programming Languages and Systems, 21(4): 848~894. Kahl W (2003). A Modular Interpreter Built with Monad Transformers, *Course Lectures on* 

Labra G; Luengo D & Cueva L (2001). A Language Prototyping System Using Modular

Peterson J; Hammond K & Augustsson L, et al (1997). Report on the Programming

Thompson S (1996). *Haskell: The Craft of Functional Programming*, Addison-Wesley, Harlow,

Okasaki C & Gill A (1998). Fast Mergeable Integer Maps, *Workshop on ML*, September, pp.77-

Morrison D (1968). PATRICIA-- Practical Algorithm To Retrieve Information Coded In

Zhang X; Gupta R; Zhang Y (2003). Precise Dynamic Slicing Algorithms, *Proceedings of the* 

Ottenstein K & Ottenstein M (1984). The program dependence graph in a software development environment, *ACM SIGPLAN Notices*, vol.19, no. 5, pp. 177-184.

*25th International Conference on Software Engineering*, IEEE CS Press, Washington

Alphanumeric, *Journal of the ACM*, vol. 15, no. 4, pp. 514-534.

*Proceedings of the SIGPLAN Conference on Programming Language Design and* 

Monadic Semantics, *Workshop on Language Definitions, Tools and Applications*,

Language Haskell 1.4, A Non-strict Purely Functional Language, *Yale University* 

http://www.lfcs.informatics.ed.ac.uk/reports/90/ECS-LFCS- 90-113/

Venkatesh G (1990). Semantics of Program Slicing, *Bellcore TM-ARH-018561*.

*Computational Science. LNCS 3994*. Reading: Springer-verlag, pp.748-756. Zhang Y Z; Labra G & del R (2006). A Monadic Program Slicer, *ACM SIGPLAN Notices*, vol.

41, no. 5, pp. 30-38.

of Yale, Yale.

/index.html

*Information Sciences*, vol. 50, no. 5, pp. 657-670.

*Computational Logic*, vol. 4, no. 1, pp. 1-32.

*113*, University of Edinburgh,

*Based Approach*. Addison & Wesley.

*Functional Programming*, CAS 781.

*Technical Report, YALEU/DCS /RR-1106*.

*Implementation*, pp. 28-40.

*LDTA'01*, Netherlands.

England.

DC, pp. 319-329.

86.

*In: Alexandrov V N, Albada G D, Sloot P M, et al, eds. 6th International Conference on* 

#### **10. References**


Weiser M (1984). Program Slicing. *IEEE Transaction on Software Engineering*, vol. 16, no. 5, pp.

Kamkar M (1995). An Overview and Comparative Classification of Program Slicing Techniques. *Journal of Systems and Software*, vol. 31, no. 3, pp. 197-214. Tip F (1995). A Survey of Program Slicing Techniques. *Journal of Programming Languages*,

Harman M & Hierons R (2001). An Overview of Program Slicing. *Software Focus*, vol. 2, no.

Hausler P (1989). Denotational Program Slicing. *Proceeding of 22th Annual Hawaii* 

Ouarbya L; Danicic S & Daoudi M, et al (2002). A Denotational Interprocedural Program

Venkatesh G (1991). The Semantic Approach to Program Slicing. *ACM SIGPLAN Conference on Programming Language Design and Implementation*, Toronto, Canada, pp. 26-28. Moggi E (1991). Notions of Computation and Monads. *Information and Computation*, vol. 93,

Liang S & Hudak P (1996). Modular Denotational Semantics for Compiler Construction.

Wansbrough K (1997). A Modular Monadic Action Semantics. Master thesis, University of

Mosses P (1998). Semantics, Modularity, and Rewriting Logic. *Proceeding of 2nd International* 

Zhang Y Z & Xu B W (2004). A Survey of Semantic Description Frameworks for Programming Languages. *ACM SIGPLAN Notices*, vol. 39, no. 3, pp.14-30. Wadler P (1992). Comprehending monads. *Mathematical Structures in Computer Science*, vol.

Zhang Y Z; Xu B W & Shi L, et al (2004). Modular Monadic Program Slicing, *The 28th Annual* 

Zhang Y Z; Xu B W & Labra G (2005). A Formal Method for Program Slicing, *Australian Software Engineering Conference*, *ASWEC'05*, Australian, pp. 140-148. Zhang Y Z & Xu B W (2005). A Slice Monad Transformer and Its Applications in Program

Espinosa D (1995), "Semantic Lego", PhD dissertation, Columbia University, Columbia. Liang S; Hudak P & M. Jones (1995). Monad Transformers and Modular Interpreters, *22nd* 

Slicer. *Proceeding of 9th IEEE Working Conference on Reverse Engineering*, IEEE Press,

*Proceeding of 6th European Synposium on Programming Languages and Systems*,

*Workshop on Rewriting Logic and its Applications, ENTCS 15*, Elesvier Press,

*ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages*,

*International Computer Software and Applications Conference, COMPSAC 2004*, Hong

Slicing, *The 10th IEEE International Conference on Engineering of Complex Computer* 

Binkley D & Gallagher K (1996). Program Slicing. *Advances in Computers*, vol. 43, pp. 1-50. Gallagher K & Lyle J (1991). Using Program Slicing in Software Maintenance. *IEEE* 

*Transactions on Software Engineering*, vol. 17, no. 8, pp. 751-761.

*International Conference on System Sciences*, vol. 2, pp. 486-495.

*ESOP'96*. *LNCS 1058*, Springer-Verlag, Berlin, pp. 219-234.

*POPL'95*, ACM Press, New York, pp. 333-343.

*Systems,* ICECCS'05, Shanghai, China, pp. 147-154.

**10. References** 

498-509.

3, pp. 85-92.

pp. 55-92.

vol.3, no.3, pp.121-189.

Virginia, pp. 181-189.

Auckland, Auckland.

Netherlands.

2, pp. 461-493.

Kong, China, pp. 66-71.


http://www.lfcs.informatics.ed.ac.uk/reports/90/ECS-LFCS- 90-113/


**3** 

*Brazil* 

**CCMF, Computational Context Modeling** 

**to Develop Context-Aware Web Applications** 

The purpose of software is to help people to perform their activities and fulfill their objectives. In this regard, the human-software relationship could be enhanced if software could adapt to changes automatically during its utilization (Brézillon, 1999). Context is defined by (Dey, 2001) as any type of information which characterizes an entity. An entity is any person, place or object that is relevant to the interaction between users and software. According to (Dey & Abowd, 1999) context-awareness is the capability of software to use context to offer services to users. For instance, a context-aware system may trigger an alarm in a device near to a user to remind him of the departure time of a planned trip (Yamato et al. 2011). In this example, context is used to provide a service to the user: the reminding of a personal activity. It is not, however, a trivial task to associate context with software in the same level of abstraction as humans do when they communicate with each other. (Dey & Abowd, 1999) considers that such ability is naturally inherited from the richness of the human languages and the common understanding of how the world works and from an implicit understanding of daily situations. Thus, with the intention of enhancing the offering of services from software to humans, it is important to transmit these capacities to

As maintained by (Sheng & Benatallah, 2005), web services have become a promising technology for the development of internet-oriented software. Services are autonomous platform-independent software that executes tasks ranging from providing simple answers to users requests to the execution of complex processes. Web services are services that utilize the internet and its open technologies, e.g. WSDL, Web Service Description Language, SOAP, Simple Object Access Protocol, UDDI, Universal Description, Discovery and Integration, and XML, eXtensible Markup Language, to supply functionalities to other applications (Berbner et al. 2005). (Kapitsaki et al. 2009) assure that the handling of context is of vital importance to web services, since it promotes dynamic behavior, content adaptation and simplicity of use to end users. However, the association between web services and context is not easy to achieve because adequate mechanisms are not offered to developers in order to support the description and representation of context-related information and its

**1. Introduction** 

computational environments.

later utilization by web services.

**Framework – An Ontological Approach** 

*1UNIFACS-University of Salvador/IRT-Instituto Recôncavo de Tecnologia* 

Luis Paulo Carvalho1 and Paulo Caetano da Silva2

*2UNIFACS-University of Salvador* 


## **CCMF, Computational Context Modeling Framework – An Ontological Approach to Develop Context-Aware Web Applications**

Luis Paulo Carvalho1 and Paulo Caetano da Silva2 *1UNIFACS-University of Salvador/IRT-Instituto Recôncavo de Tecnologia 2UNIFACS-University of Salvador Brazil* 

## **1. Introduction**

62 Semantics in Action – Applications and Scenarios

Canfora G; Cimitile & De L (1998). Conditioned Program Slicing, *Information and Software* 

Harman M & Danicic S (1997). Amorphous Program Slicing, *IEEE International Workshop on Program Comprehesion*, *IWPC'97*, IEEE CS Press, Los Alamitos, pp. 70-79. Papaspyrou N (2001). A Resumption Monad Transformer and its Applications in the

Labra G; Luengo D & Cueva L, et al (2002). Reusable Monadic Semantics of Object Oriented

Semantics of Concurrency, *Technical Report CSD-SW-TR-2-01*, National Technical

Programming Languages, *Proceeding of 6th Brazilian Symposium on Programming* 

*Technology*, vol. 40, no. 11/12, pp. 595-607.

University of Athens, Software Engineering Laboratory.

*Languages*, *SBLP'02*, PUC-Rio University, Brazil.

The purpose of software is to help people to perform their activities and fulfill their objectives. In this regard, the human-software relationship could be enhanced if software could adapt to changes automatically during its utilization (Brézillon, 1999). Context is defined by (Dey, 2001) as any type of information which characterizes an entity. An entity is any person, place or object that is relevant to the interaction between users and software. According to (Dey & Abowd, 1999) context-awareness is the capability of software to use context to offer services to users. For instance, a context-aware system may trigger an alarm in a device near to a user to remind him of the departure time of a planned trip (Yamato et al. 2011). In this example, context is used to provide a service to the user: the reminding of a personal activity. It is not, however, a trivial task to associate context with software in the same level of abstraction as humans do when they communicate with each other. (Dey & Abowd, 1999) considers that such ability is naturally inherited from the richness of the human languages and the common understanding of how the world works and from an implicit understanding of daily situations. Thus, with the intention of enhancing the offering of services from software to humans, it is important to transmit these capacities to computational environments.

As maintained by (Sheng & Benatallah, 2005), web services have become a promising technology for the development of internet-oriented software. Services are autonomous platform-independent software that executes tasks ranging from providing simple answers to users requests to the execution of complex processes. Web services are services that utilize the internet and its open technologies, e.g. WSDL, Web Service Description Language, SOAP, Simple Object Access Protocol, UDDI, Universal Description, Discovery and Integration, and XML, eXtensible Markup Language, to supply functionalities to other applications (Berbner et al. 2005). (Kapitsaki et al. 2009) assure that the handling of context is of vital importance to web services, since it promotes dynamic behavior, content adaptation and simplicity of use to end users. However, the association between web services and context is not easy to achieve because adequate mechanisms are not offered to developers in order to support the description and representation of context-related information and its later utilization by web services.

CCMF, Computational Context Modeling Framework –

aware web services;

software requirements;

services.

CMS-ANS

CONON

CAMEL

ContextUML

CEManTIKA Case

(Bastida et al. 2008)

(Yamato et al. 2010)

(George and Ward 2008)

CMP

An Ontological Approach to Develop Context-Aware Web Applications 65

1. CMS-ANS (Bonino et al. 2007), Context Management Service/Awareness and Notification Service, a framework that allows context sources to publish information

2. CONON (Wang et al. 2004), CONtext Ontology, a two-layered ontology intended to

3. CMP (Simons, 2007), Context Modeling Profile, uses stereotypes to extend the class diagram of UML to model context. In the same way, ContextUML (Sheng & Benatallah, 2005) adds stereotypes to the UML (in specific, to the class diagram) to model context-

4. CEMantTIKA Case (Patrício, 2010), composed of a set of customized diagrams - based on the Eclipse platform and JBoss Drools (JBDrools, 2009) - that model context

5. (Bastida et al. 2008) proposes WSBPEL (WSBPEL, 2007), Web Service Business Process Execution Language, to model adaptation based on context information extracted from

6. (George & Ward, 2008) modify the WSBPEL engine to support the addition of context variables and sources (i.e. the sources are used to fill information in the variables); 7. CAMEL (Grassi & Sindico 2009), Context-Awareness ModEling Language, composed of UML-oriented diagrams made specifically to model context structures and adaptation; 8. (Yamato et al. 2010) proposes dynamic adaptations of composite web services utilizing semantic context metadata to select equivalent functionalities from clusters of web

Table 2 shows the result of the evaluation of the works against the proposed requirements

Table 2. Evaluation of related works against the requirements (Carvalho & Silva, 2011).

I II III IV V VI

and client software to be notified when specific contexts are acquired;

promote the sharing of context structures among agents and services;

structures and adaptations of context-aware software;

(filled cells indicate that the requirement was fulfilled).

Considering the lack of appropriate technologies to extend software to integration of context and web services, this work proposes:


This work is organized as follows: Section 2 presents related works. The framework is described in Section 3. In Section 4, CCMF is coupled with CCMD and ontologies to develop a context-aware web application (both approaches are compared). Section 5 enfolds conclusions and future works.

## **2. Related works**

(Carvalho & Silva, 2011) gathered requirements from related literature - e.g. (Topcu 2011), (Hoyos et al. 2010), (Dey et al. 2001), (Vieira, 2008), (Bulcão, 2006), (Schmidt, 2006) - with the intention of enumerating important characteristics for the modeling of the information which influences the utilization of software and to provide guidelines to base the creation of the framework. Table 1 lists the requirements.


Table 1. Requirements for developing context-aware software (Carvalho & Silva, 2011)

The following works were evaluated against the requirements (listed in Table 1):

Considering the lack of appropriate technologies to extend software to integration of context

1. A framework, CCMF (Carvalho & Silva, 2011), Computational Context Modeling Framework, which relies on the reuse of artifacts and tools to automate analysis and

3. The analysis of the coupling between the framework and the 2 targeted technologies (CCMD and Ontologies) in order to evaluate the advantages and disadvantages of each

This work is organized as follows: Section 2 presents related works. The framework is described in Section 3. In Section 4, CCMF is coupled with CCMD and ontologies to develop a context-aware web application (both approaches are compared). Section 5 enfolds

(Carvalho & Silva, 2011) gathered requirements from related literature - e.g. (Topcu 2011), (Hoyos et al. 2010), (Dey et al. 2001), (Vieira, 2008), (Bulcão, 2006), (Schmidt, 2006) - with the intention of enumerating important characteristics for the modeling of the information which influences the utilization of software and to provide guidelines to base the creation of

phase

the model

Table 1. Requirements for developing context-aware software (Carvalho & Silva, 2011)

The following works were evaluated against the requirements (listed in Table 1):

modeling of structures)

The separation into two phases promotes the decoupling of aims, allowing designers to focus on specific activities related to each development

By modeling the context focus and dimensions, designers are able to orderly identify and structure context information, promoting the readability of

The effort required to perform next development steps (e.g. modeling of adaptation) is lessened by the input of artifacts from previous phases (e.g.

So that designers can model context-aware systems

To base context adaptation on web services API,

to automate tasks of a variety of scenarios

e.g. Google Agenda API (GAgenda, 2011)

development activities related to the making of context-aware web applications; 2. The instantiation of CCMF considering 2 different case studies: (a) Integration of CCMF and CCMD (Carvalho & Silva, 2011), Computational Context Modeling Diagram (to be presented in Section 3.1); (b) The embedding of ontologies in CCMF. Both cases

intending to enable the development of context-aware web applications;

and web services, this work proposes:

approach.

**2. Related works** 

and adaptation

dimensions

phases

I

conclusions and future works.

the framework. Table 1 lists the requirements.

It must support the development of contextaware software as a two-phase task: specification of context information (structure)

II It must categorize the information into context

IV It must support the transfer of context information and artifacts between development

V It must promote the diversity of context information in a domain-independent manner

VI It has to support the reuse of distributed computing systems such as services

III It has to identify the context focus of a task

Requirement Purpose


Table 2 shows the result of the evaluation of the works against the proposed requirements (filled cells indicate that the requirement was fulfilled).


Table 2. Evaluation of related works against the requirements (Carvalho & Silva, 2011).

CCMF, Computational Context Modeling Framework –

An Ontological Approach to Develop Context-Aware Web Applications 67

Fig. 1. CCMF – Computational Context Modeling Framework (Carvalho & Silva, 2011)

1. Modeling structures of context information (upper layer of the framework) and the

2. Enabling the reuse of development artifacts, e.g. by transforming a context medium (a XSD document) into XML-based classes in order to serialize complex context data

3. Supporting the reuse of distributed computing systems such as web services. In this case, the adaptation mechanism is placed "in the cloud" and it can be reused by other

In Section 3.1, it is described a diagram, CCMD, Computational Context Modeling Diagram, which automates the execution of activity number 1 of CCMF, the modeling of context structures, i.e. CCMD can be coupled with the upper layer of CCMF to define the sets of

Provided the activities depicted in Figure 1, CCMF is capable of:

adaptation (lower layer of the framework);

computational agents over the internet.

between web services;

Since the aforementioned related works do not fulfill all of the requirements (described in Table 1), CCMF is proposed as set of activities intended to automate the analysis and development of context-aware web applications. In Section 3, the framework is described and it is discussed its association with reusable development languages, tools and artifacts. A case study in which CCMF is applied is presented in Section 4.

## **3. CCMF, Computational Context Modeling Framework**

CCMF is composed of a set of analysis and development activities which are intended to lessen the effort demanded by the development of context-aware web applications. This is achieved from: (a) the reuse of artifacts and third-part tools, modeling languages and technologies and (b) the automation of the execution of targeted activities, which are described in next paragraphs.

As shown in Figure 1, the framework is composed of two layers containing specific activities to be carried by developers. Its upper layer comprises the activities related to the definition of structures of context information and its externalization to the adaptation modeling mechanism, a WSBPEL diagram. Such externalization is made possible by the transformation of the context structures into a context medium. Provided that XML-based languages enable the interoperation of computational agents, e.g web services (Alboaie et al. 2003), XSD (XSD, 2001), XML Schema Definition, documents are used by the framework to enable the utilization of context by web services integrated to the WSBPEL diagram. Respectively, the definition of context structures and the transformation of these structures into context mediums are performed by the activities identified by numbers 1 and 2.

Once the context medium is created, it can be transformed into language-specific context classes. Considering, for instance, JAVA as the development language, XMLBeans API (XMLBeans, 2009) can be used to transform the XSD schema into JAVA serializable classes. The resulting classes can be instantiated as objects that are capable of encapsulating their attributes into XML documents. Later on, web services can rely on such documents to exchange complex context data between each other, i.e. the serialization via XML documents is necessary to interoperate web services in a manner that the information about context is used to parameterized the adaptation. The generation of the serializable classes is performed by the activity number 3.

After having modeled the context structures, the developer must define how the context information must be used to automate adaptations. This activity (identified by number 4), the first one of the framework´s lower layer, depends on WSBPEL to base the context adaptation on web services. In this case, web services must be gathered and integrated to a WSBPEL diagram in order to utilize context information to parameterize responses to situations of use. Along with the deployment of the composite context-aware web service (by activity number 5), a WSDL document is created. This document describes the web service with the purpose of allowing the remote calling of its functionalities by other computational agents (e.g. handheld-embedded applications, other web services). To ease the effort required by the creation of these agents, the WSDL document can be transformed into language-specific source code (by executing the activity identified by the number 6). The source code is intended to provide ways to client software to access the composite web service in order to be served by adaptation functionalities.

Since the aforementioned related works do not fulfill all of the requirements (described in Table 1), CCMF is proposed as set of activities intended to automate the analysis and development of context-aware web applications. In Section 3, the framework is described and it is discussed its association with reusable development languages, tools and artifacts.

CCMF is composed of a set of analysis and development activities which are intended to lessen the effort demanded by the development of context-aware web applications. This is achieved from: (a) the reuse of artifacts and third-part tools, modeling languages and technologies and (b) the automation of the execution of targeted activities, which are

As shown in Figure 1, the framework is composed of two layers containing specific activities to be carried by developers. Its upper layer comprises the activities related to the definition of structures of context information and its externalization to the adaptation modeling mechanism, a WSBPEL diagram. Such externalization is made possible by the transformation of the context structures into a context medium. Provided that XML-based languages enable the interoperation of computational agents, e.g web services (Alboaie et al. 2003), XSD (XSD, 2001), XML Schema Definition, documents are used by the framework to enable the utilization of context by web services integrated to the WSBPEL diagram. Respectively, the definition of context structures and the transformation of these structures

into context mediums are performed by the activities identified by numbers 1 and 2.

Once the context medium is created, it can be transformed into language-specific context classes. Considering, for instance, JAVA as the development language, XMLBeans API (XMLBeans, 2009) can be used to transform the XSD schema into JAVA serializable classes. The resulting classes can be instantiated as objects that are capable of encapsulating their attributes into XML documents. Later on, web services can rely on such documents to exchange complex context data between each other, i.e. the serialization via XML documents is necessary to interoperate web services in a manner that the information about context is used to parameterized the adaptation. The generation of the serializable classes is performed

After having modeled the context structures, the developer must define how the context information must be used to automate adaptations. This activity (identified by number 4), the first one of the framework´s lower layer, depends on WSBPEL to base the context adaptation on web services. In this case, web services must be gathered and integrated to a WSBPEL diagram in order to utilize context information to parameterize responses to situations of use. Along with the deployment of the composite context-aware web service (by activity number 5), a WSDL document is created. This document describes the web service with the purpose of allowing the remote calling of its functionalities by other computational agents (e.g. handheld-embedded applications, other web services). To ease the effort required by the creation of these agents, the WSDL document can be transformed into language-specific source code (by executing the activity identified by the number 6). The source code is intended to provide ways to client software to access the composite web

service in order to be served by adaptation functionalities.

A case study in which CCMF is applied is presented in Section 4.

**3. CCMF, Computational Context Modeling Framework** 

described in next paragraphs.

by the activity number 3.

Fig. 1. CCMF – Computational Context Modeling Framework (Carvalho & Silva, 2011)

Provided the activities depicted in Figure 1, CCMF is capable of:


In Section 3.1, it is described a diagram, CCMD, Computational Context Modeling Diagram, which automates the execution of activity number 1 of CCMF, the modeling of context structures, i.e. CCMD can be coupled with the upper layer of CCMF to define the sets of

CCMF, Computational Context Modeling Framework –

("Time") dimension is situated nearby the number 7.

Fig. 3. Concrete implementation of CCMD stereotypes (Carvalho & Silva, 2011)

enumerated in Figure 1 are exemplified as well as the reuse of tools and artifacts.

making of context-aware distributed systems due to the following reasons:

**3.2 Ontologies** 

services);

The instantiation of CCMF and its coupling with CCMD is described in Section 4.1. The framework is used to implement a context-aware web application. The modeling activities

As stated by (Noy, 2004; Chen et al. 2004), ontologies are believed to be a key feature in the

1. Ontologies enable the sharing of knowledge by open dynamic agents (e.g. web

2. Ontologies supply semantics for intelligent agents to reason about context information; 3. Ontologies promote the interoperability among devices and computational agents.

An Ontological Approach to Develop Context-Aware Web Applications 69

A concrete implementation of the stereotypes of CCMD can be generated via EuGENia (EuGENia, 2011) plugin to embed the modeling of context in the Eclipse IDE. EuGENia defines a declarative language which abstracts away the details of the coding of diagrams. The declarative metadata of CCMD is transformed by EuGENia into artifacts which input graphic-related data into Eclipse's GMF (GMF, 2010), Graphical Modeling Framework, and EMF (EMF, 2009), Eclipse Modeling Framework. As a result, it is generated a diagram (CCMD) to be used via Eclipse IDE to model context information structures. The graphical elements of CCMD are shown in Figure 3. Next to number 1, it is represented the "Task" from which the context focus is extracted. The "Focus" is placed next to number 2. The "Preference" is modeled by the element next to the number 3. Each "Participant" is symbolized by the element pointed by number 4. The "Location" is positioned near to number 5. The "Activity" is represented by the element next to number 6. The "Temporal"

information which may interfere in situations of use of software. In Section 3.2 ontologies are introduced as a replacement for CCMD, being analyzed what are the necessary adaptations to be applied to the framework in order to enable the use of ontologies to model context-aware web applications.

### **3.1 CCMD – Computational Context Modeling Diagram**

CCMD is composed of a set of stereotypes which allows the creation of diagrams to model concepts related to computational context, e.g. the context focus and dimensions. According to (Brézillon & Pomerol, 1999) the context focus corresponds to a step in the solution of a problem or in a decision making process. (Vieira, 2008) states that the focus can represent a relationship between an agent and a task, in which the agent is responsible for performing the task. For instance, referring to the modeling of software that bases the scheduling of meetings on computational context, the focus might indicate that a secretary (the agent) must perform the task "prepare meeting". The focus is important to context modeling because it enables developers to identify specific sets of context information in relation to the task executed by an agent. Once the focus is identified, the related information can be grouped as context dimensions. As indicated by (Brézillon & Bazire, 2005) the context dimensions enables the categorization of context information and have the main purpose of helping software designers to specify, model and fill information into adequate structures (Bulcão, 2006). CCMD models the following context dimensions identified by (Abowd & Mynatt, 2000):


The task, the context focus and dimensions are illustrated in Figure 2. Number 1 points to the task. The context focus is represented by the stereotype linked to number 2. Number 3 is associated with the stereotypes that represent the context dimensions.

Fig. 2. Stereotypes of CCMD (Carvalho & Silva, 2011).

information which may interfere in situations of use of software. In Section 3.2 ontologies are introduced as a replacement for CCMD, being analyzed what are the necessary adaptations to be applied to the framework in order to enable the use of ontologies to model

CCMD is composed of a set of stereotypes which allows the creation of diagrams to model concepts related to computational context, e.g. the context focus and dimensions. According to (Brézillon & Pomerol, 1999) the context focus corresponds to a step in the solution of a problem or in a decision making process. (Vieira, 2008) states that the focus can represent a relationship between an agent and a task, in which the agent is responsible for performing the task. For instance, referring to the modeling of software that bases the scheduling of meetings on computational context, the focus might indicate that a secretary (the agent) must perform the task "prepare meeting". The focus is important to context modeling because it enables developers to identify specific sets of context information in relation to the task executed by an agent. Once the focus is identified, the related information can be grouped as context dimensions. As indicated by (Brézillon & Bazire, 2005) the context dimensions enables the categorization of context information and have the main purpose of helping software designers to specify, model and fill information into adequate structures (Bulcão, 2006). CCMD

models the following context dimensions identified by (Abowd & Mynatt, 2000): 1. "Location" represents spatial characteristics of the context information;

associated with the stereotypes that represent the context dimensions.

Fig. 2. Stereotypes of CCMD (Carvalho & Silva, 2011).

2. "Temporal" ("Time") comprises any date/time-related information of the context; 3. "Participant" represents entities (other than the agent) which participates in the

4. "Motivation" ("Preferences") is related to the objectives of the agent and participants; 5. "Activity" corresponds to activities performed during the execution of the task.

The task, the context focus and dimensions are illustrated in Figure 2. Number 1 points to the task. The context focus is represented by the stereotype linked to number 2. Number 3 is

context-aware web applications.

execution of the task;

**3.1 CCMD – Computational Context Modeling Diagram** 

A concrete implementation of the stereotypes of CCMD can be generated via EuGENia (EuGENia, 2011) plugin to embed the modeling of context in the Eclipse IDE. EuGENia defines a declarative language which abstracts away the details of the coding of diagrams. The declarative metadata of CCMD is transformed by EuGENia into artifacts which input graphic-related data into Eclipse's GMF (GMF, 2010), Graphical Modeling Framework, and EMF (EMF, 2009), Eclipse Modeling Framework. As a result, it is generated a diagram (CCMD) to be used via Eclipse IDE to model context information structures. The graphical elements of CCMD are shown in Figure 3. Next to number 1, it is represented the "Task" from which the context focus is extracted. The "Focus" is placed next to number 2. The "Preference" is modeled by the element next to the number 3. Each "Participant" is symbolized by the element pointed by number 4. The "Location" is positioned near to number 5. The "Activity" is represented by the element next to number 6. The "Temporal" ("Time") dimension is situated nearby the number 7.

Fig. 3. Concrete implementation of CCMD stereotypes (Carvalho & Silva, 2011)

The instantiation of CCMF and its coupling with CCMD is described in Section 4.1. The framework is used to implement a context-aware web application. The modeling activities enumerated in Figure 1 are exemplified as well as the reuse of tools and artifacts.

### **3.2 Ontologies**

As stated by (Noy, 2004; Chen et al. 2004), ontologies are believed to be a key feature in the making of context-aware distributed systems due to the following reasons:


CCMF, Computational Context Modeling Framework –

An Ontological Approach to Develop Context-Aware Web Applications 71

Fig. 4. O-CCMF – Ontology-Driven Computational Context Modeling Framework

(Table 1).

**4. Case studies** 

requirements defined by (Antoniou et al. 2007):

The two versions of CCMF, the one coupled with CCMD and O-CCMF are exemplified as study cases in Section 4. They are used to develop the same context-aware web application. Later on, both approaches are evaluated against the requirements identified in Section 2

As a proof of concept of the application of CCMF and O-CCMF, it is proposed the creation of a context-aware meeting alert application. The frameworks are used to model and develop an application that must send alerts to participants of meetings according to the

Considering that the abovementioned advantages ensure that ontologies side with the purpose of CCMF, which is to promote the reuse and interoperability of distributed computational agents (e.g. web services) with the intention of automating the creation of context-aware web applications, it is proposed the coupling of CCMF with an ontological approach. In this case, it must be surveyed how ontologies are capable of supporting the representation of context structures and the generation of context-aware adaptations mechanisms.

(Bontas et al. 2005) defines ontology reuse as the process in which available (ontological) knowledge is used to generate new ontologies. By reusing existent ontologies the cost of implementation is reduced, since it avoids the manual codification of a new one. Moreover, two different ontologies can be bound together as one to represent concepts of broader domains, i.e. a given ontology can be associated with others with the intention of modeling concepts of a domain in order to represent the sum of the information represented by each of the combined ontologies. Therefore, the framework must be evolved as to allow the collecting and binding of ontologies with the intention of supplying structures of information to model computational context.

Another aspect of ontologies that motivates modifications on the development activities of CCMF is related to their capability of having a dual purpose (Reinisch et al. 2008): ontologies are able to represent knowledge and also to store and generate instances of such knowledge to interoperate agents. In comparison with the set of activities of the former framework (Figure 1), the handling of CCMD and the XSD/XML-based artifacts can be replaced by ontologies, because they can be accessed directly by the web services of the context-aware composition in order to enable the saving and retrieval of the context information. As a consequence, the effort required to model a context-aware composite web service is lessened, because it is not necessary to deal with the instantiation and manipulation of serializable objects and XML documents, i.e. the utilization of ontologies causes the removal and substitution of activities from the original framework (the one illustrated in Figure 1 of Section 3).

The modified framework is shown in Figure 4. The activity identified by number 1 represents the collecting and binding of existent ontologies in the making of a new one, which must be suitable to model information of the context-aware application's domain. The second activity is that of customizing the ontology to better represent context information. This activity can be exemplified by the definition of associations between natively dissociated classes and/or the addition of new classes and attributes to candidate ontologies. The third activity creates the context adaptation and bases it on the utilization of composite web services. The web services of the composition are able to add and select instantiated individuals from the ontology in substitution to serializations via XML documents. The deployment of the composite web service is performed by activity number 4. The generated WSDL contract is reused in the making of client software by activity number 5.

Thus, the adapted framework, hereafter O-CCMF, Ontology-driven Computational Context Modeling Framework, is able to (re)use ontologies through a smaller set of activities dedicated to the development of context-aware web-applications.

Considering that the abovementioned advantages ensure that ontologies side with the purpose of CCMF, which is to promote the reuse and interoperability of distributed computational agents (e.g. web services) with the intention of automating the creation of context-aware web applications, it is proposed the coupling of CCMF with an ontological approach. In this case, it must be surveyed how ontologies are capable of supporting the representation of context structures and the generation of context-aware adaptations

(Bontas et al. 2005) defines ontology reuse as the process in which available (ontological) knowledge is used to generate new ontologies. By reusing existent ontologies the cost of implementation is reduced, since it avoids the manual codification of a new one. Moreover, two different ontologies can be bound together as one to represent concepts of broader domains, i.e. a given ontology can be associated with others with the intention of modeling concepts of a domain in order to represent the sum of the information represented by each of the combined ontologies. Therefore, the framework must be evolved as to allow the collecting and binding of ontologies with the intention of supplying structures of

Another aspect of ontologies that motivates modifications on the development activities of CCMF is related to their capability of having a dual purpose (Reinisch et al. 2008): ontologies are able to represent knowledge and also to store and generate instances of such knowledge to interoperate agents. In comparison with the set of activities of the former framework (Figure 1), the handling of CCMD and the XSD/XML-based artifacts can be replaced by ontologies, because they can be accessed directly by the web services of the context-aware composition in order to enable the saving and retrieval of the context information. As a consequence, the effort required to model a context-aware composite web service is lessened, because it is not necessary to deal with the instantiation and manipulation of serializable objects and XML documents, i.e. the utilization of ontologies causes the removal and substitution of activities from the original framework (the one

The modified framework is shown in Figure 4. The activity identified by number 1 represents the collecting and binding of existent ontologies in the making of a new one, which must be suitable to model information of the context-aware application's domain. The second activity is that of customizing the ontology to better represent context information. This activity can be exemplified by the definition of associations between natively dissociated classes and/or the addition of new classes and attributes to candidate ontologies. The third activity creates the context adaptation and bases it on the utilization of composite web services. The web services of the composition are able to add and select instantiated individuals from the ontology in substitution to serializations via XML documents. The deployment of the composite web service is performed by activity number 4. The generated WSDL contract is reused in the making of client software by activity

Thus, the adapted framework, hereafter O-CCMF, Ontology-driven Computational Context Modeling Framework, is able to (re)use ontologies through a smaller set of activities

dedicated to the development of context-aware web-applications.

mechanisms.

information to model computational context.

illustrated in Figure 1 of Section 3).

number 5.

Fig. 4. O-CCMF – Ontology-Driven Computational Context Modeling Framework

The two versions of CCMF, the one coupled with CCMD and O-CCMF are exemplified as study cases in Section 4. They are used to develop the same context-aware web application. Later on, both approaches are evaluated against the requirements identified in Section 2 (Table 1).

## **4. Case studies**

As a proof of concept of the application of CCMF and O-CCMF, it is proposed the creation of a context-aware meeting alert application. The frameworks are used to model and develop an application that must send alerts to participants of meetings according to the requirements defined by (Antoniou et al. 2007):

CCMF, Computational Context Modeling Framework –

 <attribute name="title" type="string"></attribute> <attribute name="description" type="string"></attribute>

<complexType name="meeting">

maxOccurs="unbounded"/>

<sequence>

</sequence>

</complexType>

An Ontological Approach to Develop Context-Aware Web Applications 73

Fig. 6. Context medium exported by CCMD as a XSD document (Carvalho & Silva, 2011).

 <element name="location" type="gagenda:location" minOccurs="1" maxOccurs="1"/> <element name="temporal" type="gagenda:temporal" minOccurs="1" maxOccurs="1"/>

<element name="participant" type="gagenda:participant" minOccurs="0"

document which serializes context information related to a meeting.

 description="Meeting professor Paulo Caetano to talk about the dissertation" xmlns:data="http://www.data.agenda.adapters.google.unifacs.edu.br">

<data:meetings title="Meeting professor Paulo Caetano"

Ba, Brasil"> </data:location>

</data:temporal>

</data:participant> </data:meetings>

Fig. 7. XML-based serializable information (Carvalho & Silva, 2011).

used to automate the adaptation of the meeting alert application.

Once the modeling of the context structures is made available, it must be defined how the software is adapted to situations of use. Prior to designing the adaptation, using the BPEL Visual Designer for Eclipse IDE (BPELEclipse, 2010), web services has to be found so to make possible the collecting and processing of the context data. Table 3 lists services API's

<data:location address="Rua Ponciano de Oliveira, 126, Rio Vermelho, CEP 41950-275, Salvador,

<data:temporal start="2011-08-23T17:30:00.000-03:00" end="2011-08-23T18:30:00.000-03:00">

 <data:participant name="luis paulo" email="luispsc@yahoo.com.br"></data:participant> <data:participant name="paulo caetano" email="paulocaetano.dasilva@gmail.com">

The context medium (XSD document) enables the framework to transform the context structure into other formats of reusable artifacts. For instance, in order to interoperate the web services, with the purpose of implementing the context adaptation of the meeting alert software, the XSD document can be converted into serializable JAVA classes (by the XMLBeans API). This transformation corresponds to the third activity of the framework and it generates a library that makes possible the exchanging of context data between web services, e.g. a certain web service fills context information into a serializable object, which automates the creation of a XML document that is serialized toward other web services. By receiving the XML document as an input, the targeted web service deserializes the context information back into a high level JAVA object. Figure 7 illustrates an example of a XML


#### **4.1 Developing the meeting alert application with CCMF**

The first activity of the framework is that of modeling the context information structures. Figure 5 shows a graphical instantiation of CCMD which is used to represent the context data that parameterizes the context adaptation of the meeting alert application. The element next to number 1 represents the task under which a context focus is identified (next to number 2). The focus aids designers in determining the specific set of context information that is necessary to enable the adaptation, i.e. the combination of tasks and focus helps designers to restrain the scope of analysis of context structures. Once the focus is identified, the datasets of context information can be added to CCMD. The meeting is symbolized by the element next to number 3. The location of the meeting is represented by the element next to number 5. The temporal dimension is represented by the element identified by number 6 and contains information about the starting and ending datetime of the meeting. The list of participants is represented by the element next to number 4. Each participant has its own geographic location (latitude/longitude coordinates) which is represented by the element next to number 7. The locations of the participants are used to calculate their distances from the location of the meeting. The preferable weather condition is represented by the element next to number 8.

Fig. 5. Meeting alert context information modeled by CCMD (Carvalho & Silva, 2011)

After using CCMD to define context structures, a context medium must be generated (activity 2 of the framework). The purpose of the context medium is to integrate the framework with different sources of context information (e.g. CCMD). Figure 6 shows an excerpt from a XSD document generated by the CCMD that represents the meeting and its location, its participants and the date/time-related information.

1. If the participant is located near to the place where the meeting is going to happen, he

2. If the participant is located in a place far from where the meeting is going to happen,

3. If the meeting is going to happen in the rush hour, 10 minutes are added to the interval;

5. If the meeting's subject is about giving a class, 30 minutes are added in order to allow

The first activity of the framework is that of modeling the context information structures. Figure 5 shows a graphical instantiation of CCMD which is used to represent the context data that parameterizes the context adaptation of the meeting alert application. The element next to number 1 represents the task under which a context focus is identified (next to number 2). The focus aids designers in determining the specific set of context information that is necessary to enable the adaptation, i.e. the combination of tasks and focus helps designers to restrain the scope of analysis of context structures. Once the focus is identified, the datasets of context information can be added to CCMD. The meeting is symbolized by the element next to number 3. The location of the meeting is represented by the element next to number 5. The temporal dimension is represented by the element identified by number 6 and contains information about the starting and ending datetime of the meeting. The list of participants is represented by the element next to number 4. Each participant has its own geographic location (latitude/longitude coordinates) which is represented by the element next to number 7. The locations of the participants are used to calculate their distances from the location of the meeting. The preferable weather condition is represented by the element

Fig. 5. Meeting alert context information modeled by CCMD (Carvalho & Silva, 2011)

location, its participants and the date/time-related information.

After using CCMD to define context structures, a context medium must be generated (activity 2 of the framework). The purpose of the context medium is to integrate the framework with different sources of context information (e.g. CCMD). Figure 6 shows an excerpt from a XSD document generated by the CCMD that represents the meeting and its

must receive an alerting message 5 minutes before the meeting;

the message must be sent within 40 minutes;

the participant to prepare himself for the class.

**4.1 Developing the meeting alert application with CCMF** 

4. If it is raining, another 10 minutes are added;

next to number 8.


Fig. 6. Context medium exported by CCMD as a XSD document (Carvalho & Silva, 2011).

The context medium (XSD document) enables the framework to transform the context structure into other formats of reusable artifacts. For instance, in order to interoperate the web services, with the purpose of implementing the context adaptation of the meeting alert software, the XSD document can be converted into serializable JAVA classes (by the XMLBeans API). This transformation corresponds to the third activity of the framework and it generates a library that makes possible the exchanging of context data between web services, e.g. a certain web service fills context information into a serializable object, which automates the creation of a XML document that is serialized toward other web services. By receiving the XML document as an input, the targeted web service deserializes the context information back into a high level JAVA object. Figure 7 illustrates an example of a XML document which serializes context information related to a meeting.

```
<data:meetings title="Meeting professor Paulo Caetano" 
 description="Meeting professor Paulo Caetano to talk about the dissertation" 
xmlns:data="http://www.data.agenda.adapters.google.unifacs.edu.br"> 
 <data:location address="Rua Ponciano de Oliveira, 126, Rio Vermelho, CEP 41950-275, Salvador, 
Ba, Brasil"> 
 </data:location> 
 <data:temporal start="2011-08-23T17:30:00.000-03:00" end="2011-08-23T18:30:00.000-03:00"> 
 </data:temporal> 
 <data:participant name="luis paulo" email="luispsc@yahoo.com.br"></data:participant> 
 <data:participant name="paulo caetano" email="paulocaetano.dasilva@gmail.com"> 
</data:participant> 
</data:meetings>
```
Fig. 7. XML-based serializable information (Carvalho & Silva, 2011).

Once the modeling of the context structures is made available, it must be defined how the software is adapted to situations of use. Prior to designing the adaptation, using the BPEL Visual Designer for Eclipse IDE (BPELEclipse, 2010), web services has to be found so to make possible the collecting and processing of the context data. Table 3 lists services API's used to automate the adaptation of the meeting alert application.

CCMF, Computational Context Modeling Framework –

An Ontological Approach to Develop Context-Aware Web Applications 75

to rain (test placed near to number 6), another 10 minutes are added. If the event happens during the rush hour, the interval is increased in 10 minutes (condition evaluated next to number 7). If the event is about giving a class2, another 30 minutes are added to the interval

so that the participant will be able to ready himself in order to give the class.

Fig. 9. Context adaptation based on WSBPEL (Carvalho & Silva, 2011).

2 The description of the event is searched for the word "class" and similars.

web service.

After the modeling of the context adaptation, the BPEL designer is able to generate a WSDL document that externalizes the new generated composite web service to client applications. Figure 10 shows an example of the result of the adaptation supplied by the meeting alert


Table 3. Services API's used to automate the adaptation of the meeting alert application

The XML document in Figure 7 contains data retrieved from a web service based on the Google Agenda API. Figure 8 shows an example of an event added to the user's agenda. Next to number 1, it is shown the title of the event. Its description is placed near to number 2. Next to 3, the event's starting and ending date and time are shown. Event's location is identified by the number 4. The list of participants is placed near to number 5.


Fig. 8. Google Agenda as a source of context information (Carvalho & Silva, 2011)

The WSBPEL diagram illustrated in Figure 9 models a workflow in which the web services of Table 3 are used to automate the context adaptation. The action identified by the number 1 request events from Google Agenda. The location of the event is processed to determine the weather condition in the area where the meeting is going to happen. This is performed by the action identified by number 2. Since the location of the event is not expressed as a latitude-longitude pair, Google Geocoding is executed to translate the address of the event into geographic coordinates (next to number 3). The distance from each participant to the event's location is calculated by Google Directions (near to the number 4). Once all of the context information is retrieved and processed by the web services, the WSBPEL diagram evaluates the amount of time within which the alert messages must be sent to participants. If the participant is located near to event's location1, the message is sent within 5 minutes. If not, the message is sent within 40 minutes (conditional test next to number 5). If it is going

 1 Participants within a radius of 3.000 meters are considered near to event's location.

Google Geocoding (GGCoding, 2011) It converts address-based locations of meetings to geographic coordinates

meeting's location

The XML document in Figure 7 contains data retrieved from a web service based on the Google Agenda API. Figure 8 shows an example of an event added to the user's agenda. Next to number 1, it is shown the title of the event. Its description is placed near to number 2. Next to 3, the event's starting and ending date and time are shown. Event's location is

Table 3. Services API's used to automate the adaptation of the meeting alert application

identified by the number 4. The list of participants is placed near to number 5.

Fig. 8. Google Agenda as a source of context information (Carvalho & Silva, 2011)

1 Participants within a radius of 3.000 meters are considered near to event's location.

The WSBPEL diagram illustrated in Figure 9 models a workflow in which the web services of Table 3 are used to automate the context adaptation. The action identified by the number 1 request events from Google Agenda. The location of the event is processed to determine the weather condition in the area where the meeting is going to happen. This is performed by the action identified by number 2. Since the location of the event is not expressed as a latitude-longitude pair, Google Geocoding is executed to translate the address of the event into geographic coordinates (next to number 3). The distance from each participant to the event's location is calculated by Google Directions (near to the number 4). Once all of the context information is retrieved and processed by the web services, the WSBPEL diagram evaluates the amount of time within which the alert messages must be sent to participants. If the participant is located near to event's location1, the message is sent within 5 minutes. If not, the message is sent within 40 minutes (conditional test next to number 5). If it is going

It calculates the distance from each participant to to

**API/Service Usage**  Google Agenda It supplies information about meetings Yahoo Weather Forecast It offers information about weather conditions

Google Geodirections API (GDirections, 2011)

to rain (test placed near to number 6), another 10 minutes are added. If the event happens during the rush hour, the interval is increased in 10 minutes (condition evaluated next to number 7). If the event is about giving a class2, another 30 minutes are added to the interval so that the participant will be able to ready himself in order to give the class.

Fig. 9. Context adaptation based on WSBPEL (Carvalho & Silva, 2011).

After the modeling of the context adaptation, the BPEL designer is able to generate a WSDL document that externalizes the new generated composite web service to client applications. Figure 10 shows an example of the result of the adaptation supplied by the meeting alert web service.

 2 The description of the event is searched for the word "class" and similars.

CCMF, Computational Context Modeling Framework –

Fig. 12. Classes of *OWEATHER* (Gajderowicz, 2008).

using CCMD (in Figure 5).

pointed by number 3).

SOUPA

phenomena (e.g Gusting, Rain).

An Ontological Approach to Develop Context-Aware Web Applications 77

the three layers of abstraction of the *OWEATHER* ontology. The upper layer, Class Level 1, contains a top generic Weather class under which grouping classes (e.g. Wind, Precipitation at Class Level 2) are defined. Class Level 3 contains classes that represent specific natural

The union of SOUPA and *OWEATHER* produces a new ontology, MAO (Meeting Alert Ontology), which enables the modeling of the context structures for the meeting alert application. Table 4 relates the classes contained in MAO to the context information defined

Source ontology Ontology classes CCMD classes

The resulting ontology can be populated by information related to meetings. In Figure 13, number 1 points to a "MeetingEvent" individual (instance of the "MeetingEvent" class of MAO) being exhibited by Protégé (Protégé, 2011). Protégé is a free, open-source platform that provides a suite of tools to construct domain models and knowledge-based applications with ontologies. Number 2 identified the class-to-class properties of the event, i.e. its relation with other complex classes of the ontology. The "hasParticipant" element represents an association between an event and its participants, i.e. the individuals "LuisPaulo" and "PauloCaetano" that are instances of the class "Person". The "hasStart" and "hasEnd" elements are related, respectively, to the starting and ending date and time of the event (they are instances of the "Time" class). The "locatedAt" element is intended to represent the spatial location of the event, i.e. the place where the meeting is intended to happen. The ontology also supplies information about the meeting's title and description (elements

**O**WEATHER Weather Weather

Table 4. Ontology classes to represent context information.

Meeting + Event (MeetingEvent) Meeting Location Location Time Temporal Geo-M Geolocation Person Participant

To participant identified by 'luispsc@yahoo.com.br' the alert message must be sent within 50 minutes! To participant identified by 'paulocaetano.dasilva@gmail.com' the alert message must be sent within 15 minutes!

Fig. 10. Adaptation supplied by the meeting alert web service (Carvalho & Silva, 2011)

### **4.2 Coupling CCMF and Ontologies**

The O-CCMF can, as well, be used to develop the meeting alert application. In this section, its activities are performed to re-implement the software.

The first activity of O-CCMF is that of finding the appropriate ontologies to enable the modeling of context information. For instance, in relation to the meeting alert application, SOUPA, Standard Ontology for Ubiquitous and Pervasive Applications, (Chen et al. 2004) can be used for this purpose. SOUPA comprises two sub-ontologies, SOUPA Core and SOUPA Extension, which contain, among others, classes suitable for the representation of meetings. In Figure 11, such classes are highlighted (in yellow).

Fig. 11. Classes of SOUPA (Chen et al. 2004).

Considering the set of context information contained in Figure 5, SOUPA must be coupled with other ontology in order to represent weather conditions, since the adaptation requires the evaluation of such information prior to furnishing context adaptation. With that purpose, the *OWEATHER*, Weather Ontology, (Gajderowicz, 2008) is bound to SOUPA to provide classes which enable the modeling of weather-related information. Figure 12 shows

To participant identified by 'luispsc@yahoo.com.br' the alert message must be sent

To participant identified by 'paulocaetano.dasilva@gmail.com' the alert message

Fig. 10. Adaptation supplied by the meeting alert web service (Carvalho & Silva, 2011)

The O-CCMF can, as well, be used to develop the meeting alert application. In this section,

The first activity of O-CCMF is that of finding the appropriate ontologies to enable the modeling of context information. For instance, in relation to the meeting alert application, SOUPA, Standard Ontology for Ubiquitous and Pervasive Applications, (Chen et al. 2004) can be used for this purpose. SOUPA comprises two sub-ontologies, SOUPA Core and SOUPA Extension, which contain, among others, classes suitable for the representation of

Considering the set of context information contained in Figure 5, SOUPA must be coupled with other ontology in order to represent weather conditions, since the adaptation requires the evaluation of such information prior to furnishing context adaptation. With that purpose, the *OWEATHER*, Weather Ontology, (Gajderowicz, 2008) is bound to SOUPA to provide classes which enable the modeling of weather-related information. Figure 12 shows

**4.2 Coupling CCMF and Ontologies** 

must be sent within 15 minutes!

within 50 minutes!

Fig. 11. Classes of SOUPA (Chen et al. 2004).

its activities are performed to re-implement the software.

meetings. In Figure 11, such classes are highlighted (in yellow).

the three layers of abstraction of the *OWEATHER* ontology. The upper layer, Class Level 1, contains a top generic Weather class under which grouping classes (e.g. Wind, Precipitation at Class Level 2) are defined. Class Level 3 contains classes that represent specific natural phenomena (e.g Gusting, Rain).

Fig. 12. Classes of *OWEATHER* (Gajderowicz, 2008).

The union of SOUPA and *OWEATHER* produces a new ontology, MAO (Meeting Alert Ontology), which enables the modeling of the context structures for the meeting alert application. Table 4 relates the classes contained in MAO to the context information defined using CCMD (in Figure 5).


Table 4. Ontology classes to represent context information.

The resulting ontology can be populated by information related to meetings. In Figure 13, number 1 points to a "MeetingEvent" individual (instance of the "MeetingEvent" class of MAO) being exhibited by Protégé (Protégé, 2011). Protégé is a free, open-source platform that provides a suite of tools to construct domain models and knowledge-based applications with ontologies. Number 2 identified the class-to-class properties of the event, i.e. its relation with other complex classes of the ontology. The "hasParticipant" element represents an association between an event and its participants, i.e. the individuals "LuisPaulo" and "PauloCaetano" that are instances of the class "Person". The "hasStart" and "hasEnd" elements are related, respectively, to the starting and ending date and time of the event (they are instances of the "Time" class). The "locatedAt" element is intended to represent the spatial location of the event, i.e. the place where the meeting is intended to happen. The ontology also supplies information about the meeting's title and description (elements pointed by number 3).

CCMF, Computational Context Modeling Framework –

getEventEntries

#Event

Google Geocoding

Meeting Alert Ontology

( O)

Fig. 14. Using the MAO ontology to interoperate web services.

getEventLocation

#Location

#Geo-M

toGeolocation

manner.

Considering the scenario depicted in Figure 14, the context adaptation is less influenced by the overhead of modeling serializations via XML. In the making of the meeting alert web application, for instance, it was possible to produce web services which took no parameters as input and, likewise, returned no output, because the context information was selected and saved directly to the ontology (no serialization of context information needed). As a consequence, the modeling of the context adaptation can rather rely on choreographed web services in substitution to orchestrations. Figure 15 illustrates the difference between orchestration and choreography (Gábor et al. 2004). The orchestration (left side of Figure 15) requires that the execution of web services is controlled by one agent which describes how services interact with each other. The WSBPEL, for instance, is an orchestrator of web services. In a choreography (right side of Figure 15), each web service involved in the process describes the part they play in the interaction which is performed in a collaborative

Google Directions

The choreography can be exemplified by the interaction between the Geocoding and Directions web services. The Directions web service is able to extract the geographic coordinates inserted into the ontology by the Geocoding web service. This is performed in a i.e. choreographed manner, since the direct manipulation of the meeting alert ontology by

An Ontological Approach to Develop Context-Aware Web Applications 79

Google Agenda

getEventTime

getEventParticipant

#Person

#Weather

Yahoo Weather

getWeatherConditi

#Time

calculateDistanc

#Geo-M

Fig. 13. The "Event" individual of the meeting alert ontology.

Although, it is possible to combine different ontologies in the production of a new one, e.g. the combination of SOUPA and *OWEATHER* produces MAO, it is necessary to adapt the ontology to better express the context information. For instance, Event and Meeting were two dissociated concepts of SOUPA, but, considering the meeting alert application, they had to be combined in a single class, MeetingEvent, so that it would supply a unique way to represent meeting-related events. This adaptation exemplifies the execution of activity 2 of the O-CCMF (as illustrated in Figure 4).

The meeting event information of Figure 13 was retrieved from a web service based on the Google Agenda API and stored in MAO using the OWL API (OWLAPI, 2011). The composite web service (the one illustrated in Figure 9 of Section 4.1) was altered with the intention of using the OWL API and ontologies (SOUPA and *OWEATHER* grouped in MAO) as a replacement for the XML serializable classes (i.e. the JAVA XML Beans classes originated from XSD documents). In this case, the ontologies promote the sharing of a common vocabulary that replaces the serialization of context information via XML documents. As a concrete example of this form of interoperability, Figure 14 illustrates how individuals retrieved from Google Agenda are added to MAO with the intention to parameterize the execution of other web services, e.g. the one originated from the Google Geocoding API. This is done by the "getEventEntries" method that selects event entries found in user's agenda and inserts them in the meeting alert ontology. Once the Geocoding converts the address of the meeting to geographic coordinates, they are compared to the current location of the participants to determine how far/near they are from meeting's location. The location is also used by Yahoo Weather Forecast to evaluate the rain likelihood, i.e. the service analyzes the weather-related information to determine if it is going to rain in the area where the meeting is intended to happen.

Although, it is possible to combine different ontologies in the production of a new one, e.g. the combination of SOUPA and *OWEATHER* produces MAO, it is necessary to adapt the ontology to better express the context information. For instance, Event and Meeting were two dissociated concepts of SOUPA, but, considering the meeting alert application, they had to be combined in a single class, MeetingEvent, so that it would supply a unique way to represent meeting-related events. This adaptation exemplifies the execution of activity 2 of

The meeting event information of Figure 13 was retrieved from a web service based on the Google Agenda API and stored in MAO using the OWL API (OWLAPI, 2011). The composite web service (the one illustrated in Figure 9 of Section 4.1) was altered with the intention of using the OWL API and ontologies (SOUPA and *OWEATHER* grouped in MAO) as a replacement for the XML serializable classes (i.e. the JAVA XML Beans classes originated from XSD documents). In this case, the ontologies promote the sharing of a common vocabulary that replaces the serialization of context information via XML documents. As a concrete example of this form of interoperability, Figure 14 illustrates how individuals retrieved from Google Agenda are added to MAO with the intention to parameterize the execution of other web services, e.g. the one originated from the Google Geocoding API. This is done by the "getEventEntries" method that selects event entries found in user's agenda and inserts them in the meeting alert ontology. Once the Geocoding converts the address of the meeting to geographic coordinates, they are compared to the current location of the participants to determine how far/near they are from meeting's location. The location is also used by Yahoo Weather Forecast to evaluate the rain likelihood, i.e. the service analyzes the weather-related information to determine if it is going to rain in the area where

Fig. 13. The "Event" individual of the meeting alert ontology.

the O-CCMF (as illustrated in Figure 4).

the meeting is intended to happen.

Fig. 14. Using the MAO ontology to interoperate web services.

Considering the scenario depicted in Figure 14, the context adaptation is less influenced by the overhead of modeling serializations via XML. In the making of the meeting alert web application, for instance, it was possible to produce web services which took no parameters as input and, likewise, returned no output, because the context information was selected and saved directly to the ontology (no serialization of context information needed). As a consequence, the modeling of the context adaptation can rather rely on choreographed web services in substitution to orchestrations. Figure 15 illustrates the difference between orchestration and choreography (Gábor et al. 2004). The orchestration (left side of Figure 15) requires that the execution of web services is controlled by one agent which describes how services interact with each other. The WSBPEL, for instance, is an orchestrator of web services. In a choreography (right side of Figure 15), each web service involved in the process describes the part they play in the interaction which is performed in a collaborative manner.

The choreography can be exemplified by the interaction between the Geocoding and Directions web services. The Directions web service is able to extract the geographic coordinates inserted into the ontology by the Geocoding web service. This is performed in a i.e. choreographed manner, since the direct manipulation of the meeting alert ontology by

CCMF, Computational Context Modeling Framework –

It must support the

It must categorize the information into context

It has to identify the context

It must support the transfer of context information and artifacts between development

It must promote the diversity of context information in a domain-independent manner

It has to support the reuse of distributed computing systems

choreographed (O-CCMF) web services.

**5. Conclusion and future work** 

functionalities sources.

adaptation

dimensions

phases

focus of a task

such as services

development of context-aware software as a two-phase task: specification of context information (structure) and

An Ontological Approach to Develop Context-Aware Web Applications 81

CCMD is replaced by combined ontologies to represent context information. WSBPEL is maintained to

No specific data structures represent the focus and dimensions. It could be, though, added to ontologies (e.g. MAO) during the execution of the customization activity (activity 2 of the framework)

The usage of ontology is twofold: it represents the context and it enables the interoperability of web services by

Although ontologies are known for representing specific knowledge domains, they can be combined to support broader

Achieved by the usage of choreographed

create the adaptation

instantiated individuals

concepts

web services

Requirement CCMF O-CCMF

CCMD is coupled to the framework to represent context structures. WSBPEL diagram models

stereotypes that represent the context focus and

CCMD does not constrain

Achieved by the usage of orchestrated web services

CCMF and O-CCMF are capable of assisting developers in the creation of context-aware web applications. Adopting one or the other as development framework is a matter of deciding what it is the intended context information source: diagrams such as CCMD or ontologies. Another criterion would be the intended method of creating composite web services to serve context adaptation to client software: orchestrated (CCMF) or

Related works, as those described in Section 2, are likely to subject the modeling and development of context-aware applications to the utilization of ontologies or diagramsoriented solutions (e.g. stereotyped UML class diagram). By introducing CCMF, a modeldriven framework, and its ontology-oriented variation, O-CCMF, this work points to a heterogeneous scenario where context information is collected from different sources and adaptations are served by varied web services. Such diversification demands adaptive approaches from development solutions, as, for instance, the ability to deal with different modeling technologies (e.g. ontologies, diagrams) in a decoupled manner. Instead of favoring one specific form of development of context-aware web applications, CCMF and O-CCMF promote the reuse of a mixed set of artifacts, tools, API's, information and

In regard to CCMF, its coupling with CCMD intends to grant to developers an immersive environment in which concepts of computational context (context focus and dimensions)

the adaptation

CCMD contains

The transference is performed by XML documents and XML serialization API's

dimensions

the domain

Table 5. Evaluating the frameworks against the requirements.

Fig. 15. Choreography and orchestration of web services (Peltz, 2003).

the web services promotes an internalized sharing of context information, without the intervention of controlling agents, such as an orchestration mechanism. As an opposite example, the WSBPEL diagram (e.g. the one illustrated in Figure 9, Section 4.1) is responsible for transferring the context information from one service to other, i.e. using WSBPEL, software designers must model an appropriate sequence of actions so that the context information is interchanged among services. In this case, the WSBPEL workflow acts like a controlling agent.

#### **4.3 Comparing CCMF and O-CCMF**

Provided the two scenarios of development of context-aware web applications, the ontological approach, O-CCMF, leads on a reduced framework in comparison with the former version, CCMF, because ontologies have the characteristic of being artifacts suited for both the storage and sharing of information among computational agents. The meeting alert ontology, for instance, represents structures of context information and, too, it encloses the instances of its own ontological classes (individuals). On the contrary, considering that CCMF utilizes CCMD to only model the context information with no regard as to afford the mechanisms to allow the direct storage and instantiation of its classes as concrete objects, it is required from CCMF the transformation of CCMD into XSD/XML documents to support the sharing of context information. Consequently, as computational agents are able to add and select individuals directly from ontologies, the context adaptation can be automated by a workflow that groups and interoperates web services in a collaborative manner, i.e. the adaptation is served to client software by choreographed web services as a replacement to orchestrated ones, in a way that the orchestration eases the modeling of the adaptation by not requiring the provision of serializations.

Table 5 relates the requirements identified in Section 2 (Table 1) to CCMF and O-CCMF, indicating how they were fulfilled by each framework.

Web service

Web Web

the web services promotes an internalized sharing of context information, without the intervention of controlling agents, such as an orchestration mechanism. As an opposite example, the WSBPEL diagram (e.g. the one illustrated in Figure 9, Section 4.1) is responsible for transferring the context information from one service to other, i.e. using WSBPEL, software designers must model an appropriate sequence of actions so that the context information is interchanged among services. In this case, the WSBPEL workflow acts

Web services orchestration Web services choreography

Provided the two scenarios of development of context-aware web applications, the ontological approach, O-CCMF, leads on a reduced framework in comparison with the former version, CCMF, because ontologies have the characteristic of being artifacts suited for both the storage and sharing of information among computational agents. The meeting alert ontology, for instance, represents structures of context information and, too, it encloses the instances of its own ontological classes (individuals). On the contrary, considering that CCMF utilizes CCMD to only model the context information with no regard as to afford the mechanisms to allow the direct storage and instantiation of its classes as concrete objects, it is required from CCMF the transformation of CCMD into XSD/XML documents to support the sharing of context information. Consequently, as computational agents are able to add and select individuals directly from ontologies, the context adaptation can be automated by a workflow that groups and interoperates web services in a collaborative manner, i.e. the adaptation is served to client software by choreographed web services as a replacement to orchestrated ones, in a way that the orchestration eases the modeling of the adaptation by

Table 5 relates the requirements identified in Section 2 (Table 1) to CCMF and O-CCMF,

Fig. 15. Choreography and orchestration of web services (Peltz, 2003).

like a controlling agent.

**4.3 Comparing CCMF and O-CCMF** 

Web service

Web Web

not requiring the provision of serializations.

indicating how they were fulfilled by each framework.


Table 5. Evaluating the frameworks against the requirements.

CCMF and O-CCMF are capable of assisting developers in the creation of context-aware web applications. Adopting one or the other as development framework is a matter of deciding what it is the intended context information source: diagrams such as CCMD or ontologies. Another criterion would be the intended method of creating composite web services to serve context adaptation to client software: orchestrated (CCMF) or choreographed (O-CCMF) web services.

## **5. Conclusion and future work**

Related works, as those described in Section 2, are likely to subject the modeling and development of context-aware applications to the utilization of ontologies or diagramsoriented solutions (e.g. stereotyped UML class diagram). By introducing CCMF, a modeldriven framework, and its ontology-oriented variation, O-CCMF, this work points to a heterogeneous scenario where context information is collected from different sources and adaptations are served by varied web services. Such diversification demands adaptive approaches from development solutions, as, for instance, the ability to deal with different modeling technologies (e.g. ontologies, diagrams) in a decoupled manner. Instead of favoring one specific form of development of context-aware web applications, CCMF and O-CCMF promote the reuse of a mixed set of artifacts, tools, API's, information and functionalities sources.

In regard to CCMF, its coupling with CCMD intends to grant to developers an immersive environment in which concepts of computational context (context focus and dimensions)

CCMF, Computational Context Modeling Framework –

Document, RDFWeb, xmlns.com.

Janeiro, Brasil, November

*Networking and Services*, August

London, England, pp. 304-307

91-7636-431-3. Växjö, Sweden

gajderowicz-msc-thesis.pdf

BR/apis/maps/documentation/directions/

BR/apis/maps/documentation/geocoding/

Available from http://www.eclipse.org/modeling/gmp

EMF (2009). Available from http://www.eclipse.org/modeling/emf

*interaction*, vol. 16

BR/apis/calendar/

ICMC-USP, São Carlos, São Paulo, Brasil

France, pp. 29-40

An Ontological Approach to Develop Context-Aware Web Applications 83

Bontas, E. P.; Mochol, M. & Tolksdorf, R. (2005). Cases on Ontology reuse. *Proceedings of the* 

Brickley, D. & Miller, L (2003). FOAF vocabulary specification. RDFWeb Namespace

Bulcão, R. (2006). Um processo de software e um modelo ontológico para apoio ao

Carvalho, L. P. & Silva, P. C. (2011). CCMD – Computational Context Modeling Diagram –

Chen, H.; Perich, F.; Finin, T. & Joshi, A. (2004). SOUPA: Standard Ontology for Ubiquitous

Dey, A. K. (2001). Understanding and Using Context. *Personal and Ubiquitous Computing, Special issue on Situated Interaction and Ubiquitous Computing* 5, 1, pp. 4-7 Dey, A. K. & Abowd, G. D. (1999). Towards a better understanding of context and context-

Dey, A. K.; Abowd, G. D. & Salber, D. (2001). A conceptual framework and a toolkit for

EuGENia (2011). Available from http://www.eclipse.org/gmt/epsilon/doc/eugenia/ Gábor, V.; Andersson, B. & Wohed, P. (2004). An Ontology based Analysis of BPEL4WS and

GAgenda (2011). Google Agenda API. Available from http://code.google.com/intl/pt-

Gajderowicz, B. (2011). Using decision trees for inductively driven semantic integration and

GDirections (2011). Google Directions API. Available from http://code.google.com/intl/pt-

GGCoding (2011). Google GeoCoding API. Available from http://code.google.com/intl/pt-

GMF (2010). Graphical Modeling Framework (Graphical Modeling Project - GMP).

*and Interdisciplinary Conference, CONTEXT-05*, v. LNAI 3554, Springer Verlag, Paris,

desenvolvimento de aplicações sensíveis a contexto. Serviço de pós-graduação do

And WSBPEL Integration. *IADIS Applied Computing International Conference*. Rio de

and Pervasive Applications. *1st Annual Int'l Conf. on Mobile and Ubiquitous Systems:* 

awareness, *Proceedings of the first international symposium on handheld and ubiquitous computing*, H. Gellersen, Ed. Lecture Notes in computer science. Springer-Verlag,

supporting the rapid prototyping of context-aware applications, *Human-computer* 

WSCI. *Proceedings of the 3rd Nordic Conference on Web Services (NCWS 2004)*, ISBN

ontology matching. Thesis, Ryerson University, Program of Computer Science. Available from http://www.scs.ryerson.ca/~bgajdero/msc\_thesis/document/b-

BPELEclipse (2010). BPEL Eclipse Designer. Available from http://www.eclipse.org/bpel Brézillon, P. (1999). Context in problem solving: a survey. *Knowl. Eng. Rev. 14*, 1, pp. 47-80. Brézillon, P. & Bazire, M. (2005). Understanding Context Before Using It. *5th International* 

*5th International Conference on Knowledge Management*

orients the design of context information and adaptation. An advantage that is not provided by O-CCMF. However, contrary to ontologies, CCMD (and CCMF, consequently) does not rely on a single artifact to represent classes and instances of context information. Thus, CCMF has to be coupled with extra mechanisms (e.g serializable JAVA classes) to workaround such limitation which is naturally overcame by O-CCMF.

Both frameworks rely on web services to automate context adaptation and to serve it to remote clients across the internet. One advantage that arises from this approach is that of making possible the addition of further context information and adaptation steps to enhance context-awareness mechanisms. For instance, improvements in the Google Agenda API can be automatically propagated to client software without needing to distribute modification patches. Conversely, faulty web services might lessen the quality of served adaptations. For example, in case a specific functionality of the Google Agenda API either becomes deprecated or fails to retrieve some important context information, client software may not succeed in supplying context-awareness to end users.

Currently, the following tasks must be carried out in order to supplement this work:


## **6. References**


orients the design of context information and adaptation. An advantage that is not provided by O-CCMF. However, contrary to ontologies, CCMD (and CCMF, consequently) does not rely on a single artifact to represent classes and instances of context information. Thus, CCMF has to be coupled with extra mechanisms (e.g serializable JAVA classes) to

Both frameworks rely on web services to automate context adaptation and to serve it to remote clients across the internet. One advantage that arises from this approach is that of making possible the addition of further context information and adaptation steps to enhance context-awareness mechanisms. For instance, improvements in the Google Agenda API can be automatically propagated to client software without needing to distribute modification patches. Conversely, faulty web services might lessen the quality of served adaptations. For example, in case a specific functionality of the Google Agenda API either becomes deprecated or fails to retrieve some important context information, client software may not

Currently, the following tasks must be carried out in order to supplement this work:

1. The ontology-driven framework must be evaluated against further study cases to analyze weather the WSBPEL diagram can be decoupled from the framework in order to favor other composition mechanisms, e.g. the WSCI, Web Service Composition Interface (Gábor et al. 2004), which is representative of the choreography approach; 2. Reasoning mechanisms based on the semantics supplied by ontologies must be surveyed in order to enhance the adaptation. For instance, the conditional test that evaluate if the participant is the same location of the event could be inferred from the

3. The ontological approach promotes the binding of existent ontologies in the definition of domain-specific context information. Therefore, the searching for fitting available ontologies must be adequately supported by the framework. It must be surveyed how this activity can be better assisted by the coupling O-CCMF with available third-part

Abowd, G. D. & Mynatt. E. D. (2000). Charting past, present, and future research in ubiquitous computing. *ACM Trans. Comput.-Hum. Interact. 7*, pp. 29-58 Alboaie, L.; Buraga, S. C. & Alboaie, S. (2003). tuBiG: a layered infrastructure to provide

Antoniou, G.; Bikakis, A.; Karamolegou, A. & Papachristodoulou, N. (2007). A context-

Bastida, L.; Nieto, F. J. & Tola, R. (2008). Context-aware service composition: a methodology

Bonino, L. O.; Wijnen, R. P. & Vink, P. (2007). A service-oriented middleware for context-

*Parallel and distributed computing*, Washington, DC, USA, pp. 9-14

*Metadata, Semantics and Ontologies 2007*, Vol. 2, No.3, pp. 147 – 156

*in SOA environments*, New York, USA, pp. 19-24

*pervasive and ad-hoc computing*. New York, NY, USA

support for Grid functionalities, *Proceedings of the second international conference on* 

aware meeting alert using semantic web and rule technology. *International Journal of* 

and a case study, *Proceedings of the 2nd international workshop on Systems development* 

aware applications. In *Proceedings of the fifth international workshop on middleware for* 

workaround such limitation which is naturally overcame by O-CCMF.

succeed in supplying context-awareness to end users.

ontology by the web service that schedules the alerts;

tools and processes.

**6. References** 


Brézillon, P. (1999). Context in problem solving: a survey. *Knowl. Eng. Rev. 14*, 1, pp. 47-80.


**Section 2** 

**Applications: Semantic Cache, E-Health,** 

**Sport Video Browsing, and Power Grids** 


## **Section 2**

## **Applications: Semantic Cache, E-Health, Sport Video Browsing, and Power Grids**

84 Semantics in Action – Applications and Scenarios

George, A. A. & Ward, P. A. (2008). An architecture for providing context in WS-BPEL

Hoyos, J. R.; Molina, J. G. & Botia, J. A. (2010). MLContext: A Context-Modeling Language

JBDrools (2009). JBoss Drools. Available from http://www.jboss.org/drools/drools-flow Noy, N. F. (2004). Semantic integration: A survey of ontology-based approaches. *SIGMOD* 

OWLAPI (2011). The OWL API. Available from http://owlapi.sourceforge.net/index.html Patrício, R. F. (2010). CEManTIKA CASE: uma ferramenta de apoio ao desenvolvimento de

Peltz, C. (2003). Web Service Orchestration and Choreography. Available from http://wpage.unina.it/rcanonic/didattica/at/documenti/wsOrchestration.pdf

Sheng, Q. Z. & Benatallah, B. (2005). ContextUML: A UML-based modeling language for

Topcu, F. (2011). Context modeling and reasoning. Available from http://www.snet.tuberlin.de/fileadmin/fg220/courses/SS11/snet-project/context-modeling-and-

Vieira, V. 2008. CEManTIKA: A domain-independent framework for designing contextsensitive systems, Centro de Informática - UFPE, Recife, Pernambuco, Brasil XMLBeans (2009). Apache XML Beans 2.5. Available from http://xmlbeans.apache.org/ XSD (2001). W3C XML Schema 1.1. Available from http://www.w3.org/XML/Schema Yamato, Y.; Nakano, Y. & Sunaga, H. (2010). *Context-aware service composition and change-over using BPEL engine and semantic web*. Intech Open Access Publisher. Rijeka, Croatia YWForecast (2011). Yahoo Weather. Available from http://developer.yahoo.com/weather/ Wang, X. H.; Gu, T.; Zhang, D. Q. & Pung, H. K. (2004). Ontology based context modeling

*distributed systems modeling and simulation conference*, San Diego, CA, USA WSBPEL (2007). OASIS Web Service Business Process Execution Language (WSBPEL) 2.0.

Available from http://www.oasis-open.org/committees/wsbpel/

Building Automation Systems using Ontologies. *Proc. of the 34th Annual Conference of the IEEE Industrial Electronics Society (IECON '08)*, Nov. 2008, pp. 2736–2741 Schmidt, D. C. (2006). Guest Editor's Introduction: Model-Driven Engineering. IEE

model-driven development of context-aware web services, *Proceedings of the International Conference on Mobile Business*, Washington, DC, USA, pp. 206-212 Simons, C. 2007. CMP: A UML context modeling profile for mobile distributed systems,

*Proceedings of the 40th Annual Hawaii International Conference on System Sciences*,

and reasoning using OWL, *Proceedings of the 2004 communication networks and* 

Sistemas Sensíveis ao Contexto, UFPE, Recife, Pernambuco, Brasil

Protégé (2011). Protégé Ontology Editor. Available from http://protege.stanford.edu/ Reinisch, C.; Granzer, W.; Fraus, F. & Kastner, W. (2008). Integration of Heterogeneous

*collaborative research: meeting of minds*, New York, NY, USA, Article 22 Grassi, V. & Sindico, A. (2009). Model driven development of context aware software

York, USA, pp. 1-5

*Record*, 33(4), Dec. 2004

Computer 39(2), pp. 25-31

Hawaii, USA

reasoning\_topcu.pdf

*Association of Software Science and Technology*

processes, *Proceedings of the 2008 conference of the center for advanced studies on* 

systems, *Proceedings of International Workshop on Context-Oriented Programming*, New

for Context-Aware Systems, *Proceedings of Electronic Communications of the European* 

**4** 

¶(12pt)

*Pakistan* 

*Faculty of Computing,* 

**Semantic Cache System** 

*Centre for Distributed and Semantic Computing,* 

*Mohammad Ali Jinnah University Islamabad,* 

Munir Ahmad, Muhammad Abdul Qadir, Tariq Ali,

Muhammad Azeem Abbas and Muhammad Tanvir Afzal

One of the economical ways to develop a very large scale database is to distribute it among multiple server nodes. Main problem in these types of systems is retrieval of data within significant time; especially when network or server load is high. This task becomes more critical when data is to be retrieved from the database against frequent queries. Cache is used to increase the retrieval performance of mobile computing and distributed database systems. Whenever data is found locally from the cache it is termed as cache hit. Percentage of user posed queries that can be processed (partially or fully) locally from cache is called hit ratio. So, the cache system should be designed in a way that will increase the hit ratio. Improvement in hit ratio ensures efficient reuse of stored data. Due to efficient reuse of

stored data, lesser amount of data is required to be retrieved from remote location.

Typically, cache is organized in three ways, as page, tuple, and semantic. Unit of transfer in page cache is *page* (multiple tuples) and in tuple cache is a *tuple*. In page cache irrelevant tuples may be retrieved for a user query. Retrieval of irrelevant data causes to wastage of valuable resources. Tuple cache overcomes this problem by stopping the retrieval of irrelevant tuples. Major problem (retrieving portion of tuple instead of complete tuple) still exists which cannot be handled using both (page & tuple) of these caching models. These caching schemes (page & tuple) are not able to identify whether the answer is contained in the cache in case of query not fully matched (partial matched). In page and tuple cache schemes all of the data is retrieved from remote site even in the presence of partial data on cache. In simple words portion of page or portion of tuple cannot be reused in the presences of page or tuple caching. As a result hit ratio is not up to that extent to which it should be. To answer the queries partially from local site concept of semantic cache is introduced. Semantic cache has an ability to increase the hit ratio up to possible extent. Semantic cache provides better performance than page and tuple cache and this system is referred as semantic caching. Semantic caching provides the significance workload reduction in distributed systems, especially in mobile computing as well as improves the performance. In semantic caching the semantic descriptions of processed query with actual contents are stored. Next posed query is processed for stored semantic descriptions of data in the cache

**1. Introduction** 

## **Semantic Cache System**

Munir Ahmad, Muhammad Abdul Qadir, Tariq Ali, Muhammad Azeem Abbas and Muhammad Tanvir Afzal *Centre for Distributed and Semantic Computing,* 

> *Faculty of Computing, Mohammad Ali Jinnah University Islamabad, Pakistan*

## **1. Introduction**

One of the economical ways to develop a very large scale database is to distribute it among multiple server nodes. Main problem in these types of systems is retrieval of data within significant time; especially when network or server load is high. This task becomes more critical when data is to be retrieved from the database against frequent queries. Cache is used to increase the retrieval performance of mobile computing and distributed database systems. Whenever data is found locally from the cache it is termed as cache hit. Percentage of user posed queries that can be processed (partially or fully) locally from cache is called hit ratio. So, the cache system should be designed in a way that will increase the hit ratio. Improvement in hit ratio ensures efficient reuse of stored data. Due to efficient reuse of stored data, lesser amount of data is required to be retrieved from remote location.

Typically, cache is organized in three ways, as page, tuple, and semantic. Unit of transfer in page cache is *page* (multiple tuples) and in tuple cache is a *tuple*. In page cache irrelevant tuples may be retrieved for a user query. Retrieval of irrelevant data causes to wastage of valuable resources. Tuple cache overcomes this problem by stopping the retrieval of irrelevant tuples. Major problem (retrieving portion of tuple instead of complete tuple) still exists which cannot be handled using both (page & tuple) of these caching models. These caching schemes (page & tuple) are not able to identify whether the answer is contained in the cache in case of query not fully matched (partial matched). In page and tuple cache schemes all of the data is retrieved from remote site even in the presence of partial data on cache. In simple words portion of page or portion of tuple cannot be reused in the presences of page or tuple caching. As a result hit ratio is not up to that extent to which it should be.

To answer the queries partially from local site concept of semantic cache is introduced. Semantic cache has an ability to increase the hit ratio up to possible extent. Semantic cache provides better performance than page and tuple cache and this system is referred as semantic caching. Semantic caching provides the significance workload reduction in distributed systems, especially in mobile computing as well as improves the performance.

In semantic caching the semantic descriptions of processed query with actual contents are stored. Next posed query is processed for stored semantic descriptions of data in the cache

¶(12pt)

Semantic Cache System 89

**Definition 8**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW*> and <*D, R, A, SA, P, C>* respectively*;* **Predicate Matching** is a process in which user query's condition <*QW*> is matched with condition indexed by semantic enabled schema <*P*>.

**Definition 9**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW, PA*> and <*D, R, A, SA, P, C*>; **Query Trimming** is a process in which user query (*QU*) is divided

**Definition 10**: Given a user predicate *QW* and cached predicate *P*; **Predicate Implication** 

**Definition 11**: Given a user predicate *QW* and cached predicate *P*; **Predicate Satisfiablity**

**Definition 12**: Given a user predicate *QW* and cached predicate *P*; **Predicate Unsatisfiablity**

**Definition 13**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW, PA*> and <*D, R, A, SA, P, C*>; **Query Implications** holds if and only if *QS* A as well as

**Definition 14**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW, PA*> and <*D, R, A, SA, P, C*>; **Query Satisfiability** holds if and only if *QS ∩ A != Φ* as well as

**Definition 15**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW, PA*> and <*D, R, A, SA, P, C*>; **Query Unsatisfiaility** holds either *QS ∩ A = Φ* or predicate

This section presents the brief related work to semantic caching in the context of semantic indexing and query (*SELECT and PROJECT*) processing for relational databases. For detail work survey done by Ahmad et al. (Ahmad et al, 2008) can be considered and for aggregate queries is discussed by Cai et al (Cai et al, 2005). Semantic caching is extensively studied by researchers in both relational and XML databases. In fact, query processing and cache management are two main areas of semantic cache system. In this section we have described

Results of already executed queries are cached to generate more efficient query plans for centralized systems (Roussopoulosl, 1991). Some strategies are defined for cache to prefecth the data by using semantics (Kang et al, 2006). Query refinement technique is introduced to enhance the response time in multimedia databases (Chakrabarti et al, 2000). A predicate based scheme for cache is presented by Keller et al. for client server applications (Keller and Basu, 1996). A scheme with the name of Intelligent Cache Management (Chen et al, 1994) and its extensions are introduced (Ahmed et al, 2005, Altinel et al, 2003, Bashir and Qadir, 2007) to reduce the overhead of page and tuple cache. To answer the queries partially from local site; concept of semantic cache on the base of implication (Sun et al, 1989) and the base of description logic (Ali et al, 2010, Ali and Qadir, 2010) is introduced to increase the hit ratio up to possible extent (Bashir and Qadir 2007, Ahmad et al, 2008a). Idea of amending query is introduced to increase the hit

the state of the art query processing as well as semantic indexing schemes.

(*QW→P*) holds if and only if *QW* completely overlapped with *P*.

holds if and only if *QW* partially overlapped with *P*.

holds if and only if *QW* is not overlapped with *P*.

predicate implication/satisfiability holds.

implication/satisfiability does not holds.

into probe and remainder query.

predicate implication holds.

**3. State of the art** 

and posed query is divided into probe (portion available at cache) and remainder (portion that is not available at cache and have to retrieved from cache) queries. In this context we can say that there are two major activities query processing and cache management are involved in semantic caching. So, efficiency of the semantic caching will depends on these two major activities (query processing and cache management). Query processing is the process which returns the result against user posed query. In semantic cache query processing is done by dividing the user query into probe and remainder queries on the base of query matching. In fact, efficiency of query processing will depend on the efficiency of division process (query trimming) of user query into sub queries (probe and remainder) as well as on retrieval time against both probe and remainder queries. However, efficiency of query trimming will depends on the semantic indexing. In fact, semantic indexing at cache is a major activity of cache management. In this context we can say that efficient semantic caching system demands efficient query processing and indexing scheme. In this chapter we have discussed the state of art query processing techniques and semantic indexing scheme. We also have presented a query processing scheme sCacheQP and its complete working. Working of sCacheQP is explained with the help of case study.

## **2. Definitions**

This section presents some definitions that are used in rest of the chapter.

**Definition 1**: Given a user query *Qu* = *πAσP(R)*; where '*A*' is set of attributes required by user, '*P*' is a condition (WHERE clause) of the user query, and '*R*' is a relation. **User Query's Semantics** will be 3-tupple <*QS, QF, QW*> where *QS* is a set of required attributes, *QF* is a relation, and *QW* is a condition.

**Definition 2**: Given a database *D = {Ri}* and its attributes set *A = UARi, 1 ≤ i ≤n,* **Semantic Enabled Schema** will be 6-tupple <*D, R, A, SA, P, C*> where '*D*' is the name of database, '*R*' is name of relation, '*A*' is a set of attributes, is a status of attributes, '*P*' is predicate (condition) on which data has been retrieved and cached, and '*C*' is the refrence of contents.

**Definition 3**: Given a user query *Qu* = *πAσP(R)* and *QC* having semantics <*D, R, A, SA, P, C*>; **Data Set Du** and **Dc** will be retrieved rows in the result of execution of *Qu and QC* respectively*.*

**Definition 4**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW*> and <*D, R, A, SA, P, C>* respectively*;* **Probe Query (pq)** will be **Du** *∩* **Dc***.*

**Definition 5**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW*> and <*D, R, A, SA, P, C>* respectively*;* **Remainder Query (rq)** will be (**Du** *-* **Dc***).*

**Definition 6**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW*> and <*D, R, A, SA, P, C>* respectively*;* **Query Matching** is a process in which user query's semantics <*QS, QF, QW*> are matched with semantic enabled schema <*D, R, A, SA, P, C*>. It is further divided into two processes; attribute and predicate matching.

**Definition 7**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW*> and <*D, R, A, SA, P, C>* respectively*;* **Attribute Matching** is a process in which user query's attributes <*QS*> are matched with attributes indexed by semantic enabled schema <*A*>. Common attributes (*CA*) *QS ∩ A* and difference attributes (*DA*) *QS - A* are calculated for probe and remainder queries respectively.

and posed query is divided into probe (portion available at cache) and remainder (portion that is not available at cache and have to retrieved from cache) queries. In this context we can say that there are two major activities query processing and cache management are involved in semantic caching. So, efficiency of the semantic caching will depends on these two major activities (query processing and cache management). Query processing is the process which returns the result against user posed query. In semantic cache query processing is done by dividing the user query into probe and remainder queries on the base of query matching. In fact, efficiency of query processing will depend on the efficiency of division process (query trimming) of user query into sub queries (probe and remainder) as well as on retrieval time against both probe and remainder queries. However, efficiency of query trimming will depends on the semantic indexing. In fact, semantic indexing at cache is a major activity of cache management. In this context we can say that efficient semantic caching system demands efficient query processing and indexing scheme. In this chapter we have discussed the state of art query processing techniques and semantic indexing scheme. We also have presented a query processing scheme sCacheQP and its complete working.

Working of sCacheQP is explained with the help of case study.

This section presents some definitions that are used in rest of the chapter.

<*D, R, A, SA, P, C>* respectively*;* **Probe Query (pq)** will be **Du** *∩* **Dc***.*

<*D, R, A, SA, P, C>* respectively*;* **Remainder Query (rq)** will be (**Du** *-* **Dc***).*

further divided into two processes; attribute and predicate matching.

**Definition 1**: Given a user query *Qu* = *πAσP(R)*; where '*A*' is set of attributes required by user, '*P*' is a condition (WHERE clause) of the user query, and '*R*' is a relation. **User Query's Semantics** will be 3-tupple <*QS, QF, QW*> where *QS* is a set of required attributes, *QF* is a

**Definition 2**: Given a database *D = {Ri}* and its attributes set *A = UARi, 1 ≤ i ≤n,* **Semantic Enabled Schema** will be 6-tupple <*D, R, A, SA, P, C*> where '*D*' is the name of database, '*R*' is name of relation, '*A*' is a set of attributes, is a status of attributes, '*P*' is predicate (condition) on which data has been retrieved and cached, and '*C*' is the refrence of contents. **Definition 3**: Given a user query *Qu* = *πAσP(R)* and *QC* having semantics <*D, R, A, SA, P, C*>; **Data Set Du** and **Dc** will be retrieved rows in the result of execution of *Qu and QC*

**Definition 4**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW*> and

**Definition 5**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW*> and

**Definition 6**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW*> and <*D, R, A, SA, P, C>* respectively*;* **Query Matching** is a process in which user query's semantics <*QS, QF, QW*> are matched with semantic enabled schema <*D, R, A, SA, P, C*>. It is

**Definition 7**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW*> and <*D, R, A, SA, P, C>* respectively*;* **Attribute Matching** is a process in which user query's attributes <*QS*> are matched with attributes indexed by semantic enabled schema <*A*>. Common attributes (*CA*) *QS ∩ A* and difference attributes (*DA*) *QS - A* are calculated for probe

**2. Definitions** 

respectively*.*

relation, and *QW* is a condition.

and remainder queries respectively.

**Definition 8**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW*> and <*D, R, A, SA, P, C>* respectively*;* **Predicate Matching** is a process in which user query's condition <*QW*> is matched with condition indexed by semantic enabled schema <*P*>.

**Definition 9**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW, PA*> and <*D, R, A, SA, P, C*>; **Query Trimming** is a process in which user query (*QU*) is divided into probe and remainder query.

**Definition 10**: Given a user predicate *QW* and cached predicate *P*; **Predicate Implication**  (*QW→P*) holds if and only if *QW* completely overlapped with *P*.

**Definition 11**: Given a user predicate *QW* and cached predicate *P*; **Predicate Satisfiablity** holds if and only if *QW* partially overlapped with *P*.

**Definition 12**: Given a user predicate *QW* and cached predicate *P*; **Predicate Unsatisfiablity** holds if and only if *QW* is not overlapped with *P*.

**Definition 13**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW, PA*> and <*D, R, A, SA, P, C*>; **Query Implications** holds if and only if *QS* A as well as predicate implication holds.

**Definition 14**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW, PA*> and <*D, R, A, SA, P, C*>; **Query Satisfiability** holds if and only if *QS ∩ A != Φ* as well as predicate implication/satisfiability holds.

**Definition 15**: Given a user query *QU* and cached query *QC* with semantics < *QS, QF, QW, PA*> and <*D, R, A, SA, P, C*>; **Query Unsatisfiaility** holds either *QS ∩ A = Φ* or predicate implication/satisfiability does not holds.

## **3. State of the art**

This section presents the brief related work to semantic caching in the context of semantic indexing and query (*SELECT and PROJECT*) processing for relational databases. For detail work survey done by Ahmad et al. (Ahmad et al, 2008) can be considered and for aggregate queries is discussed by Cai et al (Cai et al, 2005). Semantic caching is extensively studied by researchers in both relational and XML databases. In fact, query processing and cache management are two main areas of semantic cache system. In this section we have described the state of the art query processing as well as semantic indexing schemes.

Results of already executed queries are cached to generate more efficient query plans for centralized systems (Roussopoulosl, 1991). Some strategies are defined for cache to prefecth the data by using semantics (Kang et al, 2006). Query refinement technique is introduced to enhance the response time in multimedia databases (Chakrabarti et al, 2000). A predicate based scheme for cache is presented by Keller et al. for client server applications (Keller and Basu, 1996). A scheme with the name of Intelligent Cache Management (Chen et al, 1994) and its extensions are introduced (Ahmed et al, 2005, Altinel et al, 2003, Bashir and Qadir, 2007) to reduce the overhead of page and tuple cache. To answer the queries partially from local site; concept of semantic cache on the base of implication (Sun et al, 1989) and the base of description logic (Ali et al, 2010, Ali and Qadir, 2010) is introduced to increase the hit ratio up to possible extent (Bashir and Qadir 2007, Ahmad et al, 2008a). Idea of amending query is introduced to increase the hit

Semantic Cache System 91

Fig. 1. Working of sCacheQP.

OUTPUT: FR (Result against QU)

2. *PORTIONS* := *SPLIT*\_*QUERY* (QU) 3. *Reject := CHECK*\_*REJECTION* (*PORTIONS)*

5. *CA, DA :=*1st\_Level\_Query\_Rewriter (*QS)*

6. If(DA != empty) goto step 7 otherwise goto step 8

8. If (CA !=empty) goto step 9 otherwise goto step 14.

12. pq, rq2:=PredicateMerging *(C2, NC2, Coc, Cou)*

10. *C1, NC1:=* ExplicitySemanticMatching(*MC, Dvc, Dvu ,Opc, Opu*) 11. *C2, NC2:=* ImplicitSemanticMatching(*Mc*, *NMCC, NMCU, C1, NC1*)

4. if (Reject =false) goto step 5 otherwise Reject query and goto step 15

9. *MC, NMCC, NMCU, Dvc, Dvu ,Opc, Opu, Coc, Cou*:=Semantic\_Extraction(QW, SW*)*

**Algrithm1**: sCacheQP INPUT: QU (User query)

PROCEDURE:

1. Initialization: pq:=NULL rq1:=NULL rq2:=NULL

7. rq1 := πDAσQP(QR)

Fig. 2. Main Algorithm of sCacheQP.

13. aq :=*GEN\_AMEND\_QUERY*() 14. FR:= Rebuilder(pq,rq1,rq2)

15. Exit.

ratio (Ren et al, 2003), graph based query trimming to enhance efficiency (Abbas et al, 2010) and 112 rules (Bashir and Qadir, 2007b) are defined to reduce query processing time by efficient query matching. Rules (112) are only applicable for simple queries (excluding the disjunct or conjunct operator). Jonsson and colleagues presents query matching scheme by using predicate of query (Jonsson et al, 2006). This scheme is not able to handle the SELECT CLAUSE of SQL queries. Query matching algorithm that reduces query processing time in the domain of relational database is also studied in our previous work (Ahmad et al, 2008a, Ahmad et al, 2008, Ahmad et al, 2009]. Work in semantic cache in other domains like web [Lee et al, 1995, Luo et al, 2003] and XML (Chen et al, 2002, Sanaullah et al, 2008) also studied in literature. Importance of semantic cache and disadvantages of page and tuple cache is presented [Ren et al, 2003, Dar et al, 1996] by providing comparisons of semantic cache with page and tuple cache.

There are different structures used to index the semantic description like; flat structure (Dar et al, 1996), 3-level hierarchal (Sumalatha et al, 2007a, 2007b, 2007c) segments (Ren et al, 2003), and 4-HiSIS (Bashir and Qadir, 2007). When semantics of queries are store in a flat structure (Dar et al, 1996) the query matching process is very expensive (time consuming) (Godfrey et al, 1997, Ahmad et al, 2008, Ahmad et al, 2009). Cache is divided into segments (Ren et al, 2003) and chunks (Deshpande et al, 1998) to reduce the cost. Runtime complexity and caching efficiency is improved by division of cache into segments and chunks. List of chunks is build on the base of previous queries and then this list of chunks is used to split the user posed (new) queries into two portions; one answered locally from cache and the second computed remotely (Deshpande et al, 1998). 4-level hierarchal semantic indexing scheme (*4-HiSIS*) is introduced to accelerate the semantic matching (Bashir and Qadir, 2006). In 4-HiSIS; semantic matching accomplished in four steps. At first; database name is matched. After successful matching of the database name; the relation name is matched in the second step. At third, attributes are matched at successful matching of relation match. In the final step predicate matching is performed on the based of successful matching of first three steps. There is a limitation of 4-HiSIS in the context of incompleteness; because there is no refrence of actual contents of cache is stored in 4-HiSIS. This limitation is overcomed by the graph based semantic indexing scheme (Ahmad et al, 2010) by storing the refrence of actual contents. In graph based semantic indexing scheme the matching procedure is performed in five steps. At the state of art graph based indexing is most efficient semantic indexing scheme. It also have a limitation; it has no ability to process the " *Select \*"* type and incorrect queries in cache system.

State of the art semantic cache system has limitation in both areas (query processing and cache management i.e semantic indexing schemes). In this chapter we have presented the new scheme for semantic cache query processing. We named this system as sCacheQP. sCacheQP has an ability to overcome the limitaion in the context of query processing which is the main area of the semantic cahce system.

## **4. Semantic Cache Query Processing (sCacheQP)**

This section presents the sCacheQP which is a complete procedure of query processing that overcomes the limitaions of the previous systems. Working and main driver algorithm of the sCacheQP is given in Figure 1 and Figure 2 respectively.

ratio (Ren et al, 2003), graph based query trimming to enhance efficiency (Abbas et al, 2010) and 112 rules (Bashir and Qadir, 2007b) are defined to reduce query processing time by efficient query matching. Rules (112) are only applicable for simple queries (excluding the disjunct or conjunct operator). Jonsson and colleagues presents query matching scheme by using predicate of query (Jonsson et al, 2006). This scheme is not able to handle the SELECT CLAUSE of SQL queries. Query matching algorithm that reduces query processing time in the domain of relational database is also studied in our previous work (Ahmad et al, 2008a, Ahmad et al, 2008, Ahmad et al, 2009]. Work in semantic cache in other domains like web [Lee et al, 1995, Luo et al, 2003] and XML (Chen et al, 2002, Sanaullah et al, 2008) also studied in literature. Importance of semantic cache and disadvantages of page and tuple cache is presented [Ren et al, 2003, Dar et al, 1996] by

There are different structures used to index the semantic description like; flat structure (Dar et al, 1996), 3-level hierarchal (Sumalatha et al, 2007a, 2007b, 2007c) segments (Ren et al, 2003), and 4-HiSIS (Bashir and Qadir, 2007). When semantics of queries are store in a flat structure (Dar et al, 1996) the query matching process is very expensive (time consuming) (Godfrey et al, 1997, Ahmad et al, 2008, Ahmad et al, 2009). Cache is divided into segments (Ren et al, 2003) and chunks (Deshpande et al, 1998) to reduce the cost. Runtime complexity and caching efficiency is improved by division of cache into segments and chunks. List of chunks is build on the base of previous queries and then this list of chunks is used to split the user posed (new) queries into two portions; one answered locally from cache and the second computed remotely (Deshpande et al, 1998). 4-level hierarchal semantic indexing scheme (*4-HiSIS*) is introduced to accelerate the semantic matching (Bashir and Qadir, 2006). In 4-HiSIS; semantic matching accomplished in four steps. At first; database name is matched. After successful matching of the database name; the relation name is matched in the second step. At third, attributes are matched at successful matching of relation match. In the final step predicate matching is performed on the based of successful matching of first three steps. There is a limitation of 4-HiSIS in the context of incompleteness; because there is no refrence of actual contents of cache is stored in 4-HiSIS. This limitation is overcomed by the graph based semantic indexing scheme (Ahmad et al, 2010) by storing the refrence of actual contents. In graph based semantic indexing scheme the matching procedure is performed in five steps. At the state of art graph based indexing is most efficient semantic indexing scheme. It also have a limitation; it has no ability to process the " *Select \*"* type and

State of the art semantic cache system has limitation in both areas (query processing and cache management i.e semantic indexing schemes). In this chapter we have presented the new scheme for semantic cache query processing. We named this system as sCacheQP. sCacheQP has an ability to overcome the limitaion in the context of query processing which

This section presents the sCacheQP which is a complete procedure of query processing that overcomes the limitaions of the previous systems. Working and main driver algorithm of

providing comparisons of semantic cache with page and tuple cache.

incorrect queries in cache system.

is the main area of the semantic cahce system.

**4. Semantic Cache Query Processing (sCacheQP)** 

the sCacheQP is given in Figure 1 and Figure 2 respectively.

Fig. 1. Working of sCacheQP.

Fig. 2. Main Algorithm of sCacheQP.

Semantic Cache System 93

When it has been decided that data is available at cache then second step of sCacheQP is performed. In this step query is divided into two sub queries called probe and remainder queries called query trimming. This process accomplished in two stages. At first stage vertical partition takes place and the attributes that are not available (DA) at cache directly sent to the server as rq1 (remainder query) with original predicate. We called it 1st level query rewriter (Ahmad et al, 2009) and its algorithm is given in figure 5. The query rq1 will

Rest of attributes; that are common in both user and cached query forwarded to the predicate processor which worked at second stage. Predicate processor consists of four sub modules; semantic extractor, Explicit Semantic Matcher, Implicit Semantic Matcher, and Predicate Merger. At this stage predicate is simplified by just separating the portions of it on the base of conjunct and disjunct operators. Then semantics of user's query predicate with respect to the cached predicate is extracted in the form of matching columns (*Mc*- similar in both user query predicate and cached predicate), non-matching columns of cache (*NMc*columns in cached query that are not matched with user query) and non-matching columns of user query (*NMu*- columns in user query that are not matched with cached query). Some other information like; data value of cache predicate (*DVc*), data value of user predicate, (*DVu*), comparison operator in cache predicate (*Opc*), comparison operator in user predicate

*DA:= Attributes exist in Qs but not in Schema*

**Algorithm 5**: *1st\_Level\_Query\_Rewritter* (*QS)* 

 *CA:= Attributes exist in both Qs and Schema* 

INPUT: *QS* (*SELECT* Clause)

*Return CA, DA*

OUTPUT: rq1, *CA* PROCEDURE:

(*Opu*). Algorithm to extract the semantics of predicate is given below in figure 6.

1. *MC [n]*:= List of Columns Present in both *QW , SW* 2. *NMCC[n]*:= List of Columns Present in *SW* but not in *QW* 3. *NMCU [n]*:= List of columns present in *QW* but not in *SW*

{*Coc[n], Cou[n], MC[n],, NMCC[n],, NMCU[n], Opc[n], Opu[n], Dvc[n], Dvu[n]*}

10. return *Coc[n], Cou[n], MC[n],, NMCC[n],, NMCU[n], Opc[n], Opu[n],* 

Fig. 6. Algorithm to Extract Semantics from User Query.

*Dvc[n], Dvu[n]*

**Algorithm 6:** Semantics Extractor

4. *Opc[n]* := operator set of *SW* 5. *Opu[n]* := Operator present set of *QW* 6. *Dvc[n]* := Data values in *SW* 7. *Dvu[n]* := Data values in *QW* 8. *Coc[n]*:= Connective Operators in *Sw* 9. *Cou[n]*:= Connective Operators in *Qw*

**Input:** *QW ,SW* **Output:** 

**Procedure:** 

**4.2 Query trimming** 

be computed as follow:

Fig. 5. Algorithm for 1st Level Query Rewriter.

rq1 = πDAσQP(QR)

### **4.1 Query matching**

In semantic cache, user posed query is matched with the stored semantics on cache. In this process the decision is taken place either data is available at cache or not. Query matching process is accomplished in two sub process splitter and rejecter. Splitter will accept the user query *QU* from the user interface and splits the query on the base of three clauses (*SELECT, FROM, WHERE*) of the query. These three portions are called *QS* (*SELECT: projected attributes in the user query*), *QF* (*FROM*: Relation) and *QW* (*WHERE*: selected rows/tupples on specific condition); and send to the rejecter for initial level checking. *QW* will be empty if there is no condition on user posed query (Ahmad et al, 2009). Algorithm for splitting the user query is presented by Ahmad et al. (Ahmad et al, 2009) and given in figure 3.

Fig. 3. Algorithm to Split Query.

Responsibility of rejecter is to checks the validity of user posed query by sending the list of selected attributes (*QS*), relation (*QF*) and predicate attributes (*PA*) on the schema based indexing semantics. Predicate attribute is extracted by rejecter from *QW* and included in the list. If attributes list of QS, *QF* and *PA* matched with stored schema then processing will be continued otherwise query will be rejected and processing will be stopped. Rejecter also builds *QS* in the case of '\*' by retrieving all attributes from schema as a list if predicate attribute exist in schema (Ahmad et al, 2009). Algorithm to validate the user query is presented by Ahmad et al. (Ahmad et al, 2009) and given in figure 4.

```
Algorithm 4: CHECK_REJECTION (QS, QF, PA) 
INPUT: QS, QF, PA
OUTPUT: True/False( 
PROCEDURE: 
1. If all attributes of QS present in schema
          If relation of QF present in schema
                 If PA is present in schema 
                  If(Qs='*')
                         return false and build Qs from schema 
                    Else return true 
 Else return true 
 Else return true 
Else return true
```
Fig. 4. Algorithm to Reject Incorrect Queries.

In semantic cache, user posed query is matched with the stored semantics on cache. In this process the decision is taken place either data is available at cache or not. Query matching process is accomplished in two sub process splitter and rejecter. Splitter will accept the user query *QU* from the user interface and splits the query on the base of three clauses (*SELECT, FROM, WHERE*) of the query. These three portions are called *QS* (*SELECT: projected attributes in the user query*), *QF* (*FROM*: Relation) and *QW* (*WHERE*: selected rows/tupples on specific condition); and send to the rejecter for initial level checking. *QW* will be empty if there is no condition on user posed query (Ahmad et al, 2009). Algorithm for splitting the user query is

Responsibility of rejecter is to checks the validity of user posed query by sending the list of selected attributes (*QS*), relation (*QF*) and predicate attributes (*PA*) on the schema based indexing semantics. Predicate attribute is extracted by rejecter from *QW* and included in the list. If attributes list of QS, *QF* and *PA* matched with stored schema then processing will be continued otherwise query will be rejected and processing will be stopped. Rejecter also builds *QS* in the case of '\*' by retrieving all attributes from schema as a list if predicate attribute exist in schema (Ahmad et al, 2009). Algorithm to validate the user query is

*QS := SELECT CLAUSE*

presented by Ahmad et al. (Ahmad et al, 2009) and given in figure 3.

**Algorithm 2**: *SPLIT*\_*QUERY( )*  INPUT: QU (User query) OUTPUT: *QS, QW, QF* PROCEDURE:

 *QW := WHERE CLAUSE QF := FROM CLAUSE*

Return *QS, QW, QF.* 

presented by Ahmad et al. (Ahmad et al, 2009) and given in figure 4.

**Algorithm 4**: *CHECK*\_*REJECTION* (*QS, QF, PA)* 

1. If all attributes of *QS present in schema*

 *Else return true Else return true* 

*If relation of QF present in schema If PA is present in schema* 

*Else return true* 

*return false and build Qs from schema* 

 *If(Qs='\*')*

**4.1 Query matching** 

Fig. 3. Algorithm to Split Query.

Fig. 4. Algorithm to Reject Incorrect Queries.

*Else return true*

INPUT: *QS, QF, PA* OUTPUT: True/False( PROCEDURE:

## **4.2 Query trimming**

When it has been decided that data is available at cache then second step of sCacheQP is performed. In this step query is divided into two sub queries called probe and remainder queries called query trimming. This process accomplished in two stages. At first stage vertical partition takes place and the attributes that are not available (DA) at cache directly sent to the server as rq1 (remainder query) with original predicate. We called it 1st level query rewriter (Ahmad et al, 2009) and its algorithm is given in figure 5. The query rq1 will be computed as follow:

rq1 = πDAσQP(QR)

**Algorithm 5**: *1st\_Level\_Query\_Rewritter* (*QS)*  INPUT: *QS* (*SELECT* Clause) OUTPUT: rq1, *CA* PROCEDURE:  *CA:= Attributes exist in both Qs and Schema DA:= Attributes exist in Qs but not in Schema Return CA, DA*

Fig. 5. Algorithm for 1st Level Query Rewriter.

Rest of attributes; that are common in both user and cached query forwarded to the predicate processor which worked at second stage. Predicate processor consists of four sub modules; semantic extractor, Explicit Semantic Matcher, Implicit Semantic Matcher, and Predicate Merger. At this stage predicate is simplified by just separating the portions of it on the base of conjunct and disjunct operators. Then semantics of user's query predicate with respect to the cached predicate is extracted in the form of matching columns (*Mc*- similar in both user query predicate and cached predicate), non-matching columns of cache (*NMc*columns in cached query that are not matched with user query) and non-matching columns of user query (*NMu*- columns in user query that are not matched with cached query). Some other information like; data value of cache predicate (*DVc*), data value of user predicate, (*DVu*), comparison operator in cache predicate (*Opc*), comparison operator in user predicate (*Opu*). Algorithm to extract the semantics of predicate is given below in figure 6.

```
Algorithm 6: Semantics Extractor 
Input: QW ,SW
Output: 
{Coc[n], Cou[n], MC[n],, NMCC[n],, NMCU[n], Opc[n], Opu[n], Dvc[n], Dvu[n]} 
Procedure: 
 1. MC [n]:= List of Columns Present in both QW , SW
  2. NMCC[n]:= List of Columns Present in SW but not in QW
  3. NMCU [n]:= List of columns present in QW but not in SW
  4. Opc[n] := operator set of SW
  5. Opu[n] := Operator present set of QW 
  6. Dvc[n] := Data values in SW
  7. Dvu[n] := Data values in QW 
  8. Coc[n]:= Connective Operators in Sw
  9. Cou[n]:= Connective Operators in Qw 
  10. return Coc[n], Cou[n], MC[n],, NMCC[n],, NMCU[n], Opc[n], Opu[n], 
    Dvc[n], Dvu[n]
```
Fig. 6. Algorithm to Extract Semantics from User Query.

Semantic Cache System 95

As we have discussed that Explicit Semantic Matching algorithm is based on the boundary value and basic comparison operator. On the base of boundary value and comparison

Remember that predicate matching algorithm having better time complexity is an alternative of satisfiability/implication (Guo et al, 1996) used to help process query in the literature (Ren et al, 2003, Jonsson et al, 2006). Computed values C1, NC1 and NMu sent to the Implicit Semantic Matching algorithm to remove the additional information. Algorithm

operators; algorithm 7 will trim the predicate into probe and remainder queries.

*Mc[n]*, *NMCC[n], NMCU[n], C1[n], NC1[n]* 

1. If (*NMCC[i]* = null) and (*NMCU[i]* = null) then

2. Else If (*NMCC[i]* !=null) and (*NMCU[i]* = null) then

4. Else If (*NMCC[i]* != null) and (*NMCU[i]* != null) then a. *C2:= (C1[i]) + (NMCU[i]) + (NMc[i])* 

*b.NC2:=((C1[i])+R(NMCC[i])+NMCU[i])V((NC1[i])+( NMCU[i]))* 

N-Cached= V(Ci ΛNCj) where 1<i,j<n and i!=j

Cached = VCi where 1<i<n N-Cached= VNCi where 1<i<n

N-Cached= V((Ci ΛNCj) V(UiΛR(Uj)) wher1<i,j<n and i!=j

N-Cached= V(Ci ΛNCj) where 1<i,j<n and i!=j

b. *NC2 := ((C1[i]+ R (NMCC[i]))V(NC1[i])* 3. Else If (*NMCC[i]* = null) and (*NMCU[i]* != null) then

8 is used to perform this job that is given below in figure 8.

*C2, NC2*

a. *C2:= C1[i]* b. *NC2 := NC1[i]*

**Algorithm 8**: *ImplicitSemanticMatching*

Initiliaze *C2:=Null, NC2:=Null,i:=Null* 

a. *C2:= (C1[i]) + (NMCC[i])*

a. *C2:= (C1[i]) + (NMCU[i]) b. NC2 := NC1[i] + NMCU[i]* 

5. Else If (*Mc[i] =* = null) then *a. C2:= (NMCC[i]) + (NMCU[i]) b. NC2 := NMCU[i] + R(NMCC[i])* 

**Algorithm 9**: *PredicateMerging*

*C2, NC2, Coc, Cou* 

*Cached, N-Cached* 

Cached = ΛCi

Cached = ΛCi

Fig. 8. Algorithm for Implicit Matching.

Input:

Output:

Procedure: If (*Coc &Cou)є* **Λ** Cached = ΛCi

If(*Coc &Cou)є***V**

If(*Coc є***V** *&Couє*Λ)

If(*Cou є***V** *&Cocє*Λ)

Input:

Output:

Procedure:

Repeat from i to n

Fig. 9. Algorithm to Merge the Predicate.

```
Algorithm 7: ExplicitSemanticMatching
Input:{Mc[n], OPc[n], OPu[n], DVc[n], DVu[n], CC[n i], CU[n]} 
Output: {C1[n], NC1[n]} 
Method: 
Initilaize: C1[n]:=Null, NC1[n]:=Null, i:=0 
Repeat from i to n 
If(DVc[i] < DVu[i]) 
    If((OPc[i] є{!=, >,>=}Λ OPu[i] є{>, >=,=})) 
     C1[i]←CC[i] OPc[i] DVc[i] 
     NC1[i]←Null 
    else if(OPu[i] є{<, <=, !=}) 
     C1[i] ←CC [i]OPc[i] DVc[i] 
            NC1[i]←(CU[i] OPu[i]DVu[i])∧(CC[i]Rev(OPc[i]) DVc[i]) 
 else 
           C1[i] ←Null 
 NC1[i]←(CU[i] OPu[i] DVu[i]) 
else if(DVc[i] >DVu[i]) 
 if((OPc[i]є{!=,<,<=}ΛOPu[i]є{<,<=,=})) 
           C1[i] ←CC[i] OPc [i]DVc[i] 
     NC1[i]←Null 
 else if(((OPc[i] є{!=, >, =, <=, <}ΛOPu[i]є{>, >=, !=})) 
     C1[i] ←CC[i] OPc[i] DVc[i] 
     NC1[i]←(CU[i] OPu[i] DVu[i])∧(CC[i]Rev(OPc[i]) DVc[i]) 
    else
           C1[i] ←Null 
     NC1[i]←(CU[i] OPu[i]DVu[i]) 
else if((DVc[i] = DVu[i]) 
 if(((OPc[i]є{>=}Λ( OPu[i] є{ >, =})) 
 V (OPc[i] = OPu[i])V((OPc[i] є{<=})Λ (OPu[i] є{<,=})) 
 V (OPc[i] є{!=}Λ OPu[i]є{<,>})) 
            C1 ←CC OPc DVc 
            NC1←Null 
 else if(((OPc[i] є{>, <=}Λ OPu[i] є{ >=, !=})
 V (OPc[i]є{<, >=}ΛOPu[i]є{<=, !=}) 
 V (OPc[i]є{!=,=}ΛOPu[i]є{<=, >=})) 
            C1[i] ←CC[i] OPc [i]DVc[i] 
             NC1[i]←(CU[i] OPu[i] DVu[i])∧(CC[i] Rev(OPc[i]) DVc[i]) 
 else 
           C1[i] ←Null 
 NC1[i]←(CU[i] OPu[i] DVu[i]) 
else if((DVc[i]!= DVu[i]) Λ ((OPc[i] є{=, !=}Λ OPu[i] є{!=}))
            C1[i] ←CC[i] OPc[i] DVc[i] 
             NC1[i]←(CU[i] OPu[i] DVu[i])∧(CC[i] Rev(OPc[i]) DVc[i]) 
else 
           C1[i] ←Null 
            NC1←(CU[i] OPu[i] DVu[i])
```
Fig. 7. Algorithm to Evaluate Predicate.

After extraction of semantics *Mc, DVc, DVu, Opc,* and *Opu* sent to the Explicit Semantic Matcher. Explicit Semantic Matcher trims the predicate into two portions; one for remainder (*C1*) and other for probe query (*NC1*). Explicit Semantic Matching algorithm is based on the boundary values as well as on the nature of comparison operators. There are 112 rules defined on the base of boundary values and basic comparison operators (<, <=, >, >=, = =, !=). Algorithm 7 given in figure 7 is used to match and trims the predicate. The output of predicate matching algorithm is predicate that is available at cache (C1) and predicate that is not available at cache (NC1). Working of the algorithm is explained above.

*NC1[i]*←(*CU[i] OPu[i]DVu[i]*)∧(*CC[i]Rev*(*OPc[i]*) *DVc[i]*)

*Input:{Mc[n], OPc[n], OPu[n], DVc[n], DVu[n], CC[n i], CU[n]}* 

*Algorithm 7: ExplicitSemanticMatching*

*Initilaize: C1[n]:=Null, NC1[n]:=Null, i:=0* 

*If((OPc[i] є{!=, >,>=}Λ OPu[i] є{>, >=,=})) C1[i]*←*CC[i] OPc[i] DVc[i]* 

*Output: {C1[n], NC1[n]}* 

 *NC1[i]*←Null *else if(OPu[i] є{<, <=, !=}) C1[i]* ←*CC [i]OPc[i] DVc[i]* 

*else if(DVc[i] >DVu[i])* 

*else if((DVc[i] = DVu[i])* 

*C1[i]* ←Null *NC1[i]*←(*CU[i] OPu[i] DVu[i]*)

 *NC1[i]*←Null

 *if((OPc[i]є{!=,<,<=}ΛOPu[i]є{<,<=,=})) C1[i]* ←*CC[i] OPc [i]DVc[i]* 

 *C1[i]* ←*CC[i] OPc[i] DVc[i]* 

*C1[i]* ←Null *NC1[i]*←(*CU[i] OPu[i]DVu[i]*)

 *if(((OPc[i]є{>=}Λ( OPu[i] є{ >, =}))* 

 *V (OPc[i] є{!=}Λ OPu[i]є{<,>})) C1* ←*CC OPc DVc NC1*←Null  *else if(((OPc[i] є{>, <=}Λ OPu[i] є{ >=, !=}) V (OPc[i]є{<, >=}ΛOPu[i]є{<=, !=}) V (OPc[i]є{!=,=}ΛOPu[i]є{<=, >=})) C1[i]* ←*CC[i] OPc [i]DVc[i]* 

*C1[i]* ←Null *NC1[i]*←(*CU[i] OPu[i] DVu[i]*)

*C1[i]* ←Null

 *else if(((OPc[i] є{!=, >, =, <=, <}ΛOPu[i]є{>, >=, !=}))* 

 *V (OPc[i] = OPu[i])V((OPc[i] є{<=})Λ (OPu[i] є{<,=}))* 

*else if((DVc[i]!= DVu[i]) Λ ((OPc[i] є{=, !=}Λ OPu[i] є{!=})) C1[i]* ←*CC[i] OPc[i] DVc[i]* 

*NC1*←(*CU[i] OPu[i] DVu[i]*)

 *NC1[i]*←(*CU[i] OPu[i] DVu[i]*)∧(*CC[i]Rev*(*OPc[i]*) *DVc[i]*)

*Repeat from i to n If(DVc[i] < DVu[i])* 

*Method:* 

 *else* 

*else*

After extraction of semantics *Mc, DVc, DVu, Opc,* and *Opu* sent to the Explicit Semantic Matcher. Explicit Semantic Matcher trims the predicate into two portions; one for remainder (*C1*) and other for probe query (*NC1*). Explicit Semantic Matching algorithm is based on the boundary values as well as on the nature of comparison operators. There are 112 rules defined on the base of boundary values and basic comparison operators (<, <=, >, >=, = =, !=). Algorithm 7 given in figure 7 is used to match and trims the predicate. The output of predicate matching algorithm is predicate that is available at cache (C1) and predicate that is

 *NC1[i]*←(*CU[i] OPu[i] DVu[i]*)∧(*CC[i] Rev*(*OPc[i]*) *DVc[i]*)

 *NC1[i]*←(*CU[i] OPu[i] DVu[i]*)∧(*CC[i] Rev*(*OPc[i]*) *DVc[i]*)

not available at cache (NC1). Working of the algorithm is explained above.

Fig. 7. Algorithm to Evaluate Predicate.

*else* 

 *else* 

As we have discussed that Explicit Semantic Matching algorithm is based on the boundary value and basic comparison operator. On the base of boundary value and comparison operators; algorithm 7 will trim the predicate into probe and remainder queries.

Remember that predicate matching algorithm having better time complexity is an alternative of satisfiability/implication (Guo et al, 1996) used to help process query in the literature (Ren et al, 2003, Jonsson et al, 2006). Computed values C1, NC1 and NMu sent to the Implicit Semantic Matching algorithm to remove the additional information. Algorithm 8 is used to perform this job that is given below in figure 8.

```
Algorithm 8: ImplicitSemanticMatching
Input: 
            Mc[n], NMCC[n], NMCU[n], C1[n], NC1[n] 
Output: 
            C2, NC2
Procedure: 
Initiliaze C2:=Null, NC2:=Null,i:=Null 
Repeat from i to n 
     1. If (NMCC[i] = null) and (NMCU[i] = null) then 
          a. C2:= C1[i]
          b. NC2 := NC1[i]
     2. Else If (NMCC[i] !=null) and (NMCU[i] = null) then 
          a. C2:= (C1[i]) + (NMCC[i])
          b. NC2 := ((C1[i]+ R (NMCC[i]))V(NC1[i])
     3. Else If (NMCC[i] = null) and (NMCU[i] != null) then 
          a. C2:= (C1[i]) + (NMCU[i]) 
          b. NC2 := NC1[i] + NMCU[i] 
     4. Else If (NMCC[i] != null) and (NMCU[i] != null) then 
          a. C2:= (C1[i]) + (NMCU[i]) + (NMc[i]) 
          b.NC2:=((C1[i])+R(NMCC[i])+NMCU[i])V((NC1[i])+( NMCU[i])) 
     5. Else If (Mc[i] = = null) then 
          a. C2:= (NMCC[i]) + (NMCU[i]) 
          b. NC2 := NMCU[i] + R(NMCC[i])
```
Fig. 8. Algorithm for Implicit Matching.

```
Algorithm 9: PredicateMerging
Input: 
          C2, NC2, Coc, Cou 
Output: 
          Cached, N-Cached 
Procedure: 
If (Coc &Cou)є Λ
 Cached = ΛCi 
 N-Cached= V(Ci ΛNCj) where 1<i,j<n and i!=j 
If(Coc &Cou)єV 
          Cached = VCi where 1<i<n 
          N-Cached= VNCi where 1<i<n 
If(Coc єV &CouєΛ) 
          Cached = ΛCi 
          N-Cached= V(Ci ΛNCj) where 1<i,j<n and i!=j 
If(Cou єV &CocєΛ) 
          Cached = ΛCi 
          N-Cached= V((Ci ΛNCj) V(UiΛR(Uj)) 
                     wher1<i,j<n and i!=j
```
Fig. 9. Algorithm to Merge the Predicate.

Semantic Cache System 97

SR S SA SP SC

S8 e\_ID, eName Sal Age

S10 e\_ID, eName Age

S22 sName, Grade, Gender

Schema based hierarchal scheme reduces the number of comparisons to find out whether data is available at cache or not. Only 'n' comparisons are required to check availability of data on cache. Table 1 can be rearranged according to our proposed schema based semantic

**Names Fields Status Condition Content** 

eName True P1 1 Age false Null Null Sal True P2 2 e\_ID True P3 1

Gender false Null Null Grade false Null Null sName false Null Null

S1 e\_ID P1 1 S2 eName P1 2 S3 Sal P1 3 S4 Age P1 4 S5 e\_ID, eName P3 5 S6 e\_ID, Sal P4 6 S7 e\_ID, Age P5 7

S9 e\_ID Sal Age P7 9

S11 eName Sal P9 11 S12 eName Age P10 12 S13 Sal Age,eName P11 13 S14 e\_ID, eName, Sal P12 14 S15 Sal, Age P3 15

S16 sName P21 16 S17 Grade P21 17 S18 Gender P21 18 S19 Gender, Grade P22 19 S20 Gender, sName P23 20 S21 sName, Grade P24 21

P6 8

P8 10

P25 22

employee

students

Table 1. Possible segments for Given Database.

**Table** 

Employee

Students

indexing scheme as in Table 2.

DB Name

University

Table 2. Schema Based Indexing.

Generated cached (C2) and non-cached (NC2) predicates by the Implicit Semantic Matcher are combined by predicate merger. Algorithm to merge the predicate is given in figure 9.

The computed predicates are then sent to the 2nd level query rewriter. Finally, probe and remainder queries will be computed by the 2nd level query rewriter as follow:

pq = *SELECT* CA From QF *WHERE* Ccache rq2 = *SELECT* CA FROM QF *WHERE* N-Cached

pq will be executed locally ad rq will sent to the server. Then result of both will be sent to the rebuilder to combine the result.

## **4.3 Query rebuilding**

Rebuilder receives the result form server (SR) which is retrieved across remainder queries (rq1 and rq2) and results from cache (*CR*) across probe query (pq) combines both as a final result *FR.* Final result is viewed to the user and also updated in the cache contents if required.

## **5. Case study**

To validate our proposed semantic indexing and query processing, we consider the case study of university.

Figure 10 presents the schema of university with two relations employee and students having 4 and 3 fields respectively.

Fig. 10. Schema for University.

For the above given schema of university; there are 15 and 7 segments possible across employee and students relations respectively according to previous work (Ren et al., 2003). In simple words, we can say that there are 15 queries are possible against employee and similarly 7 for students as given in table 1.

In the above example there are 22 possible queries that make separate segments. So the formula to calculate possible segments across a single relation over 'n' attributes is "2n-1". Then add segments across each relation. As in example 15+7 =22: (24-1=15 &23-1=7). Hence, 22 segments are to be visited to check availability of data on cache in the worst case which increases the response time drastically.

Generated cached (C2) and non-cached (NC2) predicates by the Implicit Semantic Matcher are combined by predicate merger. Algorithm to merge the predicate is given in figure 9.

The computed predicates are then sent to the 2nd level query rewriter. Finally, probe and

pq will be executed locally ad rq will sent to the server. Then result of both will be sent to

Rebuilder receives the result form server (SR) which is retrieved across remainder queries (rq1 and rq2) and results from cache (*CR*) across probe query (pq) combines both as a final result *FR.* Final result is viewed to the user and also updated in the cache contents if

To validate our proposed semantic indexing and query processing, we consider the case

Figure 10 presents the schema of university with two relations employee and students

For the above given schema of university; there are 15 and 7 segments possible across employee and students relations respectively according to previous work (Ren et al., 2003). In simple words, we can say that there are 15 queries are possible against employee and

In the above example there are 22 possible queries that make separate segments. So the formula to calculate possible segments across a single relation over 'n' attributes is "2n-1". Then add segments across each relation. As in example 15+7 =22: (24-1=15 &23-1=7). Hence, 22 segments are to be visited to check availability of data on cache in the worst case which

remainder queries will be computed by the 2nd level query rewriter as follow:

pq = *SELECT* CA From QF *WHERE* Ccache rq2 = *SELECT* CA FROM QF *WHERE* N-Cached

the rebuilder to combine the result.

**4.3 Query rebuilding** 

required.

**5. Case study** 

study of university.

having 4 and 3 fields respectively.

Fig. 10. Schema for University.

similarly 7 for students as given in table 1.

increases the response time drastically.


Table 1. Possible segments for Given Database.

Schema based hierarchal scheme reduces the number of comparisons to find out whether data is available at cache or not. Only 'n' comparisons are required to check availability of data on cache. Table 1 can be rearranged according to our proposed schema based semantic indexing scheme as in Table 2.


Table 2. Schema Based Indexing.

Semantic Cache System 99

**e\_ID eName Age Sal** 

114 Adeel 31 42000

115 Komal 37 17000

116 Mahreen 39 30450

117 Tabinda 39 28850

118 Yaseen 40 24450

119 Anees 45 30000

120 Komal 50 30000

All of three queries should be rejected at initial level; but according to all of previous work

*1.SELECT* eName, Age *FROM* emMloyee *WHERE* age>30

*2.SELECT* eName, Age *FROM* employee *WHERE* gpa>3.0

*3.SELECT* ename, rollno *FROM* employee *WHERE* age>30

It is a beauty of our proposed schema based indexing scheme that all of three queries will be rejected and query processing time will be saved. According to our proposed semantic caching architecture list of projected attributes (eName, Age in first query), relation (emMloyee in first query) and predicate attribute is checked from schema based indexing scheme; and query will be rejected due to unavailability of "emMloyee" relation in schema. Similarly, query 2 and 3 will be rejected due to unavailability of "gpa" and "rollno" in

By rejecting query at initial stage; query processing can be saved. In this context our

**Case-II:** In this case we will take an example that covers the handling of queries having \* in

Let us consider that user has already posed the following query and result has been stored

*SELECT* eName, Age *FROM* employee *WHERE* age>30

Data for above query will be retrieved and stored on cache will be as given in table 5.

Table 4. Contents on cache in case-I.

SELECT CLAUSE.

in cache.

Now let us consider user is going to pose following three queries.

(predicate attribute is incorrect)

(Projected attribute is incorrect)

(relation is incorrect)

query will be posted on server due to unavailability of data on cache.

employee table respectively and there will be no probe and remainder.

proposed semantic caching scheme has better performance than previous.

Table 2 represents structure of schema based semantic indexing instead of actual contents. There is only need to compare/match 4 and 3 fields instead of 15 and 7 segments respectively according to previous work. Also it has the ability to reject invalid queries at initial level instead of further processing.

For detailed discussion and simplicity, we consider only employee table of university database. Let us consider there is employee table on server with 4 fields defined in university schema in Figure 10. Employee table on server is given in table 3 below.


Table 3. Employee Table on Database.

Now we divide our case study into five cases in such a way that one can easily understand our contribution and novelty of our approach. For simplicity, each of five cases is discussed standalone and not linked with other. Each case should be considered separately. We have considered that cache is managed from initial for each case.

Case-I: In this case we will take an example that covers the query rejection at initial level.

Let us consider that user has already posed the following query and result has been stored in cache.

*SELECT* \* *FROM* employee *WHERE* age>30

Data on cache will be as given in table 4.

Table 2 represents structure of schema based semantic indexing instead of actual contents. There is only need to compare/match 4 and 3 fields instead of 15 and 7 segments respectively according to previous work. Also it has the ability to reject invalid queries at

For detailed discussion and simplicity, we consider only employee table of university database. Let us consider there is employee table on server with 4 fields defined in

**e\_ID eName Age Sal** 

110 Asad 20 25000

111 Ali 22 22000

112 Kashif 25 25000

113 Abid 30 15000

114 Adeel 31 42000 115 Komal 37 17000

116 Mahreen 39 30450

117 Tabinda 39 28850

118 Yaseen 40 24450

119 Anees 45 30000

120 Komal 50 30000

Now we divide our case study into five cases in such a way that one can easily understand our contribution and novelty of our approach. For simplicity, each of five cases is discussed standalone and not linked with other. Each case should be considered separately. We have

Case-I: In this case we will take an example that covers the query rejection at initial level.

Let us consider that user has already posed the following query and result has been stored

*SELECT* \* *FROM* employee *WHERE* age>30

university schema in Figure 10. Employee table on server is given in table 3 below.

initial level instead of further processing.

Table 3. Employee Table on Database.

Data on cache will be as given in table 4.

in cache.

considered that cache is managed from initial for each case.


Table 4. Contents on cache in case-I.

Now let us consider user is going to pose following three queries.

*1.SELECT* eName, Age *FROM* emMloyee *WHERE* age>30 (relation is incorrect) *2.SELECT* eName, Age *FROM* employee *WHERE* gpa>3.0 (predicate attribute is incorrect) *3.SELECT* ename, rollno *FROM* employee *WHERE* age>30 (Projected attribute is incorrect)

All of three queries should be rejected at initial level; but according to all of previous work query will be posted on server due to unavailability of data on cache.

It is a beauty of our proposed schema based indexing scheme that all of three queries will be rejected and query processing time will be saved. According to our proposed semantic caching architecture list of projected attributes (eName, Age in first query), relation (emMloyee in first query) and predicate attribute is checked from schema based indexing scheme; and query will be rejected due to unavailability of "emMloyee" relation in schema. Similarly, query 2 and 3 will be rejected due to unavailability of "gpa" and "rollno" in employee table respectively and there will be no probe and remainder.

By rejecting query at initial stage; query processing can be saved. In this context our proposed semantic caching scheme has better performance than previous.

**Case-II:** In this case we will take an example that covers the handling of queries having \* in SELECT CLAUSE.

Let us consider that user has already posed the following query and result has been stored in cache.

*SELECT* eName, Age *FROM* employee *WHERE* age>30

Data for above query will be retrieved and stored on cache will be as given in table 5.

Semantic Cache System 101

After computation of predicate semantics, predicate for probe and remainder query will be computed by using predicate matching algorithm (actually 112 rules are used here). At first, main class of algorithm is selected, here data value of user (DVu=30) is equal to the data value of cache (DVc = 30). So, class 3 of predicate matching will be selected then priority of relational operator will be computed. Relational operator in both queries is '>'; it is low priority (defined in previous work; (Bashir and Qadir, 2007a)) operator. So, following porton

MC = Age NMC = NULL NMU = NULL

So predicate for probe and remainder will be computed we say it C1 (for cached) and NC1

*If((OPc є{!=, >,>=}Λ OPu є{>, >=,=}))* 

 *C1* ←*CC OPc DVc NC1*←Null

Due to simple predicate rules defined for complex queries will not be applied. Finally subtraction algorithm will be applied to generate final predicate for probe and remainder queries. As it is computed that NMc and NMu both are Null. So, first case of subtraction

So, there will be no change in predicate of probe and remainder query. Then, probe query

pq = SELECT eName, Age FROM employee WHERE Age>30

1. **If** (*NMc* = null) and (*NMu* = null)

(pq) and second remainder query (rq2) will be generated as below.

**then** a. **C**<sup>2</sup> := *C1* b. **N**C2 := *NC1*

C1 =Age>30 NC1 = Null

In the last step result of rq1, pq and rq2 is combined by rebuilder. **Case-III:** In this case generation of amending query is elaborated.

rq2 = Null

First of semantics of predicate will be computed by semantic extractor as follow.

of the algorithm will be executed. given below.

Here, Cc is Age, operator is '>' and DVc is 30.

(for non-cached).

algorithm will be applied.


Table 5. Contents on cache in case-II.

Note that e\_ID is not required but retrieved. It is due to the requirement of key-contained (Ren et al., 2003) contents. Now let us assume that user has posed the following query.

*SELECT* \* *FROM* employee *WHERE* age>30

Now all of the fields of employee required; but according to previous work common set will be calculated (intersection of cached attributes and user's query attributes). There is no way defined to calculate the common set of '\*' and some attributes. Here we can say that all of the cached attributes for employee are required, but how can it be decided which of the attributes are not in cache and should retrieved from server.

Here again we need schema at cache (first we need schema for zero level query rejection). If schema is available at cache then SELECT CLAUSE with '\*' can be handled easily. By this hit ratio is improved. Splitter splits the query and sent it to the rejecter. Rejecter checks the list of fields with relation and predicate attribute from schema based indexing semantics. Query will not be rejected due to availability of all member of list at schema. Common and difference set of attributes will be computed and sent to the 1st level query matcher. i.e. CA and DA will be computed. Remainder query (rq1) with difference attributes (here is only one difference attribute that is 'Sal') will be generated by 1st level query matcher like below.

rq1 = *SELECT* Sal *FROM* employee *WHERE* Age>30

Common attributes (e\_ID, Age, eName) will be sent to the Query Generator (QG). Query Generator will generate probe and remainder query on the base of predicate matching. Conditioned attribute (Age) is already retrieved; so there is no need of amending query in this case.

**e\_ID eName Age**

114 Adeel 31

115 Komal 37

116 Mahreen 39

117 Tabinda 39

118 Yaseen 40

119 Anees 45

120 Komal 50

Note that e\_ID is not required but retrieved. It is due to the requirement of key-contained (Ren et al., 2003) contents. Now let us assume that user has posed the following query.

Now all of the fields of employee required; but according to previous work common set will be calculated (intersection of cached attributes and user's query attributes). There is no way defined to calculate the common set of '\*' and some attributes. Here we can say that all of the cached attributes for employee are required, but how can it be decided which of the

*SELECT* \* *FROM* employee *WHERE* age>30

Here again we need schema at cache (first we need schema for zero level query rejection). If schema is available at cache then SELECT CLAUSE with '\*' can be handled easily. By this hit ratio is improved. Splitter splits the query and sent it to the rejecter. Rejecter checks the list of fields with relation and predicate attribute from schema based indexing semantics. Query will not be rejected due to availability of all member of list at schema. Common and difference set of attributes will be computed and sent to the 1st level query matcher. i.e. CA and DA will be computed. Remainder query (rq1) with difference attributes (here is only one difference attribute that is 'Sal') will be generated by 1st level query matcher like below.

Common attributes (e\_ID, Age, eName) will be sent to the Query Generator (QG). Query Generator will generate probe and remainder query on the base of predicate matching. Conditioned attribute (Age) is already retrieved; so there is no need of amending query in

rq1 = *SELECT* Sal *FROM* employee *WHERE* Age>30

Table 5. Contents on cache in case-II.

this case.

attributes are not in cache and should retrieved from server.

First of semantics of predicate will be computed by semantic extractor as follow.

$$\begin{array}{c} \text{M}\_{\text{C}} = \text{Age} \\ \text{N} \text{M}\_{\text{C}} = \text{N} \text{ULL} \\ \text{N} \text{M}\_{\text{U}} = \text{N} \text{ULL} \end{array}$$

After computation of predicate semantics, predicate for probe and remainder query will be computed by using predicate matching algorithm (actually 112 rules are used here). At first, main class of algorithm is selected, here data value of user (DVu=30) is equal to the data value of cache (DVc = 30). So, class 3 of predicate matching will be selected then priority of relational operator will be computed. Relational operator in both queries is '>'; it is low priority (defined in previous work; (Bashir and Qadir, 2007a)) operator. So, following porton of the algorithm will be executed. given below.

*If((OPc є{!=, >,>=}Λ OPu є{>, >=,=})) C1* ←*CC OPc DVc NC1*←Null

Here, Cc is Age, operator is '>' and DVc is 30.

So predicate for probe and remainder will be computed we say it C1 (for cached) and NC1 (for non-cached).


Due to simple predicate rules defined for complex queries will not be applied. Finally subtraction algorithm will be applied to generate final predicate for probe and remainder queries. As it is computed that NMc and NMu both are Null. So, first case of subtraction algorithm will be applied.

> 1. **If** (*NMc* = null) and (*NMu* = null) **then** a. **C**<sup>2</sup> := *C1* b. **N**C2 := *NC1*

So, there will be no change in predicate of probe and remainder query. Then, probe query (pq) and second remainder query (rq2) will be generated as below.

> pq = SELECT eName, Age FROM employee WHERE Age>30 rq2 = Null

In the last step result of rq1, pq and rq2 is combined by rebuilder.

**Case-III:** In this case generation of amending query is elaborated.

Semantic Cache System 103

Let us consider that user has already posed the following query and result has been stored

**e\_ID eName Age**

114 Adeel 31

115 Komal 37

116 Mahreen 39

117 Tabinda 39

118 Yaseen 40

119 Anees 45

120 Komal 50

Note that e\_ID is not required but retrieved. It is due to the requirement of key-contained (Ren et al., 2003) contents. Now let us assume that user has posed the following query.

Now all of the required fields are matched with cached query. Splitter splits the query and sent it to the rejecter. Rejecter checks the list of fields with relation and predicate attribute from schema based indexing semantics. Query will not be rejected due to availability of all member of list at schema. Common and difference set of attributes will be computed and sent to the 1st level query matcher. i.e. CA and DA will be computed. Remainder query (rq1) will be null due to empty set of difference attributes (All required attributes are exist on

*SELECT* eName, Age *FROM* employee

*WHERE* eName = 'Komal'

Common attributes (Age, eName) will be sent to the Query Generator (QG). Query Generator will generate probe and remainder query on the base of predicate matching. Conditioned attribute (Age) is already retrieved; so there is no need of amending query in

rq1 = *Null*

cache). So, remainder query by 1st level query matcher will be like below.

First of semantics of predicate will be computed by semantic extractor as follow.

Data for above query will be retrieved and stored on cache will be as given in table 7.

*SELECT* eName, Age *FROM* employee *WHERE* Age>30

in cache.

this case.

Table 7. Contents on cache in case-IV.

Let us consider that user has already posed the following query and result has been stored in cache.

*SELECT* eName, Sal *FROM* employee *WHERE* age>30

Data for above query will be retrieved and stored on cache will be as given in table 6.


Table 6. Contents on cache in case-III.

Note that e\_ID is not required but retrieved. It is due to the requirement of key-contained (Ren et al., 2003) contents. Now let us assume that user has posed the following query.

```
SELECT eName, Sal FROM employee WHERE age>35
```
Here, generation of amending query is discussed. Remaining procedure will be same as discussed in case-II.

Note that data across eName and Sal is present on cache, but predicate attribute (Age) is not on cache. Now some one cannot select the data from cache due to absence of predicate attribute; because some one cannot decide which of the data satisfy the selection criteria (Age>35). To solve this problem, another query called amending query (Ren et al., 2003) to retrieve primary attribute from server on user select criteria as below.

aq = *SELECT* e\_ID *FROM* employee *WHERE* Age>35

Then retrieved primary keys will be mapped with keys on cache and data will be presented to user. By this hit ratio is increased.

**Case-IV:** In this case efficient predicate matching to improve hit ratio by using subtraction algorithm is elaborated with example.

Let us consider that user has already posed the following query and result has been stored

Data for above query will be retrieved and stored on cache will be as given in table 6.

*SELECT* eName, Sal *FROM* employee *WHERE* age>30

**E\_ID eName Sal** 

114 Adeel 42000

115 Komal 17000

116 Mahreen 30450

117 Tabinda 28850

118 Yaseen 24450

119 Anees 30000

120 Komal 30000

Note that e\_ID is not required but retrieved. It is due to the requirement of key-contained (Ren et al., 2003) contents. Now let us assume that user has posed the following query.

Here, generation of amending query is discussed. Remaining procedure will be same as

*SELECT* eName, Sal *FROM* employee *WHERE* age>35

Note that data across eName and Sal is present on cache, but predicate attribute (Age) is not on cache. Now some one cannot select the data from cache due to absence of predicate attribute; because some one cannot decide which of the data satisfy the selection criteria (Age>35). To solve this problem, another query called amending query (Ren et al., 2003) to

Then retrieved primary keys will be mapped with keys on cache and data will be presented

aq = *SELECT* e\_ID *FROM* employee *WHERE* Age>35

**Case-IV:** In this case efficient predicate matching to improve hit ratio by using subtraction

retrieve primary attribute from server on user select criteria as below.

in cache.

Table 6. Contents on cache in case-III.

to user. By this hit ratio is increased.

algorithm is elaborated with example.

discussed in case-II.

Let us consider that user has already posed the following query and result has been stored in cache.

*SELECT* eName, Age *FROM* employee *WHERE* Age>30

Data for above query will be retrieved and stored on cache will be as given in table 7.


Table 7. Contents on cache in case-IV.

Note that e\_ID is not required but retrieved. It is due to the requirement of key-contained (Ren et al., 2003) contents. Now let us assume that user has posed the following query.

$$\text{SELECT eName, Age } \textit{FROM} \text{ employee} \\ \text{WHERE eName='Koma'} $$

Now all of the required fields are matched with cached query. Splitter splits the query and sent it to the rejecter. Rejecter checks the list of fields with relation and predicate attribute from schema based indexing semantics. Query will not be rejected due to availability of all member of list at schema. Common and difference set of attributes will be computed and sent to the 1st level query matcher. i.e. CA and DA will be computed. Remainder query (rq1) will be null due to empty set of difference attributes (All required attributes are exist on cache). So, remainder query by 1st level query matcher will be like below.

$$\text{rq1} = \text{Nu} \,\text{ll}$$

Common attributes (Age, eName) will be sent to the Query Generator (QG). Query Generator will generate probe and remainder query on the base of predicate matching. Conditioned attribute (Age) is already retrieved; so there is no need of amending query in this case.

First of semantics of predicate will be computed by semantic extractor as follow.

Semantic Cache System 105

overlapped (partially & fully) queries locally. The major challenges of semantic caching are efficient query processing and cache management. For efficient query processing we have proposed and demonstrated the working of sCacheQP system. We have provided complete working and algorithms of sCacheQP. Case study is given to elaborate the sCacheQP. In future, we have a plan to implement the system for data mining and data warehousing.

Abbas, M.A., Qadir, M.A., Ahmad, M., Ali, T., Sajid, N.A, (2011) "Graph Based Query

Ali, T., Qadir, M.A., (2010) "DL based Subsumption Analysis for Relational Semantic Cache

Ahmad, M., Asghar, A., Qadir, M.A., Ali, T. (2010) "Graph Based Query Trimming Algorithm

Ahmad, M., Qadir, M.A., and Sanaullah, M. (2008b) , "Query Processing over Relational

Ahmad, M., Qadir, M.A., and Sanaullah, M. (2009) "An Efficient Query Matching Algorithm

Ahmed, M.U, Zaheer, R.A, and Qadir, M.A., (2005). "Intelligent cache management for data

Altnel, M., Bornhövd, C., Krishnamurthy, C., Mohan, C., Pirahesh, H., and Reinwald, B.,

Bashir, M.F and Qadir, M.A., (2007). "ProQ – Query Processing Over Semantic Cache For

Bashir, M.F., Zaheer, R.A., Shams, Z.M. and Qadir, M.A., (2007). "SCAM: Semantic Caching

Chakrabarti, K., Porkaew. K., and Mehrotra, S., (2000). "Efficient Query Refinement in

*Conference, INMIC 2008, IEEE, Karachi, Pakistan, December 2008.*

*Emergent Digital EcoSystem*, MEDES10, Bangkok, Thailand, October 2010. Ahmad, M., Qadir, M.A., Razzaque, A., and Sanaullah, M. (2008a), "Efficient Query

*(ICIET) 2010, Karachi Pakistan, 14-16 June 2010* 

Trimming of Conjunctive Queries in Semantic Caching", *IEEE International Conference on Emerging Technologies (ICET 2011)*, Islamabad, Pakistan, September 5-6, 2011. Ali, T., Qadir, M.A., Ahmad, M. (2010) "Translation of relational queries into Description

Logic for semantic cache query processing" *Information and Emerging Technologies* 

Query Processing and Management"10th International Conference on Knowledge Management and Knowledge Technologies, Messe Congress Graz, Austria. 1–3

for Relational Data Semantic Cache", *The International Conference on Management of* 

Processing over Semantic Cache". *Intelligent Systems and Agents, ISA 2008,* indexed by IADIS digital library (www.iadis.net/dl). Held within IADIS Multi Conference on Computer Science and Information Systems (MCCSIS 2008), Amsterdam,

Databases with Semantic Cache: A Survey". *12th IEEE International Multitopic* 

for Relational Data Semantic Cache". *2nd IEEE conference on computer, control and* 

grid"; *In Proceedings of the Australasian Workshop on Grid Computing and E-Research,* 

(2003)., "Cache Tables: Paving the Way for an Adaptive Database Cache", *Proceedings of the 29th VLDB Conference*, VLDB Endowment, Berlin, Germany, pp. 718-729. Bashir, M.F and Qadir, M.A., (2006). "HiSIS: 4–Level Hierarchical Semantic Indexing for

Efficient Content Matching over Semantic Cache". *INMIC, IEEE,* Islamabad,

Data Grid", *Center for Distributed and Semantic Computing,* Mohammad Ali Jinnah

Architecture for Efficient Content Matching over Data Grid". *AWIC,S*pringer

Multimedia Databases", *16th International conference on Data Engineering*, IEEE, 2000.

**7. References** 

September 2010

Netherland. 22-27 July 2008

*communication, IC409, 2009.*

Pakistan, pp. 211-214.

New South Wales, Australia, 2005.

University, Islamabad, Pakistan 2007.

Heidelberg, Berlin, 2007. pp. 41-46.

$$\begin{array}{l} \mathsf{M}\_{\mathrm{C}} = \mathrm{Null} \\ \mathrm{NM}\_{\mathrm{C}} = \mathrm{Age} \\ \mathrm{NM}\_{\mathrm{U}} = \mathrm{e} \mathrm{Name} \end{array}$$

After computation of predicate semantics, predicate for probe and remainder query will be computed by using predicate matching algorithm (actually 112 rules are used here). Probe and remainder query will be on the base of these rules like:

$$\begin{array}{c} \mathsf{CC-QM} \leftarrow \mathit{Null} \\ \mathsf{CNC-QM} \leftarrow \mathit{Null} \end{array}$$

i.e.

C1 =Null NC1 = Null

Due to simple predicate rules defined for complex queries will not be applied. Finally subtraction algorithm will be applied to generate final predicate for probe and remainder queries. As it is computed, that NMc and NMu both are not null. So, fourth case of subtraction algorithm will be applied.

> 4. **Else If** (*NMc* != null) and (*NMu* != null) **then** a. **C**<sup>2</sup> := (*C*1) + (*NMu*) + (*NMc*) b. **N**C2 := ((*C*1) + R (*NMc)+NMu*)V((NC1)+(NMu))

So, predicate for probe and remainder query will be like below.

**C**<sup>2</sup> := (Age>30) and (eName = 'Komal') **N**C2 := (Age<=30) and (eName = 'Komal')

Then, probe query (pq) and second remainder query (rq2) will be generated as below.

pq = SELECT eName, Age FROM employee WHERE (Age>30) and (eName = 'Komal') rq2 = SELECT eName, Age FROM employee WHERE (Age<=30) and (eName = 'Komal')

In the last step result of rq1, pq and rq2 is combined by rebuilder.

#### **6. Conclusion**

Caching proved very helpful to reduce the data access latency for distributed and large scale database systems by storing the data against already executed queries. Main problem in caching is to identify the overlapping of required data with stored data. Page and tuple cache are not able to identify the partial overlapping. Semantic cache is capable to answer the

MC = Null NMC = Age NMU = eName

After computation of predicate semantics, predicate for probe and remainder query will be computed by using predicate matching algorithm (actually 112 rules are used here). Probe

> *CC-QM* ← *Null CNC-QM* ← *Null*

Due to simple predicate rules defined for complex queries will not be applied. Finally subtraction algorithm will be applied to generate final predicate for probe and remainder queries. As it is computed, that NMc and NMu both are not null. So, fourth case of

4. **Else If** (*NMc* != null) and (*NMu* != null) **then**

b. **N**C2 := ((*C*1) + R (*NMc)+NMu*)V((NC1)+(NMu))

C1 =Null NC1 = Null

Then, probe query (pq) and second remainder query (rq2) will be generated as below.

pq = SELECT eName, Age FROM employee WHERE (Age>30) and (eName = 'Komal') rq2 = SELECT eName, Age FROM employee WHERE (Age<=30) and (eName = 'Komal')

**C**<sup>2</sup> := (Age>30) and (eName = 'Komal') **N**C2 := (Age<=30) and (eName = 'Komal')

a. **C**<sup>2</sup> := (*C*1) + (*NMu*) + (*NMc*)

Caching proved very helpful to reduce the data access latency for distributed and large scale database systems by storing the data against already executed queries. Main problem in caching is to identify the overlapping of required data with stored data. Page and tuple cache are not able to identify the partial overlapping. Semantic cache is capable to answer the

and remainder query will be on the base of these rules like:

So, predicate for probe and remainder query will be like below.

In the last step result of rq1, pq and rq2 is combined by rebuilder.

subtraction algorithm will be applied.

**6. Conclusion** 

i.e.

overlapped (partially & fully) queries locally. The major challenges of semantic caching are efficient query processing and cache management. For efficient query processing we have proposed and demonstrated the working of sCacheQP system. We have provided complete working and algorithms of sCacheQP. Case study is given to elaborate the sCacheQP. In future, we have a plan to implement the system for data mining and data warehousing.

## **7. References**


**5** 

*Pakistan* 

**Semantic Interoperability in E-Health** 

*National University of Computer and Emerging Sciences, Islamabad,* 

Saman Iftikhar1, Wajahat Ali Khan1, Farooq Ahmad1 and Kiran Fatima2

One of the challenges faced nowadays by the healthcare industry is semantic interoperability. It is the ability of a healthcare system to share information and have that information properly interpreted by the receiving system in the same sense as intended by the transmitting system. Semantic Web (aka Web 3.0) provides the enabling technologies to achieve semantic interoperability. Web Services, as a catalyst in this process, provide seamless communication of information between healthcare systems thus providing better

Semantic technologies are emerging and several applications ranging from business process management to information security have demonstrated encouraging prospects of its benefits. Role of semantics is also very vital for achieving interoperability in sharing of health records. The aim of this chapter is to establish research and development in the domain of Health Level 7 (HL7) as an application to provide e-health services for the diverse communities. Through this research process, we intend to develop HL7 interface software for healthcare information systems that will provide semantic interoperability between the communicating medical systems. The objective is to facilitate e-health services that are interoperable among a number of domains in this field such as laboratory, patient administration and pharmacy. After its development and testing in the end-user environment, this software solution will be made publicly available under an open-source license. Due to its cutting-edge nature, this software solution has the potential of establishing an international repute for Pakistan in the highly profitable and potent healthcare industry. Since healthcare is a sensitive and critical area as it involves life of human beings, this project will be conducted in a manner to ensure that the resulting

This mostly involves research and implementation challenges. Some initiatives are already underway such as Health Services Specification Project (HSSP). It is a joint-venture of HL7 and Object Management Group (OMG), providing standardized service interface specifications. Following the traces of HSSP, our proposal is aimed to design and implement concrete SOA model. The ultimate goal is to define the HL7 Web services as Semantic Web

access to patient information and improved healthcare.

software is secure, reliable and maintainable.

**1. Introduction** 

**for Improved Healthcare** 

*2Department of Computer Sciences* 

*1School of Electrical Engineering and Computer Sciences National University of Sciences and Technology, Islamabad,* 


## **Semantic Interoperability in E-Health for Improved Healthcare**

Saman Iftikhar1, Wajahat Ali Khan1, Farooq Ahmad1 and Kiran Fatima2 *1School of Electrical Engineering and Computer Sciences National University of Sciences and Technology, Islamabad, 2Department of Computer Sciences National University of Computer and Emerging Sciences, Islamabad, Pakistan* 

## **1. Introduction**

106 Semantics in Action – Applications and Scenarios

Cai, J., Jia, Y., Yang, S., and Zou, P., (2005) "A Method of Aggregate Query Matching in

Chen, C.M. and Roussopoulos, N., (1994). "The Implementation and Performance

Matching," Proc. Int'l Conf. Extending Database Technology, pp. 323-336. Chen, L., Rundesteiner, E.A., Wang, S., (2002). "XCache -A Semantic Caching System for

Dar, S., Franklin, M.J., Jonnson, B.T., (1996). "Semantic Data Caching and Replacement,"

Deshpande, P.M. Ramasamy, K., and Shukla, A., (1998). "Caching Multidimensional Queries Using Chunks", *ICMD,* ACM, New York, USA, 1998, pp. 259-270. Godfrey, P. and Gryz, J., (1997). "Semantic Query Caching for Heterogeneous Databases,"

Guo, S., Sun, W., and Weiss, M.A., (1996). "Solving Satisfiability and Implication Problems

Jonsson, B. T., Arinbjarnar, M., Thorsson, B., Franklin, M., and Srivastava, D., (2006).

Kang, S.W., Kim, J., Im, S., Jung, H., and Hwang, C.S., (2006). "Cache Strategies for

Keller, A.M. and Basu, J., (1996). "A Predicate-Based Caching Scheme for Client-Server

Lee, D. and Chu, W.W., (1999). "Semantic Caching via Query Matching for Web Sources,"

Luo, Q., Naughton, J. F., Krishnamurthy, R., Cao, P., and Li, Y., (2000). "Active Query

World Wide Web and Databases Springer, London, UK, pp. 92-104. Ren, Q., Dunham, M.H., and Kumar, V., (2003). "Semantic Caching and Query Processing". Knowledge and Data Engineering, IEEE Computer Society, 2003, pp. 192-210. Roussopoulos, N. An incremental Access Method for View Cache: Concept, Algorithms, and Cost Analysis," ACM Trans.Database Systems, vol. 16, no. 3, 1991, 535-563. Sanaullah, M., Qadir, M.A., and Ahmad, M., (2008) "SCAD-XML: Semantic Cache Architecture

*Management of Data,* ACM Press, New York, pp. 618-618.

in Database Systems," *Database Systems*, ACM, pp. 270-293.

*Web-Age Information Management Workshops,* IEEE, 2006.

Proc. CIKM, ACM, Kansas City, USA, pp. 77-85.

System", ICSCN, IEEE, Chennai, India, pp. 233-237.

ICACT, IEEE, Gangwon-Do, Korea, pp. 1952-1957.

Engineering, IEEE, Piscataway, USA, pp. 1168-1175.

*Proceeding of VLDB Conference*, VLDB, pp. 330-341.

Heidelberg 2005, pp. 435-442.

Greece, pp.61-66.

ACM, New York, USA pp. 302–331.

Heidelberg, Berlin, , pp. 35-47.

Semantic Cache for Massive Database Applications". *Springer-Verlag,* Berlin

Evaluation of the ADMS Query Optimizer: Integrating Query Result Caching and

XML Queries". *In Proceedings of the 2002 ACM SIGMOD international Conference on* 

*In Proc. 4th KRDB Workshop "Intelligent Access to Heterogeneous Information"*, Athens,

"Performance and overhead of semantic cache management", *Internet Technology*,

Semantic Prefetching Data", *Proceedings of the Seventh International Conference on* 

Database Architectures", *International Journal* on *Very Large Database,* Springer,

Caching for Database Web Servers", Third International Workshop WebDB on The

for XML Data Files using XPath with Cases and Rules ". 12th IEEE International

Mapping Strategy for Improving Retrieval Effectiveness in Semantic Cache

"Dynamic Rule Set Mapping Strategy for the Design of Effective Semantic Cache",

Multitopic Conference, INMIC 2008, IEEE, Karachi, Pakistan, December 2008. Sumalatha, M.R., Vaidehi, V., Kannen, A., Rajasekar, M., Karthigaiselven, M., (2007). "Hash

Sumalatha, M.R., Vaidehi, V., Kannen, A., Rajasekar, M., Karthigaiselven, M., (2007).

Sumalatha, M.R., Vaidehi, V., Kannen, A., Rajasekar, M., Karthigaiselven, M., (2007). "Xml

Sun, X., Kamel, N.N., and Ni, L.M., (1989). "Processing Implication on Queries", Software

Query Processing – Semantic Cache System". IJCSNS, pp. 164-169.

One of the challenges faced nowadays by the healthcare industry is semantic interoperability. It is the ability of a healthcare system to share information and have that information properly interpreted by the receiving system in the same sense as intended by the transmitting system. Semantic Web (aka Web 3.0) provides the enabling technologies to achieve semantic interoperability. Web Services, as a catalyst in this process, provide seamless communication of information between healthcare systems thus providing better access to patient information and improved healthcare.

Semantic technologies are emerging and several applications ranging from business process management to information security have demonstrated encouraging prospects of its benefits. Role of semantics is also very vital for achieving interoperability in sharing of health records. The aim of this chapter is to establish research and development in the domain of Health Level 7 (HL7) as an application to provide e-health services for the diverse communities. Through this research process, we intend to develop HL7 interface software for healthcare information systems that will provide semantic interoperability between the communicating medical systems. The objective is to facilitate e-health services that are interoperable among a number of domains in this field such as laboratory, patient administration and pharmacy. After its development and testing in the end-user environment, this software solution will be made publicly available under an open-source license. Due to its cutting-edge nature, this software solution has the potential of establishing an international repute for Pakistan in the highly profitable and potent healthcare industry. Since healthcare is a sensitive and critical area as it involves life of human beings, this project will be conducted in a manner to ensure that the resulting software is secure, reliable and maintainable.

This mostly involves research and implementation challenges. Some initiatives are already underway such as Health Services Specification Project (HSSP). It is a joint-venture of HL7 and Object Management Group (OMG), providing standardized service interface specifications. Following the traces of HSSP, our proposal is aimed to design and implement concrete SOA model. The ultimate goal is to define the HL7 Web services as Semantic Web

Semantic Interoperability in E-Health for Improved Healthcare 109

Architectures (SOA) applications. SOA is a solution to handle complex business processes

Healthcare is a many-to-many business so to cater complexities and bring interoperability among heterogeneous systems; a business process model is required. SOA for our project requires certain standardized specifications to follow in order to claim the compliance with standards. These specifications are formulated mainly by coordination of HL7 and OMG group, under the name of Healthcare Services Specification Project (HSSP). HSSP gives Service Functional Models (SFM) which specifies interface specifications and not the implementation specifications. The document "Service Oriented Architecture and HL7 V3 Methodology" by Special Interest Group (SOA SIG), gives approach for implementing healthcare services in Healthcare domain. Another important document in this series is "The Practical Guide for SOA in Health Care" by HSSP gives concrete guidelines along with mega SOA architecture for Healthcare. These documents are providing main guidelines in

Although SOA framework can be used for designing interoperable systems yet it is not a proper solution for providing true interoperability, i.e. the semantic interoperability. Semantic interoperability is the way to intelligently interpret the transferred knowledge among communicating machines and provide accurate desired results. HL7 V3 provides specifications for different domains like patient administration, specimen, laboratory, observation etc. Every domain supports data and processes particular to that domain in addition to some common elements that are shared among multiple domains. The main focus of this thesis is to bring semantics in the interactions included in laboratory domain. This work refined the meaning of semantic interoperability by representing the interactions and other artifacts with ontologies rather only limited to the vocabulary representation

In HL7 the semantic interoperability can be seen from two perspectives; data and process. The potentials of semantic data interoperability remain incomplete without semantic process interoperability. Achieving interoperable data would be less effective if there is no semantics in the communication components which can only be achieved when the process is interoperable. Semantic data interoperability means understanding of the data communicated between sender and receiver in such a way that the receiver easily interprets the sender intension of sending the data and properly responds. On the other hand semantic process interoperability is the type of semantic interoperability, which helps in the decision process of the participating parties in communication of HL7 messages on the basis of data contents intended to be exchanged for automation. For bringing semantic interoperability in

HL7 V3 claims to provide semantic interoperability but it only focuses on the semantic data interoperability and semantic process interoperability is still a grey area. HL7 V3 provides data interoperability in the form of terminologies by using vocabularies like SNOMED CT (SNOMED Clinical Terms, 2009), LOINC (LOINC) and HL7's own vocabulary. But semantic interoperability cannot be catered by only taking in to account specified terminologies. To achieve semantic interoperability there is a need of a framework that can support the required constructs for semantic interoperability. Web Service Modeling Framework (WSMF) provides Web Service Modeling Ontology (WSMO) which contains the entities like

the HL7 processes, semantic web services are followed for the communication.

and to achieve interoperability.

our work for getting SOA workflows.

supported by HL7 V3 (Beeler et al., 1999).

ontologies, mediators, web services and goals.

services. Web Services Modeling Framework will provide the platform for automatic web service discovery, composition, and invocation that makes the technology scalable. This purpose of this chapter is to bring improvement in electronic health records by integrating it with semantic web services and semantic registries that will eventually lead to healthcare interoperable systems. One important part is the integration of Service Oriented Architecture (SOA) with HL7. HL7 Pakistan NUST, more specifically, has designed a prototype for the laboratory domain and it has been successfully implemented at the CITI Lab (a local testing laboratory). This successful initial prototype has provided us the baseline to enhance it by embedding semantics in the system in order to enable semantic interoperability.

## **1.1 Background and rationale**

Healthcare systems are critical and demand high accuracy, prompt availability and interoperability. The right use of information and communication system can play vital role in achieving the said requirements; but unfortunately healthcare systems are used mostly as a replacement to manual patient logging. The critical need is to encourage healthcare systems to be more efficient and provide more workable solutions like other industries that have benefited from it e.g. banking, traffic systems and so on. When a patient moves from hospital to hospital, he needs to take all the records and reports with him which is difficult to manage especially in emergency situations. Manual healthcare data system is not only prone to error and loss but also it is not feasible to manage massive data and access any particular record from it. Using healthcare data electronically results in cost-effective, easily accessible, accurate and manageable data processing solutions. In Pakistan, very few healthcare organizations so far have become capable of storing healthcare data electronically but it comes without the ability to share the information. This is mostly due to lack of awareness and implementation of information exchange standards.

Standardization provides us an effective way of communication to achieve the goal of interoperability. HL7 is one of the healthcare standards that allow communication and integration of healthcare systems and allow sharing of data around the globe. The important requirement is to capture relevant information and then make it widely available for others. Therefore, the need is to have a standard that can provide best services in terms of efficiency and reliability. HL7, as it evolves, provides us with a technical business model to fulfill this vision of a diverse, integrated health information system.

The two most important issues that the healthcare industry is facing are integration and interoperability of systems. Countries are not willing to invest in healthcare industry until and unless the healthcare systems to be adopted by them provide interoperability. HL7 is a messaging standard that is used for the exchange of medical information between different communicating parties or devices. The most commonly used versions of HL7 are HL7 V2.x and HL7 V3. HL7 V 2.x is mainly focused on the transfer of message from sender to the receiver rather on interoperability. HL7 V3 focused on the shortcomings of HL7 V2.x and overcome those by targeting semantic interoperability (Neotool). HL7 V3 is based on the standard model called Reference Information Model (RIM). Another potential capability is to make HL7 V3 based systems SOA complaint.

The innovation and the standardization of web services have set the concept of web services as the basic building blocks of information technology systems for Service Oriented

services. Web Services Modeling Framework will provide the platform for automatic web service discovery, composition, and invocation that makes the technology scalable. This purpose of this chapter is to bring improvement in electronic health records by integrating it with semantic web services and semantic registries that will eventually lead to healthcare interoperable systems. One important part is the integration of Service Oriented Architecture (SOA) with HL7. HL7 Pakistan NUST, more specifically, has designed a prototype for the laboratory domain and it has been successfully implemented at the CITI Lab (a local testing laboratory). This successful initial prototype has provided us the baseline to enhance it by embedding semantics in the system in order to enable semantic

Healthcare systems are critical and demand high accuracy, prompt availability and interoperability. The right use of information and communication system can play vital role in achieving the said requirements; but unfortunately healthcare systems are used mostly as a replacement to manual patient logging. The critical need is to encourage healthcare systems to be more efficient and provide more workable solutions like other industries that have benefited from it e.g. banking, traffic systems and so on. When a patient moves from hospital to hospital, he needs to take all the records and reports with him which is difficult to manage especially in emergency situations. Manual healthcare data system is not only prone to error and loss but also it is not feasible to manage massive data and access any particular record from it. Using healthcare data electronically results in cost-effective, easily accessible, accurate and manageable data processing solutions. In Pakistan, very few healthcare organizations so far have become capable of storing healthcare data electronically but it comes without the ability to share the information. This is mostly due to lack of

Standardization provides us an effective way of communication to achieve the goal of interoperability. HL7 is one of the healthcare standards that allow communication and integration of healthcare systems and allow sharing of data around the globe. The important requirement is to capture relevant information and then make it widely available for others. Therefore, the need is to have a standard that can provide best services in terms of efficiency and reliability. HL7, as it evolves, provides us with a technical business model to fulfill this

The two most important issues that the healthcare industry is facing are integration and interoperability of systems. Countries are not willing to invest in healthcare industry until and unless the healthcare systems to be adopted by them provide interoperability. HL7 is a messaging standard that is used for the exchange of medical information between different communicating parties or devices. The most commonly used versions of HL7 are HL7 V2.x and HL7 V3. HL7 V 2.x is mainly focused on the transfer of message from sender to the receiver rather on interoperability. HL7 V3 focused on the shortcomings of HL7 V2.x and overcome those by targeting semantic interoperability (Neotool). HL7 V3 is based on the standard model called Reference Information Model (RIM). Another potential capability is

The innovation and the standardization of web services have set the concept of web services as the basic building blocks of information technology systems for Service Oriented

awareness and implementation of information exchange standards.

vision of a diverse, integrated health information system.

to make HL7 V3 based systems SOA complaint.

interoperability.

**1.1 Background and rationale** 

Architectures (SOA) applications. SOA is a solution to handle complex business processes and to achieve interoperability.

Healthcare is a many-to-many business so to cater complexities and bring interoperability among heterogeneous systems; a business process model is required. SOA for our project requires certain standardized specifications to follow in order to claim the compliance with standards. These specifications are formulated mainly by coordination of HL7 and OMG group, under the name of Healthcare Services Specification Project (HSSP). HSSP gives Service Functional Models (SFM) which specifies interface specifications and not the implementation specifications. The document "Service Oriented Architecture and HL7 V3 Methodology" by Special Interest Group (SOA SIG), gives approach for implementing healthcare services in Healthcare domain. Another important document in this series is "The Practical Guide for SOA in Health Care" by HSSP gives concrete guidelines along with mega SOA architecture for Healthcare. These documents are providing main guidelines in our work for getting SOA workflows.

Although SOA framework can be used for designing interoperable systems yet it is not a proper solution for providing true interoperability, i.e. the semantic interoperability. Semantic interoperability is the way to intelligently interpret the transferred knowledge among communicating machines and provide accurate desired results. HL7 V3 provides specifications for different domains like patient administration, specimen, laboratory, observation etc. Every domain supports data and processes particular to that domain in addition to some common elements that are shared among multiple domains. The main focus of this thesis is to bring semantics in the interactions included in laboratory domain. This work refined the meaning of semantic interoperability by representing the interactions and other artifacts with ontologies rather only limited to the vocabulary representation supported by HL7 V3 (Beeler et al., 1999).

In HL7 the semantic interoperability can be seen from two perspectives; data and process. The potentials of semantic data interoperability remain incomplete without semantic process interoperability. Achieving interoperable data would be less effective if there is no semantics in the communication components which can only be achieved when the process is interoperable. Semantic data interoperability means understanding of the data communicated between sender and receiver in such a way that the receiver easily interprets the sender intension of sending the data and properly responds. On the other hand semantic process interoperability is the type of semantic interoperability, which helps in the decision process of the participating parties in communication of HL7 messages on the basis of data contents intended to be exchanged for automation. For bringing semantic interoperability in the HL7 processes, semantic web services are followed for the communication.

HL7 V3 claims to provide semantic interoperability but it only focuses on the semantic data interoperability and semantic process interoperability is still a grey area. HL7 V3 provides data interoperability in the form of terminologies by using vocabularies like SNOMED CT (SNOMED Clinical Terms, 2009), LOINC (LOINC) and HL7's own vocabulary. But semantic interoperability cannot be catered by only taking in to account specified terminologies. To achieve semantic interoperability there is a need of a framework that can support the required constructs for semantic interoperability. Web Service Modeling Framework (WSMF) provides Web Service Modeling Ontology (WSMO) which contains the entities like ontologies, mediators, web services and goals.

Semantic Interoperability in E-Health for Improved Healthcare 111

The HLH studio architecture as shown in Figure 1 will be used initially to create and parse HL7 V3 message using Java SIG API. The Java SIG API supports generation and parsing of any type of V 3 messages while making corresponding Hierarchical Message Description's (HMD) available. During creation of HL7 message, the HL7 builder tool consumes inmemory Refined Message Information Model (RMIM) objects and taking meta-data from HMDs to create valid serialized XML based message specified by HMD, while in message parsing, the parser tool consumes XML message, validates it against HMD and creates in

The message generation and parsing is only limited to the Laboratory and Patient Administration domains, their specifications are provided in the HL7 Normative, 2009. The message generation and parsing is the first step towards interoperability. This standard

HL7 is a standard used for information exchange among healthcare systems. SOA, on the other hand, is an architecture that enables business agility through the use of common services. HL7 stakeholders realized that by bringing these two realms at one place will generate revolutionary benefits for healthcare. SOA architecture mainly causes interoperable and easy accessible communication which HL7 V3 conventional Messaging Infrastructure

SOA framework, unlike MI, encompasses service creation, hosting and communication capabilities at one place. Our project needs the basic three elements of SOA; i.e. Producer, Consumer and registry to be realized for rejuvenating our healthcare environment. The producers will be the healthcare organizations, and the consumers are the patients, doctors and other healthcare community. As SOA provides communication according to business case, it supports the academic and business community to get enormous potentials for research and development in healthcare sector. The underlying infrastructure is based on

Based upon the lessons learned from HSSP specifications, for SOA framework there are some steps that are to be followed. As we are analyzing laboratory and patient administration domain for our case study to be implemented, the laboratory domain artifacts would be analyzed initially. We have to identify services in the laboratory domain by investigating application roles and their interactions. This will lead to decision on operations by studying storyboards, constraints, HL7 information model (DMIM) and trigger events. The description of interface specifications by studying HSSP services' specifications is carried out. The services are then implemented by Web service basic profile and implement orchestration and choreography using business process model workflows.

Once the HL7 services have been exposed as Web Services, it will be available for everyone to use over the web. The advancement in the Semantic Web has now shifted the simple Web services to the Semantic Web Services. The Semantic Web (SW) approach is to develop languages and mechanisms for expressing information in machine understandable form. The web services that are identified in the SOA framework are to be upgraded to semantic web services. In order to achieve this goal Semantic Web Service (SWS) Architecture review including WSMO, WSML and WSMX should be performed. The identification of process

message can then be communicated between communicating parties.

web services, for which HSSP is providing specifications.

The last step will be registering the services in a proper registry.

**1.3 Methodology** 

memory RMIM object graph.

(MI) cannot provide.

One technique for achieving semantic process interoperability is to use simple web services. Web services provide a standard means of interoperable communication between heterogeneous software applications. The complexity is increased for web services when semantic and syntactic heterogeneities are brought in to consideration for the transfer of messages between systems. Therefore, there is a need of using semantic web services for achieving semantic process interoperability. Semantic web services can be used for enhancing the web services capabilities in understanding semantics such that it can be more easily machine process able. This will result in better machine understanding of the web services and the communication would be more effective. Semantic web services should have proper precondition, post-condition, effects and assumptions. There are different approaches used for realizing semantic web services but WSMO is the most preferable as it is the most effective and complete approach amongst all.

There is a need to explore such sophisticated SOA technologies that make the discovery of services for requested users appropriate. In service oriented computing services are used to develop fast, economical, interoperable, evolvable, and extremely distributed applications. Services are self-governing, platform-independent entities that can be described, published, discovered, and loosely coupled. Semantic registries are required for the handling and accessing meaningful information over the semantic web. In present, the services are described, registered and accessed without semantics which is not efficient if the services are to be discovered precisely. In semantic registries the discovery of services is all about the finding of desirable services semantically which have knowledgeable significant properties and relationships. Therefore, the services have to be expressed semantically in semantic registries, so that the semantically described services can be machine comprehensible and precisely used by applications for interoperability of processes through semantic registries and results in semantic SOA.

The issue in using such standards like HL7 V3 is to provide tools and encourage its usage through making them integrated with the existing healthcare systems. Also these standards can be more utilized by following frameworks such as SOA and WSMF. These frameworks can help HL7 standard to achieve true interoperability. In this project our emphasis is to make such open source tools that will help the healthcare industry in achieving such targets.

### **1.2 Scope and objectives**


### **1.3 Methodology**

110 Semantics in Action – Applications and Scenarios

One technique for achieving semantic process interoperability is to use simple web services. Web services provide a standard means of interoperable communication between heterogeneous software applications. The complexity is increased for web services when semantic and syntactic heterogeneities are brought in to consideration for the transfer of messages between systems. Therefore, there is a need of using semantic web services for achieving semantic process interoperability. Semantic web services can be used for enhancing the web services capabilities in understanding semantics such that it can be more easily machine process able. This will result in better machine understanding of the web services and the communication would be more effective. Semantic web services should have proper precondition, post-condition, effects and assumptions. There are different approaches used for realizing semantic web services but WSMO is the most preferable as it

There is a need to explore such sophisticated SOA technologies that make the discovery of services for requested users appropriate. In service oriented computing services are used to develop fast, economical, interoperable, evolvable, and extremely distributed applications. Services are self-governing, platform-independent entities that can be described, published, discovered, and loosely coupled. Semantic registries are required for the handling and accessing meaningful information over the semantic web. In present, the services are described, registered and accessed without semantics which is not efficient if the services are to be discovered precisely. In semantic registries the discovery of services is all about the finding of desirable services semantically which have knowledgeable significant properties and relationships. Therefore, the services have to be expressed semantically in semantic registries, so that the semantically described services can be machine comprehensible and precisely used by applications for interoperability of processes through semantic registries

The issue in using such standards like HL7 V3 is to provide tools and encourage its usage through making them integrated with the existing healthcare systems. Also these standards can be more utilized by following frameworks such as SOA and WSMF. These frameworks can help HL7 standard to achieve true interoperability. In this project our emphasis is to make such open source tools that will help the healthcare industry in achieving such targets.

To develop standardized services that should be reusable, cost effective and self-

To create a hybrid platform by incorporating HL7 V3 standard and Service-Oriented

 To model complex healthcare processes in well-defined business language and to capture real life business scenarios, rather than technology-specific terminologies and

 To contribute the developed platform to the open-source community so other healthcare organizations and hospitals, within and outside the, country can reuse and

 To train a reasonable number of professionals and researchers as HL7 based IT researchers, developers as well as users of the HL7 application in the medical related

customize this solution to their specific requirements with minimum efforts.

maintainable; setting the stage for interoperability in healthcare services.

is the most effective and complete approach amongst all.

and results in semantic SOA.

**1.2 Scope and objectives** 

Architecture.

grammar.

discipline.

The HLH studio architecture as shown in Figure 1 will be used initially to create and parse HL7 V3 message using Java SIG API. The Java SIG API supports generation and parsing of any type of V 3 messages while making corresponding Hierarchical Message Description's (HMD) available. During creation of HL7 message, the HL7 builder tool consumes inmemory Refined Message Information Model (RMIM) objects and taking meta-data from HMDs to create valid serialized XML based message specified by HMD, while in message parsing, the parser tool consumes XML message, validates it against HMD and creates in memory RMIM object graph.

The message generation and parsing is only limited to the Laboratory and Patient Administration domains, their specifications are provided in the HL7 Normative, 2009. The message generation and parsing is the first step towards interoperability. This standard message can then be communicated between communicating parties.

HL7 is a standard used for information exchange among healthcare systems. SOA, on the other hand, is an architecture that enables business agility through the use of common services. HL7 stakeholders realized that by bringing these two realms at one place will generate revolutionary benefits for healthcare. SOA architecture mainly causes interoperable and easy accessible communication which HL7 V3 conventional Messaging Infrastructure (MI) cannot provide.

SOA framework, unlike MI, encompasses service creation, hosting and communication capabilities at one place. Our project needs the basic three elements of SOA; i.e. Producer, Consumer and registry to be realized for rejuvenating our healthcare environment. The producers will be the healthcare organizations, and the consumers are the patients, doctors and other healthcare community. As SOA provides communication according to business case, it supports the academic and business community to get enormous potentials for research and development in healthcare sector. The underlying infrastructure is based on web services, for which HSSP is providing specifications.

Based upon the lessons learned from HSSP specifications, for SOA framework there are some steps that are to be followed. As we are analyzing laboratory and patient administration domain for our case study to be implemented, the laboratory domain artifacts would be analyzed initially. We have to identify services in the laboratory domain by investigating application roles and their interactions. This will lead to decision on operations by studying storyboards, constraints, HL7 information model (DMIM) and trigger events. The description of interface specifications by studying HSSP services' specifications is carried out. The services are then implemented by Web service basic profile and implement orchestration and choreography using business process model workflows. The last step will be registering the services in a proper registry.

Once the HL7 services have been exposed as Web Services, it will be available for everyone to use over the web. The advancement in the Semantic Web has now shifted the simple Web services to the Semantic Web Services. The Semantic Web (SW) approach is to develop languages and mechanisms for expressing information in machine understandable form. The web services that are identified in the SOA framework are to be upgraded to semantic web services. In order to achieve this goal Semantic Web Service (SWS) Architecture review including WSMO, WSML and WSMX should be performed. The identification of process

Semantic Interoperability in E-Health for Improved Healthcare 113

would be helpful for medical research, medical education and diagnosing and curing

Figure 2 shows the generic architecture of how the SOA, semantic web services using WSMF and semantic registries would work together. The discovery would be of the services that are without semantics and semantics based discovery. The discovery without semantics would be through UDDI registry of the web services that are created in SOA framework. The semantic web service discovery would take place by Web Service Execution Environment (WSMX) and the bridge between semantic web services and simple web

**2. Semantic Electronic Medical Record (SEMR) system as SaaS service** 

provide more workable solutions that have benefited from ICT (HIMSS, 2011).

The advancement in Information and Communication Technology (ICT) is playing increasing role in healthcare and has managed to improve the efficiency of health services to common people. Health informatics plays a vital role in the integration of ICT in healthcare domain. The critical need is to encourage healthcare systems to be more efficient and

Current syntax based healthcare data systems are not only prone to error and loss but also it is not feasible to manage massive data and access any particular record from it. Therefore healthcare organizations are facing difficulties in managing the large amount of information as well as technological infrastructure. Information retrieval and analysis has turned into a very important challenge for healthcare domain. These challenges can effectively be handled with the help of semantics and cloud computing. Managing healthcare data with semantics results in cost-effective, easily accessible, accurate and manageable data processing solutions. At present EMR systems are designed for hospital operations within the premises,

Fig. 2. Generic Architecture of SOA and semantic web services.

services is provided by a mechanism called grounding in WSMO.

several diseases.

**model** 

Fig. 1. Architecture for HL7 Studio [14].

flows in HL7 is important for achieving the goal of semantic process interoperability. WSMO entities (ontologies, services, goals and mediators) modeling are the next step to be performed. For HL7 processes we would require to model Interaction ontology and Message Ontology. The interaction ontology would contain all the process artifacts (application roles, trigger events, message types and interactions) while the Message Ontology would contain information related to HL7 V3 message like transmission wrapper, control act wrapper and message payload. The WSMO entities should be modeled using WSMT tool for the semantics to completely take effect. The Adapter component implementation is also an important step as conversion from XML to WSML and WSML to XML is required for overcoming heterogeneity problem and bringing interoperability. To completely utilize the WSMO entities an execution environment WSMX should be implemented. Semantic web services will bring automatic service discovery which will make the timely information transfer of patient resulting in quick access to patient care.

The semantic services are required to be stored, published and retrieved in a repository. The semantic registry would be required for registration/publication of patient and lab domain services semantically, so that the services can be accessed for medical research, decision support systems. Analysis of HL7 standardized referenced information models will be done for semantic information management. Identification of semantic discovery and semantic matchmaking algorithms based on inference and reasoning will be done for best retrieval of requested information services. Analysis of different data exchange mechanisms will be done to exchange medical information across the interlinked semantic registries. Analysis of semantic SOA techniques to make our semantic registries service orientated to ensure semantic interoperability, flexibility and extensibility across heterogeneous environments. Analysis of different electronic health records for its feasibility and integration with semantic SOA semantic registries using HL7 V3. The semantic services related to patient and lab domain will be stored, published and retrieved from the semantic registry that

flows in HL7 is important for achieving the goal of semantic process interoperability. WSMO entities (ontologies, services, goals and mediators) modeling are the next step to be performed. For HL7 processes we would require to model Interaction ontology and Message Ontology. The interaction ontology would contain all the process artifacts (application roles, trigger events, message types and interactions) while the Message Ontology would contain information related to HL7 V3 message like transmission wrapper, control act wrapper and message payload. The WSMO entities should be modeled using WSMT tool for the semantics to completely take effect. The Adapter component implementation is also an important step as conversion from XML to WSML and WSML to XML is required for overcoming heterogeneity problem and bringing interoperability. To completely utilize the WSMO entities an execution environment WSMX should be implemented. Semantic web services will bring automatic service discovery which will make the timely information transfer of patient resulting in quick

The semantic services are required to be stored, published and retrieved in a repository. The semantic registry would be required for registration/publication of patient and lab domain services semantically, so that the services can be accessed for medical research, decision support systems. Analysis of HL7 standardized referenced information models will be done for semantic information management. Identification of semantic discovery and semantic matchmaking algorithms based on inference and reasoning will be done for best retrieval of requested information services. Analysis of different data exchange mechanisms will be done to exchange medical information across the interlinked semantic registries. Analysis of semantic SOA techniques to make our semantic registries service orientated to ensure semantic interoperability, flexibility and extensibility across heterogeneous environments. Analysis of different electronic health records for its feasibility and integration with semantic SOA semantic registries using HL7 V3. The semantic services related to patient and lab domain will be stored, published and retrieved from the semantic registry that

Fig. 1. Architecture for HL7 Studio [14].

access to patient care.

would be helpful for medical research, medical education and diagnosing and curing several diseases.

Fig. 2. Generic Architecture of SOA and semantic web services.

Figure 2 shows the generic architecture of how the SOA, semantic web services using WSMF and semantic registries would work together. The discovery would be of the services that are without semantics and semantics based discovery. The discovery without semantics would be through UDDI registry of the web services that are created in SOA framework. The semantic web service discovery would take place by Web Service Execution Environment (WSMX) and the bridge between semantic web services and simple web services is provided by a mechanism called grounding in WSMO.

## **2. Semantic Electronic Medical Record (SEMR) system as SaaS service model**

The advancement in Information and Communication Technology (ICT) is playing increasing role in healthcare and has managed to improve the efficiency of health services to common people. Health informatics plays a vital role in the integration of ICT in healthcare domain. The critical need is to encourage healthcare systems to be more efficient and provide more workable solutions that have benefited from ICT (HIMSS, 2011).

Current syntax based healthcare data systems are not only prone to error and loss but also it is not feasible to manage massive data and access any particular record from it. Therefore healthcare organizations are facing difficulties in managing the large amount of information as well as technological infrastructure. Information retrieval and analysis has turned into a very important challenge for healthcare domain. These challenges can effectively be handled with the help of semantics and cloud computing. Managing healthcare data with semantics results in cost-effective, easily accessible, accurate and manageable data processing solutions. At present EMR systems are designed for hospital operations within the premises,

Semantic Interoperability in E-Health for Improved Healthcare 115

multilingual support and prescription. In short the system provides all basic functionalities

OpenEMR Virtual Appliance (OpenEMR Virtual Appliance, 2011) is a comprehensive open source Medical Practice Management Software Appliance, which provides office scheduling, electronic medical records, prescriptions, insurance billing, accounting and access controls. This appliance has many possible applications, such as a fully functional demo, a testing/developing platform, and as the starting point in real world clinic applications. It can be run on any operating system that supports the VMware Player.

OpenMRS (OpenMRS, 2011) is a full open source healthcare system and has the ability to configure the system to new requirements without programming and to interoperate with other systems whether open or closed. Both of these open source systems are refined under Health Insurance Portability and Accountability Act (HIPAA, 2011) and CCHIT (CCHIT,

SequelMed EMR (SequelMed EMR, 2011) is a secure, patient centric, medical record, integrated with Sequel Systems' medical billing software (SequelMed EPM). The system can automate clinical documentation and have Decision Support Tools and Alerts, Integrated

ClearHealth (Clear-health, 2011) is open source software and include five major areas of healthcare practice operations including scheduling, billing, EMR, HIPAA Security and

XChart (XChart, 2011) is a paper based project by the Open Healthcare Group that promotes

SmartCare (SmartCare,2011) is software that develops EMR programs and particularly used

Zimbra (Zimbra, 2011) gives e-mail solution for government offices, education institutes and other business environments. Medical professionals can also benefit from its fast backup and recovery of mailboxes, anti-spam and anti-virus protection, this software has also support for BlackBerry and other mobile devices, and their flexible applications. All of these systems

These systems provide basic clinical practice that is helpful for patients' medical record but do not support interoperability among different workflow components such as laboratory, medical reports, patient administration, pharmacy, insurance, billing, and prescription

In order to avoid the burden of management of technological infrastructure, SaaS based solution should be used to develop the system on top of cloud infrastructure. To achieve full potential of machine process able SaaS service model based EMR, semantics need to be added. Semantics bring the benefits of unambiguous definition of service functionality and the external interfaces of services reduce human effort in integrating services to SOA, improve dynamism and stability to Web services. Our proposed system will ensure timely delivery of health care information and will ensure its confidentiality. The proposed

Patient Education Protocols, wireless and internet access to medical records.

that any hospital EMR system can provide but it is restricted to a hospital.

2011; CCHIT EMR, 2011) certified.

are open source and primary care systems.

among medical repositories, hospitals, pharmacies and clinics.

accounts receivables."

EMR, based in XML.

**2.2 Methodology** 

in Zambia.

but now have to be modified to support primary care settings of patients, mostly outside of the walls of the hospital. The traditional primary care teams also have to redesign the workflow as they add new care coordination staff and EMR technology to achieve the desired goal of improving the clinical outcome at reduced costs (Ginsburg, 2006).

Interoperability is the ability of a healthcare system to share information and have that information properly interpreted by the receiving system in the same sense as intended by the transmitting system. Standardization provides us an effective way of communication to achieve the goal of semantic interoperability. HL7 is one of the healthcare standards that allow communication of healthcare systems and allow sharing of data around the globe. The important requirement is to capture relevant information and then make it widely available for others. Therefore, the need is to have a system that can provide best services in terms of meaningful data sharing and discovery. HL7, as it evolves, helps us with a technical business model to fulfill the vision of standard based information exchange in diverse, integrated health information systems (HL7, 2011; HLH, 2011; HL7, 2009).

In the era of Semantic Web and cloud computing, there is a need and demand of such an EMR system where timely, accurate and rapid availability of healthcare services can be possible that can manage patient's health data and helps physicians and patients. EMR is basically a part of local standalone Health Information System (HIS) that is an organization's legal proprietary, it includes hospitals as well as doctors, clinicians and physicians. The basic functionality of an EMR is to allow storage, retrieval and manipulation of records. In order to communicate the information of an EMR system between different branches of a healthcare organization, there is a need to follow a standard that provides interoperability. Since healthcare is a most demanding area as more people are concerned about their health, this system should be scalable, fault tolerant, reliable, secure, timely respondent, sustainable and maintainable.

It is a rarity to deploy an Electronic Medical Record (EMR) system on cloud. Also another challenging task is to incorporate semantic web technologies in EMR's and presenting the complex medical data in a meaningful and intelligent manner in healthcare. This will require the integration of Semantics Web and SaaS model (Software as a Service, 2011) with best featured existing EMR for developing an efficient healthcare semantic web services for the cloud. Some initiatives are underway worldwide such as Health Services Specification Project (HSSP), a joint-venture of HL7 and Object Management Group (OMG, 2011), providing standardized service interface specifications.

Therefore, there is a need to design and implement semantic based healthcare service on cloud for storage, retrieval and manipulation of patient data and medical records. Thus SEMR (Semantic Electronic Medical Record) system will provide the solution for highly intensive patient and medical data sharing, semantic interoperability and management with its availability for larger community access through cloud infrastructure.

### **2.1 Related work**

Some of the current open source EMR systems are listed below with their functionalities and drawbacks.

OpenEMR is an open source clinical practice management system (OpenEMR, 2011). The system can track patient demographics, patient medical records, scheduling, billing,

but now have to be modified to support primary care settings of patients, mostly outside of the walls of the hospital. The traditional primary care teams also have to redesign the workflow as they add new care coordination staff and EMR technology to achieve the desired

Interoperability is the ability of a healthcare system to share information and have that information properly interpreted by the receiving system in the same sense as intended by the transmitting system. Standardization provides us an effective way of communication to achieve the goal of semantic interoperability. HL7 is one of the healthcare standards that allow communication of healthcare systems and allow sharing of data around the globe. The important requirement is to capture relevant information and then make it widely available for others. Therefore, the need is to have a system that can provide best services in terms of meaningful data sharing and discovery. HL7, as it evolves, helps us with a technical business model to fulfill the vision of standard based information exchange in diverse,

In the era of Semantic Web and cloud computing, there is a need and demand of such an EMR system where timely, accurate and rapid availability of healthcare services can be possible that can manage patient's health data and helps physicians and patients. EMR is basically a part of local standalone Health Information System (HIS) that is an organization's legal proprietary, it includes hospitals as well as doctors, clinicians and physicians. The basic functionality of an EMR is to allow storage, retrieval and manipulation of records. In order to communicate the information of an EMR system between different branches of a healthcare organization, there is a need to follow a standard that provides interoperability. Since healthcare is a most demanding area as more people are concerned about their health, this system should be scalable, fault tolerant, reliable, secure, timely respondent, sustainable

It is a rarity to deploy an Electronic Medical Record (EMR) system on cloud. Also another challenging task is to incorporate semantic web technologies in EMR's and presenting the complex medical data in a meaningful and intelligent manner in healthcare. This will require the integration of Semantics Web and SaaS model (Software as a Service, 2011) with best featured existing EMR for developing an efficient healthcare semantic web services for the cloud. Some initiatives are underway worldwide such as Health Services Specification Project (HSSP), a joint-venture of HL7 and Object Management Group (OMG, 2011),

Therefore, there is a need to design and implement semantic based healthcare service on cloud for storage, retrieval and manipulation of patient data and medical records. Thus SEMR (Semantic Electronic Medical Record) system will provide the solution for highly intensive patient and medical data sharing, semantic interoperability and management with

Some of the current open source EMR systems are listed below with their functionalities and

OpenEMR is an open source clinical practice management system (OpenEMR, 2011). The system can track patient demographics, patient medical records, scheduling, billing,

goal of improving the clinical outcome at reduced costs (Ginsburg, 2006).

integrated health information systems (HL7, 2011; HLH, 2011; HL7, 2009).

providing standardized service interface specifications.

its availability for larger community access through cloud infrastructure.

and maintainable.

**2.1 Related work** 

drawbacks.

multilingual support and prescription. In short the system provides all basic functionalities that any hospital EMR system can provide but it is restricted to a hospital.

OpenEMR Virtual Appliance (OpenEMR Virtual Appliance, 2011) is a comprehensive open source Medical Practice Management Software Appliance, which provides office scheduling, electronic medical records, prescriptions, insurance billing, accounting and access controls. This appliance has many possible applications, such as a fully functional demo, a testing/developing platform, and as the starting point in real world clinic applications. It can be run on any operating system that supports the VMware Player.

OpenMRS (OpenMRS, 2011) is a full open source healthcare system and has the ability to configure the system to new requirements without programming and to interoperate with other systems whether open or closed. Both of these open source systems are refined under Health Insurance Portability and Accountability Act (HIPAA, 2011) and CCHIT (CCHIT, 2011; CCHIT EMR, 2011) certified.

SequelMed EMR (SequelMed EMR, 2011) is a secure, patient centric, medical record, integrated with Sequel Systems' medical billing software (SequelMed EPM). The system can automate clinical documentation and have Decision Support Tools and Alerts, Integrated Patient Education Protocols, wireless and internet access to medical records.

ClearHealth (Clear-health, 2011) is open source software and include five major areas of healthcare practice operations including scheduling, billing, EMR, HIPAA Security and accounts receivables."

XChart (XChart, 2011) is a paper based project by the Open Healthcare Group that promotes EMR, based in XML.

SmartCare (SmartCare,2011) is software that develops EMR programs and particularly used in Zambia.

Zimbra (Zimbra, 2011) gives e-mail solution for government offices, education institutes and other business environments. Medical professionals can also benefit from its fast backup and recovery of mailboxes, anti-spam and anti-virus protection, this software has also support for BlackBerry and other mobile devices, and their flexible applications. All of these systems are open source and primary care systems.

These systems provide basic clinical practice that is helpful for patients' medical record but do not support interoperability among different workflow components such as laboratory, medical reports, patient administration, pharmacy, insurance, billing, and prescription among medical repositories, hospitals, pharmacies and clinics.

#### **2.2 Methodology**

In order to avoid the burden of management of technological infrastructure, SaaS based solution should be used to develop the system on top of cloud infrastructure. To achieve full potential of machine process able SaaS service model based EMR, semantics need to be added. Semantics bring the benefits of unambiguous definition of service functionality and the external interfaces of services reduce human effort in integrating services to SOA, improve dynamism and stability to Web services. Our proposed system will ensure timely delivery of health care information and will ensure its confidentiality. The proposed

Semantic Interoperability in E-Health for Improved Healthcare 117

and stability to Web services (Semantic SOA, 2011). Therefore the next phase is to upgrade SaaS service model based EMR to Semantic EMR system as SaaS service model for healthcare. In this phase the literature review, analysis and design of semantic web services identification and development for the purpose of fulfilling patient administration

HL7 provides semantic interoperability; therefore the communication in our proposed system is based on HL7 standard based data exchange format. This step will embed HL7

In the final step an interface of this system can also be provided for smart-phones for physicians and patients to access our system also from outside the patient care premises.

In the architecture four types of services are categorized for SEMR system SaaS service

The layered architecture is categorized as presentation layer, business logic layer, data

These services perform the logic of business processes with the help of other services. Business logic is fulfilled by the management of processes and data through data management services. For example request for patient referrals would be fulfilled through

These services provide metadata specifications and standards for message generation, database mapping, and data integration, interoperability, through data modeling, data

These services are responsible for the authorization and authentication of data transmitted

The elaborated services architecture is given in Figure 4. We demonstrated the sequence of functionality of Patient Administration service in the following paragraph. In order to use Patient Administration service from our SEMR SaaS service based system we presented patient registration scenario where a doctor registers a patient through our SEMR system Interface. The doctor will give patient demographic information in the form by using our system. As our system is semantic based, the Semantic Gateway service will perform semantic composition. The query of the doctor will be standardized with the help of Message Generation Data Management service that will

management layer and database layer. The layered architecture is shown in Figure 3.

underlying data management service provided through message generation service.

transformation and data workflows. These services help data management services.

standard in our proposed system for medical data communication.

**2.3.1 Architectural layout of SaaS based SEMR system** 

These services manage the data for business process services.

and stored for the working of data management services.

model. The EMR services are part of these categories that are semantic based.

requirements would be carried out.





generate an HL7 message.

healthcare system will be developed as semantic web services based on SaaS model and will be deployed on cloud infrastructure. This work will bring significant improvements in current EMR systems through interoperable, automated and seamless meaningful communication.

The ultimate goal is to exploit the EMR system's functionalities as semantic web services. Web Services Modeling Framework (WSMF) will provide the platform for automatic web service discovery, composition, and invocation that makes the product efficient. As EMR system, software will be developed as a service, semantic web technologies will be used to incorporate semantics in the services and the communication will take place through web services and would ensure timely delivery of medical information.

SEMR (Semantic Electronic Medical Record) system will be used to capture and manage patient's data and information by using these two approaches: Semantic Web and Software as a Service (SaaS) service model on cloud. These are emerging approaches that can bring novel way to properly manage patient's data and medical records. SOA and SaaS establish a SaaS service model by leveraging the benefits of SaaS solution and SOA infrastructure. SOA enhances reliability, reduces hardware acquisition costs, leverages existing development skills, and accelerates movement to standards based server and application consolidation. In this way SOA provides a data bridge between incompatible technologies.

Furthermore SaaS solution will provide data and system availability, secure and reliable performance, and maximum system throughput. Communication in the system will be handled by HL7 for semantic interoperability. This system is based on HL7 standard based data exchange format.

The important part of this project is the integration of Semantic Web and SaaS model with best featured existing EMR for developing an efficient healthcare semantic web services for the cloud. This mostly involves research and implementation challenges exist within best featured existing EMR for developing an efficient healthcare semantic web services for the cloud.

## **2.3 Proposed architecture**

The proposed system will be semantic based SaaS service model developed on top of cloud for healthcare domain. SOA and SaaS establish a SaaS service model by leveraging the benefits of SaaS solution and SOA infrastructure. SOA enhances reliability, reduces hardware acquisition costs, leverages existing development skills, accelerates movement to standards-based server and application consolidation, provides a data bridge between incompatible technologies (SOA, 2010).

In the requirement gathering phase literature review and analysis of basic functionalities of EMR systems and SaaS service model would be carried out. Therefore SaaS service model based EMR system would be designed in the first phase. In order to avoid the burden of management of technological infrastructure, we will use SaaS based solution by developing our proposed system on top of cloud infrastructure. To make SaaS service model based EMR and machine process able and achieve its full potential, semantics needs to be added.

Semantics brings the benefits of unambiguous definition of service functionality and external interfaces reduce human effort in integrating services to SOA, improve dynamism

healthcare system will be developed as semantic web services based on SaaS model and will be deployed on cloud infrastructure. This work will bring significant improvements in current EMR systems through interoperable, automated and seamless meaningful

The ultimate goal is to exploit the EMR system's functionalities as semantic web services. Web Services Modeling Framework (WSMF) will provide the platform for automatic web service discovery, composition, and invocation that makes the product efficient. As EMR system, software will be developed as a service, semantic web technologies will be used to incorporate semantics in the services and the communication will take place through web

SEMR (Semantic Electronic Medical Record) system will be used to capture and manage patient's data and information by using these two approaches: Semantic Web and Software as a Service (SaaS) service model on cloud. These are emerging approaches that can bring novel way to properly manage patient's data and medical records. SOA and SaaS establish a SaaS service model by leveraging the benefits of SaaS solution and SOA infrastructure. SOA enhances reliability, reduces hardware acquisition costs, leverages existing development skills, and accelerates movement to standards based server and application consolidation. In

Furthermore SaaS solution will provide data and system availability, secure and reliable performance, and maximum system throughput. Communication in the system will be handled by HL7 for semantic interoperability. This system is based on HL7 standard based

The important part of this project is the integration of Semantic Web and SaaS model with best featured existing EMR for developing an efficient healthcare semantic web services for the cloud. This mostly involves research and implementation challenges exist within best featured existing EMR for developing an efficient healthcare semantic web services for the

The proposed system will be semantic based SaaS service model developed on top of cloud for healthcare domain. SOA and SaaS establish a SaaS service model by leveraging the benefits of SaaS solution and SOA infrastructure. SOA enhances reliability, reduces hardware acquisition costs, leverages existing development skills, accelerates movement to standards-based server and application consolidation, provides a data bridge between

In the requirement gathering phase literature review and analysis of basic functionalities of EMR systems and SaaS service model would be carried out. Therefore SaaS service model based EMR system would be designed in the first phase. In order to avoid the burden of management of technological infrastructure, we will use SaaS based solution by developing our proposed system on top of cloud infrastructure. To make SaaS service model based EMR and machine process able and achieve its full potential, semantics needs to be added. Semantics brings the benefits of unambiguous definition of service functionality and external interfaces reduce human effort in integrating services to SOA, improve dynamism

services and would ensure timely delivery of medical information.

this way SOA provides a data bridge between incompatible technologies.

communication.

data exchange format.

**2.3 Proposed architecture** 

incompatible technologies (SOA, 2010).

cloud.

and stability to Web services (Semantic SOA, 2011). Therefore the next phase is to upgrade SaaS service model based EMR to Semantic EMR system as SaaS service model for healthcare. In this phase the literature review, analysis and design of semantic web services identification and development for the purpose of fulfilling patient administration requirements would be carried out.

HL7 provides semantic interoperability; therefore the communication in our proposed system is based on HL7 standard based data exchange format. This step will embed HL7 standard in our proposed system for medical data communication.

In the final step an interface of this system can also be provided for smart-phones for physicians and patients to access our system also from outside the patient care premises.

In the architecture four types of services are categorized for SEMR system SaaS service model. The EMR services are part of these categories that are semantic based.

## **2.3.1 Architectural layout of SaaS based SEMR system**

The layered architecture is categorized as presentation layer, business logic layer, data management layer and database layer. The layered architecture is shown in Figure 3.

#### - **Business process services:**

These services perform the logic of business processes with the help of other services. Business logic is fulfilled by the management of processes and data through data management services. For example request for patient referrals would be fulfilled through underlying data management service provided through message generation service.

#### - **Data management services:**

These services manage the data for business process services.

### - **Metadata services:**

These services provide metadata specifications and standards for message generation, database mapping, and data integration, interoperability, through data modeling, data transformation and data workflows. These services help data management services.

#### - **Security services**:

These services are responsible for the authorization and authentication of data transmitted and stored for the working of data management services.

The elaborated services architecture is given in Figure 4. We demonstrated the sequence of functionality of Patient Administration service in the following paragraph. In order to use Patient Administration service from our SEMR SaaS service based system we presented patient registration scenario where a doctor registers a patient through our SEMR system Interface. The doctor will give patient demographic information in the form by using our system. As our system is semantic based, the Semantic Gateway service will perform semantic composition. The query of the doctor will be standardized with the help of Message Generation Data Management service that will generate an HL7 message.

Semantic Interoperability in E-Health for Improved Healthcare 119

Semantic Gateway uses Semantic Gateway service as a part of business process services that is used for taking information from the user and resolving it by using the combination of other services. This provides the semantic annotations to services. Service ontology and Domain ontology are defined for semantic execution. Services and Ontology repositories will be the knowledge bases for the Semantic Gateway service. Reasoner is used in Semantic Gateway for inference about services semantics at runtime. Services of EMR system will be discovered semantically through Semantic Composition business process service. The business logic of this service will perform parsing, choreography, ranking and selection with the help of Semantic Gateway. Semantic gateway is shown in Figure 5. Semantic Gateway is

The advancement in Information Technology is playing increasing role in healthcare and has managed to improve the efficiency of health services to common people. Health informatics plays a vital role in the integration of Information Technology in healthcare domain. However healthcare organizations are facing problems related to communication of right information to appropriate. Due to data deluge, information retrieval and analysis has become an important problem in various fields including healthcare. Semantic web

The innovation and the standardization of web services provide basic building blocks for information exchange. To exploit web services to their full potential, semantics must be specified. Semantic web technologies play a pivotal role in bringing automation in the process flows. OWL-S provides ontologies for describing web services with the help of semantic constructs in an unambiguous and machine interpretable form. OWL-S follows layered structure of markup languages as HTML, XML, RDF and has built on OWL recommendation of W3C. Its ontologies describe domain concepts of services (e.g., travel, ebusiness, healthcare information) and business logic. The data flow and controls of the services are related to the domain ontologies through inputs, outputs, preconditions and effects. OWL-S ontologies divide service descriptions in four main parts: process model,

Currently WSMX framework provides automatic service discovery, composition and execution of web services. It provides information exchange between users and service providers and fulfill user specified goal by invoking end point web services. The main

technologies provide extensible, flexible and efficient information.

service profile, service grounding and the service.

Fig. 5. Semantic Gateway.

discussed in following sections.

**2.3.2 Semantic gateway** 

Fig. 3. Architecture for SaaS based SEMR system.

Fig. 4. Elaborated Services Architecture.

The Semantic Gateway service will then discover, select and invoke Patient Administration Business Process service through HL7 message parsing. The Patient Administration service will call the Patient data service for data management. The Patient data service will call the Authorization service to authorize the patient for viewing his medical data. This service will assign user name and password to the patient. Then the Patient data service will store the registered patient in the patient database.

Fig. 5. Semantic Gateway.

The Semantic Gateway service will then discover, select and invoke Patient Administration Business Process service through HL7 message parsing. The Patient Administration service will call the Patient data service for data management. The Patient data service will call the Authorization service to authorize the patient for viewing his medical data. This service will assign user name and password to the patient. Then the Patient data service will store the

Fig. 3. Architecture for SaaS based SEMR system.

Fig. 4. Elaborated Services Architecture.

registered patient in the patient database.

Semantic Gateway uses Semantic Gateway service as a part of business process services that is used for taking information from the user and resolving it by using the combination of other services. This provides the semantic annotations to services. Service ontology and Domain ontology are defined for semantic execution. Services and Ontology repositories will be the knowledge bases for the Semantic Gateway service. Reasoner is used in Semantic Gateway for inference about services semantics at runtime. Services of EMR system will be discovered semantically through Semantic Composition business process service. The business logic of this service will perform parsing, choreography, ranking and selection with the help of Semantic Gateway. Semantic gateway is shown in Figure 5. Semantic Gateway is discussed in following sections.

## **2.3.2 Semantic gateway**

The advancement in Information Technology is playing increasing role in healthcare and has managed to improve the efficiency of health services to common people. Health informatics plays a vital role in the integration of Information Technology in healthcare domain. However healthcare organizations are facing problems related to communication of right information to appropriate. Due to data deluge, information retrieval and analysis has become an important problem in various fields including healthcare. Semantic web technologies provide extensible, flexible and efficient information.

The innovation and the standardization of web services provide basic building blocks for information exchange. To exploit web services to their full potential, semantics must be specified. Semantic web technologies play a pivotal role in bringing automation in the process flows. OWL-S provides ontologies for describing web services with the help of semantic constructs in an unambiguous and machine interpretable form. OWL-S follows layered structure of markup languages as HTML, XML, RDF and has built on OWL recommendation of W3C. Its ontologies describe domain concepts of services (e.g., travel, ebusiness, healthcare information) and business logic. The data flow and controls of the services are related to the domain ontologies through inputs, outputs, preconditions and effects. OWL-S ontologies divide service descriptions in four main parts: process model, service profile, service grounding and the service.

Currently WSMX framework provides automatic service discovery, composition and execution of web services. It provides information exchange between users and service providers and fulfill user specified goal by invoking end point web services. The main

Semantic Interoperability in E-Health for Improved Healthcare 121

ebXML Registry (Dogac et al., 2008) give industry groups and enterprises the ability to share business semantic information and business process interfaces in form of XML. This registry has some extensions for medical data registration, annotation, discovery and retrieval in form or archetypes data definitions where registry semantic constructs are used. They provide archetype metadata ontology and describe the techniques to access archetype semantics through ebXML query facilities. They also provide mechanism, how archetype data can be retrieved from underlying clinical information systems by using ebXML Web

The FUSION Semantic Registry (Kourtesis and Paraskakis, 2006) is a semantically-enhanced Web service registry based on UDDI, SAWSDL and OWL. This registry augments and enhances the discovery facilities of typical UDDI registry and based on UDDI without changing its implementation. This registry performs matchmaking at data-level and developed by SEERC in the context of research project FUSION and released as open source software. Fusion registry has no matchmaking based on inputs, outputs, preconditions and

Artemis project (Dogac et al., 2006), exploits ontologies based on the domain knowledge exposed by the healthcare information standards like HL7, CEN TC251, ISO TC215 and GEHR. Artemis Web service architecture has no any globally agreed ontologies; rather healthcare institutes resolve their semantic differences through a mediator component. The mediator component works in a P2P manner and uses ontologies in order to facilitate

CASCOM is an agent-based approach used for semantic service discovery and coordination

Cesar Caceres is another approach that focuses on Agent-Based Semantic Service Discovery

COCOON Glue is a prototype of WSMO Discovery engine for the healthcare field to find

Registries are important in a large scale, distributed environment, such as the semantic web. They provide the necessary functionality that allows service providers to expose information of their services to potential users. Various types of approaches that are being followed for storing and accessing information over the web are registry-based discovery mechanisms (Willmott, 2005), indexing methods (UDDI) (Clement, 2004) and publish/subscribe approach (Nawaz et al., 2007). In healthcare domain there is no such mechanism of binding healthcare service providers and requesters in order to discover healthcare data for use in emergency situation. There is lack of registries that provide publish and retrieval of healthcare data through web services. There is no any healthcare services publish in a semantic way for the interoperability of health information exchanged in an efficient manner. Healthcare information is more complex and has diverse dimensions. UDDI (Paolucci, 2002; Srinivasan, 2004) and ebXML (Dogac et al., 2008) do not provide such

We proposed a framework based on OWL-S semantic layer which would provide automatic service discovery, composition, invocation and execution of web services for healthcare

services.

effects capabilities of services.

semantic negotiation among involved institutes.

semantic interoperability in healthcare domain.

**2.3.3 Methodology** 

in mobile eHealth environment (Fröhlich et al., 2007).

for medical-emergency management (Cáceresc et al., 2006).

out the most appropriate advice services (Emanuele and Cerizza, 2005).

strength of WSMO over other semantic web technologies is its discovery mechanism. WSMO is based on the Web Service Modeling Framework (WSMF). WSML is used to describe services' description into Ontologies, Web services, Goals and Mediators (WSMO).

The innovation and the standardization of web services have set the concept of web services as the basic building blocks of information technology systems for Service Oriented Architectures (SOA) applications. The idea is to explore such sophisticated SOA technologies that make the discovery of services for requested users appropriate (Erl, 2005). SOA ensures interoperability, flexibility and extensibility across heterogeneous environments. In service oriented computing services are used to develop fast, economical, interoperable, evolvable, and extremely distributed applications. Services are selfgoverning, platform-independent entities that can be described, published, discovered, and loosely coupled (Papazoglou, 2007).

Traditional approaches to services publication and discovery have generally relied on the existence of pre-defined registry services like Universal Description, Discovery and Integration (UDDI) (Clement, 2004). Often the description of a service is limited in existing registry, with little or no support for problem specific descriptions. Semantic registries with the use of OWL-S attempt to overcome this limitation and provide a rich semantic description based on ontologies. Semantic matchmaking generally focuses on the problem of identifying services on the basis of the capabilities that they provide. We proposed an OWL-S based Semantic Registry for healthcare information provision. This chapter also presents healthcare service ontology (Iftikhar et al., 2010) developed through the specifications of HL7 Service Functional Model, which is used in our Semantic Registry for publishing and discovering HL7 compliant healthcare semantic web services. HL7 is a well-known healthcare standard that provides specification for standardization of information exchanged among healthcare applications.

In paper (Srinivasan, 2004), Authors have proposed OWL-S/UDDI Matchmaker as an extension of UDDI. Before registering OWL-S based Web Services on UDDI, OWL-S/UDDI Matchmaker converts service profile of these services to UDDI data structure and then stores them on UDDI. During web service discovery, OWL-S/UDDI Matchmaker translates the services back into OWL-S format. Matching takes place between the service request and the published services advertisements present in the registry. The proposed solution enhances the UDDI registry for semantic based searching and capability based matching. UDDI registry has some inherent limitations including lack of semantic representations of contents. The matching process proposed in this paper is restricted to Inputs and Outputs matching of the service profile.

The DAML-S Matchmaker (Paolucci, 2002) was developed by the Intelligent Software Agents Group at Carnegie- Mellon University. The matchmaking system is a database where service providers can register their Web services via DAML-S descriptions through a Web interface. The system then allows service requesters to upload their service requests. The matchmaking algorithm matches the types associated with each input or output parameter. For each parameter (either input or output) there are several degrees of matching, depending on the semantic relationship between the parameters of the advertisement and the request. Based on these results a global matching result is determined.

strength of WSMO over other semantic web technologies is its discovery mechanism. WSMO is based on the Web Service Modeling Framework (WSMF). WSML is used to describe services' description into Ontologies, Web services, Goals and Mediators (WSMO). The innovation and the standardization of web services have set the concept of web services as the basic building blocks of information technology systems for Service Oriented Architectures (SOA) applications. The idea is to explore such sophisticated SOA technologies that make the discovery of services for requested users appropriate (Erl, 2005). SOA ensures interoperability, flexibility and extensibility across heterogeneous environments. In service oriented computing services are used to develop fast, economical, interoperable, evolvable, and extremely distributed applications. Services are selfgoverning, platform-independent entities that can be described, published, discovered, and

Traditional approaches to services publication and discovery have generally relied on the existence of pre-defined registry services like Universal Description, Discovery and Integration (UDDI) (Clement, 2004). Often the description of a service is limited in existing registry, with little or no support for problem specific descriptions. Semantic registries with the use of OWL-S attempt to overcome this limitation and provide a rich semantic description based on ontologies. Semantic matchmaking generally focuses on the problem of identifying services on the basis of the capabilities that they provide. We proposed an OWL-S based Semantic Registry for healthcare information provision. This chapter also presents healthcare service ontology (Iftikhar et al., 2010) developed through the specifications of HL7 Service Functional Model, which is used in our Semantic Registry for publishing and discovering HL7 compliant healthcare semantic web services. HL7 is a well-known healthcare standard that provides specification for standardization of information

In paper (Srinivasan, 2004), Authors have proposed OWL-S/UDDI Matchmaker as an extension of UDDI. Before registering OWL-S based Web Services on UDDI, OWL-S/UDDI Matchmaker converts service profile of these services to UDDI data structure and then stores them on UDDI. During web service discovery, OWL-S/UDDI Matchmaker translates the services back into OWL-S format. Matching takes place between the service request and the published services advertisements present in the registry. The proposed solution enhances the UDDI registry for semantic based searching and capability based matching. UDDI registry has some inherent limitations including lack of semantic representations of contents. The matching process proposed in this paper is restricted to Inputs and Outputs

The DAML-S Matchmaker (Paolucci, 2002) was developed by the Intelligent Software Agents Group at Carnegie- Mellon University. The matchmaking system is a database where service providers can register their Web services via DAML-S descriptions through a Web interface. The system then allows service requesters to upload their service requests. The matchmaking algorithm matches the types associated with each input or output parameter. For each parameter (either input or output) there are several degrees of matching, depending on the semantic relationship between the parameters of the advertisement and the request. Based on these results a global matching result is

loosely coupled (Papazoglou, 2007).

exchanged among healthcare applications.

matching of the service profile.

determined.

ebXML Registry (Dogac et al., 2008) give industry groups and enterprises the ability to share business semantic information and business process interfaces in form of XML. This registry has some extensions for medical data registration, annotation, discovery and retrieval in form or archetypes data definitions where registry semantic constructs are used. They provide archetype metadata ontology and describe the techniques to access archetype semantics through ebXML query facilities. They also provide mechanism, how archetype data can be retrieved from underlying clinical information systems by using ebXML Web services.

The FUSION Semantic Registry (Kourtesis and Paraskakis, 2006) is a semantically-enhanced Web service registry based on UDDI, SAWSDL and OWL. This registry augments and enhances the discovery facilities of typical UDDI registry and based on UDDI without changing its implementation. This registry performs matchmaking at data-level and developed by SEERC in the context of research project FUSION and released as open source software. Fusion registry has no matchmaking based on inputs, outputs, preconditions and effects capabilities of services.

Artemis project (Dogac et al., 2006), exploits ontologies based on the domain knowledge exposed by the healthcare information standards like HL7, CEN TC251, ISO TC215 and GEHR. Artemis Web service architecture has no any globally agreed ontologies; rather healthcare institutes resolve their semantic differences through a mediator component. The mediator component works in a P2P manner and uses ontologies in order to facilitate semantic negotiation among involved institutes.

CASCOM is an agent-based approach used for semantic service discovery and coordination in mobile eHealth environment (Fröhlich et al., 2007).

Cesar Caceres is another approach that focuses on Agent-Based Semantic Service Discovery for medical-emergency management (Cáceresc et al., 2006).

COCOON Glue is a prototype of WSMO Discovery engine for the healthcare field to find out the most appropriate advice services (Emanuele and Cerizza, 2005).

Registries are important in a large scale, distributed environment, such as the semantic web. They provide the necessary functionality that allows service providers to expose information of their services to potential users. Various types of approaches that are being followed for storing and accessing information over the web are registry-based discovery mechanisms (Willmott, 2005), indexing methods (UDDI) (Clement, 2004) and publish/subscribe approach (Nawaz et al., 2007). In healthcare domain there is no such mechanism of binding healthcare service providers and requesters in order to discover healthcare data for use in emergency situation. There is lack of registries that provide publish and retrieval of healthcare data through web services. There is no any healthcare services publish in a semantic way for the interoperability of health information exchanged in an efficient manner. Healthcare information is more complex and has diverse dimensions. UDDI (Paolucci, 2002; Srinivasan, 2004) and ebXML (Dogac et al., 2008) do not provide such semantic interoperability in healthcare domain.

### **2.3.3 Methodology**

We proposed a framework based on OWL-S semantic layer which would provide automatic service discovery, composition, invocation and execution of web services for healthcare

Semantic Interoperability in E-Health for Improved Healthcare 123

utilizes semantic-based publish-subscribe model in conjunction with UDDI. In the Service discovery process we analyzed that there are less updates and frequent searches hence push model is the right approach to use. Through our web based Semantic Registry in Figure 6 users can publish and request service descriptions from Service Publish Interface and Service Discovery Interface. The service descriptions stored in OWL-S Profile Repository as OWL-S service profile. The matching algorithm semantically enhances ontology mappings for providing services descriptions to requesters. It assigns scores to individual concepts of the advertised service by concept matching with that of the requested one and then assigns overall ranking to advertisement on the basis of individual scores. The results have shown a significant increase in precision and recall of service discovery as compared to UDDI approach. The users of the UDDI registry can also switch between traditional syntax-based and proposed semantic-based searching. Normal users can access OWL-S profiles and WSDL advertisements through inquiry API provided by UDDI registry. Semantic Matchmaker used in this work (Capability Matching Module) performs Inputs, Outputs, Preconditions and Effects and Service Category Matching. Capabilities of OWL-S web services; Preconditions and Effects represent a "state" before and after the execution of a service respectively. In this paper we enhance the OWL-S semantic web services capability matching to Inputs, Outputs, Preconditions and Effects. In this way this work will cover

data as well as functional semantics modeling aspects.

Fig. 6. Semantic Framework.

service providers and end users. Our proposed Semantic Registry would be the key foundation block upon which electronic information is exchanged in an interoperable manner among disparate communities through web services semantics. It would be an Ontology based semantic description model explicitly represents information semantics in abstract and concrete level and resolve heterogeneity.

The system will consist of entry points for the communication to take place. The OWL-S/WSDL grounding mechanism would be used for end point service invocation. In our framework, we proposed to perform goal-oriented discovery with semantic matchmaking of OWL-S ontologies. The proposed use of semantic web services specification language such as OWL-S for describing web services semantically would result in better information exchange. We will incorporate three views of services into user demand to satisfy the requirements of end users in healthcare domain. These views are: customization (who is demanding information), situation (when and where the demand is occurred) and quality (how important the demand is). Service provider, who is going to provide their services for use by appropriate users, will take advantage of the complementary strengths of OWL-S, and these three views of services.

OWL-S has classes of WSDLGrounding for realizing specific elements within WSDL for OWL-S/WSDL Grounding mechanism. This mechanism is more mature as compared to WSMX. WSMX required lowering and lifting mechanism and XSLT transformations for WSMO/WSDL groundings. WSMX also uses RDF and XML as a carrier between WSML and WSDL for grounding mechanism, where loss of semantics can be observed. WSMO provides goal oriented discovery and mediation between ontologies, web services and goals that are not provided by OWL-S. In OWL-S, there is no clear distinction between choreography and orchestration. OWL-S Process Model defines choreography and orchestration. There is no need of separate management for these two processes. In WSMO, the choreography and orchestration are specified clearly. WSMX has interfaces for choreography element (provides the necessary information for communicating with the service), and the orchestration class element, (describes how the service makes use of other services in order to achieve the goal). OWL-S has no Semantic Registry for web service discovery, selection and invocation mechanism, it depends on UDDI for web services discovery. Whereas WSMX framework has three steps of discovery, Goal Discovery, Semantic Web service Discovery and End point service Discovery using any one of the approach: keyword based, light weight and heavy weight discovery.

Our framework would work with both central and distributed computing infrastructures as shown in Figure 1. It will provide services for healthcare information provision and for collaboration of Personal Health Record (PHR) systems, Electronic Health Record (EHR) systems, Health Information Management (HIMS) systems, and other hospital and clinical systems.

Our OWLS Semantic Registry is used for HL7 compliant healthcare semantic web services and metadata publication, discovery, composition and invocation in healthcare domain. One of our major concerns is to describe the HL7 compliant healthcare services publication and discovery. Our vision is to have a Semantic Registry as the key foundation block upon which electronic health information would be exchanged among disparate communities. We already have a proactive approach for efficient discovery based on service category that

service providers and end users. Our proposed Semantic Registry would be the key foundation block upon which electronic information is exchanged in an interoperable manner among disparate communities through web services semantics. It would be an Ontology based semantic description model explicitly represents information semantics in

The system will consist of entry points for the communication to take place. The OWL-S/WSDL grounding mechanism would be used for end point service invocation. In our framework, we proposed to perform goal-oriented discovery with semantic matchmaking of OWL-S ontologies. The proposed use of semantic web services specification language such as OWL-S for describing web services semantically would result in better information exchange. We will incorporate three views of services into user demand to satisfy the requirements of end users in healthcare domain. These views are: customization (who is demanding information), situation (when and where the demand is occurred) and quality (how important the demand is). Service provider, who is going to provide their services for use by appropriate users, will take advantage of the complementary strengths of OWL-S,

OWL-S has classes of WSDLGrounding for realizing specific elements within WSDL for OWL-S/WSDL Grounding mechanism. This mechanism is more mature as compared to WSMX. WSMX required lowering and lifting mechanism and XSLT transformations for WSMO/WSDL groundings. WSMX also uses RDF and XML as a carrier between WSML and WSDL for grounding mechanism, where loss of semantics can be observed. WSMO provides goal oriented discovery and mediation between ontologies, web services and goals that are not provided by OWL-S. In OWL-S, there is no clear distinction between choreography and orchestration. OWL-S Process Model defines choreography and orchestration. There is no need of separate management for these two processes. In WSMO, the choreography and orchestration are specified clearly. WSMX has interfaces for choreography element (provides the necessary information for communicating with the service), and the orchestration class element, (describes how the service makes use of other services in order to achieve the goal). OWL-S has no Semantic Registry for web service discovery, selection and invocation mechanism, it depends on UDDI for web services discovery. Whereas WSMX framework has three steps of discovery, Goal Discovery, Semantic Web service Discovery and End point service Discovery using any one of the

Our framework would work with both central and distributed computing infrastructures as shown in Figure 1. It will provide services for healthcare information provision and for collaboration of Personal Health Record (PHR) systems, Electronic Health Record (EHR) systems, Health Information Management (HIMS) systems, and other hospital and clinical

Our OWLS Semantic Registry is used for HL7 compliant healthcare semantic web services and metadata publication, discovery, composition and invocation in healthcare domain. One of our major concerns is to describe the HL7 compliant healthcare services publication and discovery. Our vision is to have a Semantic Registry as the key foundation block upon which electronic health information would be exchanged among disparate communities. We already have a proactive approach for efficient discovery based on service category that

approach: keyword based, light weight and heavy weight discovery.

abstract and concrete level and resolve heterogeneity.

and these three views of services.

systems.

utilizes semantic-based publish-subscribe model in conjunction with UDDI. In the Service discovery process we analyzed that there are less updates and frequent searches hence push model is the right approach to use. Through our web based Semantic Registry in Figure 6 users can publish and request service descriptions from Service Publish Interface and Service Discovery Interface. The service descriptions stored in OWL-S Profile Repository as OWL-S service profile. The matching algorithm semantically enhances ontology mappings for providing services descriptions to requesters. It assigns scores to individual concepts of the advertised service by concept matching with that of the requested one and then assigns overall ranking to advertisement on the basis of individual scores. The results have shown a significant increase in precision and recall of service discovery as compared to UDDI approach. The users of the UDDI registry can also switch between traditional syntax-based and proposed semantic-based searching. Normal users can access OWL-S profiles and WSDL advertisements through inquiry API provided by UDDI registry. Semantic Matchmaker used in this work (Capability Matching Module) performs Inputs, Outputs, Preconditions and Effects and Service Category Matching. Capabilities of OWL-S web services; Preconditions and Effects represent a "state" before and after the execution of a service respectively. In this paper we enhance the OWL-S semantic web services capability matching to Inputs, Outputs, Preconditions and Effects. In this way this work will cover data as well as functional semantics modeling aspects.

Fig. 6. Semantic Framework.

Semantic Interoperability in E-Health for Improved Healthcare 125

Fig. 7. Healthcare Service Ontology.

Table 1. HL7 Service Specifications.

**Service specifications HL7 v3 Artifacts Used [5]** 

constructs)

Service Domain, Topic, Application Role, Trigger Events e.g Lab domain

Capabilities DIM/D-MIM, Application Role, Storyboards, Activity Diagrams,

ApplicationLevelAck, ControlActProcess etc. Message RIM, DIM, CIM/R-MIM, CMETs, Vocabulary and Data Types (LIM,

Use Cases, Trigger Events, (Interaction, CIM/R-MIM, LIM/ HMD, Message Type – if using message oriented level constructs) e.g

HMD, Message Type and Schema – if using actual message level

Interface Domain, Topic, Application Role, Trigger Events

Our proposed framework Figure 6 consists of following components:

	- **Adaptor:** End user request will be converted into OWL-S to make it adaptable to internal environment. OWL-S annotations of end user request will be provided to discovery component, which will perform the user oriented discovery with the help of SR, semantic matchmaking, selection & ranking components
	- **Monitoring:** This component will send the request to best selected end point web service candidate.
	- **Invoker:** The response from the end point web service will be given to the user through this component. This component will synchronize all responses from more than one end point web services.
	- **OWL-S ontologies:** Semantic descriptions of web services provided by service provider.
	- **Semantic Registry (SR):** SR manages semantic annotations of services provided by service providers in repository and handles discovery process. It also provides OWL-S to WSDL and WSDL to OWL-S translations with the help of OWL-S ontologies.
	- **Repository:** OWL-S ontologies' semantic annotations would be stored in repository to be accessed latter for discovery purpose.
	- **Services domain ontology**
	- **Discovery:** This component will perform keyword based, light weight and heavy weight discovery.
	- **Semantic matchmaking:** During discovery process similar ontologies and web services will be mediated semantically.
	- **Selection & Ranking:** Best candidate end point web service will be selected from the ranked list of web service.
	- **Reasoner:** This component will help selection & ranking component for choosing best candidate.

We developed a healthcare domain services hierarchy through HL7 Service Functional Model (Healthcare Services Specification Project (HSSP)). That services classification is used as healthcare service ontology in our registry for discovery purpose Figure 7.

In order to define HL7 service model specification as in Table 1 for healthcare services publication and discovery through Semantic Registry we consider these two types of services:


 **Adaptor:** End user request will be converted into OWL-S to make it adaptable to internal environment. OWL-S annotations of end user request will be provided to discovery component, which will perform the user oriented discovery with the

**Monitoring:** This component will send the request to best selected end point web

 **Invoker:** The response from the end point web service will be given to the user through this component. This component will synchronize all responses from more

**OWL-S ontologies:** Semantic descriptions of web services provided by service

 **Semantic Registry (SR):** SR manages semantic annotations of services provided by service providers in repository and handles discovery process. It also provides OWL-S to WSDL and WSDL to OWL-S translations with the help of OWL-S

**Repository:** OWL-S ontologies' semantic annotations would be stored in repository

**Discovery:** This component will perform keyword based, light weight and heavy

**Semantic matchmaking:** During discovery process similar ontologies and web

**Selection & Ranking:** Best candidate end point web service will be selected from

**Reasoner:** This component will help selection & ranking component for choosing

We developed a healthcare domain services hierarchy through HL7 Service Functional Model (Healthcare Services Specification Project (HSSP)). That services classification is used

In order to define HL7 service model specification as in Table 1 for healthcare services publication and discovery through Semantic Registry we consider these two types of

1. Business Services provide specific business functionality, such as "Patient Appointment", "Lab Order Management" and so on. These are often further

2. Infrastructure (Technical) Services are provided to support the business services and are not specific to healthcare, but are often subject to specific requirements derived from regulation of healthcare information, for example by professional bodies or national

legislatures. Examples include: Authorization, Logging, and Transformation.

as healthcare service ontology in our registry for discovery purpose Figure 7.

subdivided into "Process Services" and "Core Business Services".

help of SR, semantic matchmaking, selection & ranking components

Our proposed framework Figure 6 consists of following components:

**Communication Manager** 

service candidate.

**OWL-S semantic layer** 

provider.

ontologies.

**Discovery Manager:** 

**Services domain ontology** 

weight discovery.

best candidate.

services:

than one end point web services.

to be accessed latter for discovery purpose.

services will be mediated semantically.

the ranked list of web service.

Fig. 7. Healthcare Service Ontology.


Table 1. HL7 Service Specifications.

Semantic Interoperability in E-Health for Improved Healthcare 127

Profile instances to process:Parameter instances. In addition, the following properties relate Profile to expr:Condition and process:Result: hasPrecondition and hasResult as follows:

hasResultVar (a Variable) - A variable scoped to the Result block, bound by the result

withOutput (an OutputBinding of an Output Parameter of the process to a value form)

OWL-S based semantic web service is consists of three modules: Service Profile, Process Model, and Service Grounding. Service profile is used for advertisement purpose and provides data semantics. In paper (Iftikhar et al., 2010) HL7 compliant health service capabilities were provided for publication to Semantic Registry. In order to provide functional semantics in the service description, process model is also defined for functional

The Physician and patients can now query the Semantic Registry by providing service inputs, output, preconditions or effects. As service discovery will use OWL-S service profile and service process model, the service requester have to provide the required service criteria on the basis of service inputs, output, preconditions or effects. Figure 9 explains the working

condition.

inCondition (a Condition)

The working of the system consists of following phases:

 Information/Web services publication from service providers Demand oriented user discovery for healthcare information

description of services. Figure 8 explains information publication.

Fig. 8. Healthcare information publication.

of the demand oriented information provision.

hasEffect (an Effect)

OWL-S Service profile generated through Jena and OWL-API is as follows:

```
<?xml version="1.0"?> 
<rdf:RDF 
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" 
xmlns:xsd="http://www.w3.org/2001/XMLSchema#" 
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" 
xmlns:owl="http://www.w3.org/2002/07/owl#" 
<owl:versionInfo rdf:datatype="http://www.w3.org/2001/XMLSchema#string"> 
</owl:Ontology> 
 <rdfs:Class rdf:ID="Parameter"/> 
<owl:Class rdf:about="#Parameter"> </owl:Class> 
 <rdfs:Class rdf:ID="Output"/><rdfs:subClassOf rdf:resource="#Parameter"/> 
 <rdfs:Class rdf:ID="Input"/> 
 <owl:Class rdf:ID="ServiceCategory"> 
<owl:Class rdf:ID="Result"> 
 <rdfs:label>Result</rdfs:label> 
<owl:ObjectProperty rdf:ID="hasPrecondition"> 
 <rdfs:domain rdf:resource="#Process"/> 
 <rdfs:range rdf:resource="&expr;#Condition"/> 
</owl:ObjectProperty> 
<owl:ObjectProperty rdf:ID="inCondition"> 
 <rdfs:label>inCondition</rdfs:label> 
 <rdfs:domain rdf:resource="#Result"/> 
 <rdfs:range rdf:resource="&expr;#Condition"/> 
</owl:ObjectProperty> 
<owl:ObjectProperty rdf:ID="hasEffect"> 
 <rdfs:label>hasEffect</rdfs:label> 
 <rdfs:domain rdf:resource="#Result"/> 
 <rdfs:range rdf:resource="&expr;#Expression"/> 
</owl:ObjectProperty>
```
Profile is a subclass of OWLs Service Profile defined below. It is used to acknowledge that there may be different ways to profile services that are different from the way we expressed it so far (HSSP). OWL Profile ontology has no classes for modeling IOPE's. Profile instances will be able to define IOPE's using the schema offered by the Process.owl ontology defined by OWL. Additional Classes, needed to specify details of the OWL-S service profile, are also specified for publication and discovery purpose. These are Service Category, Service Parameters and Quality Rating. We have also specified the definition of Profile that provides a definition of the Profile class. Non-Functional Properties are also defined those provide a definition of properties such as name of the service, contact information, quality of the service, and additional information that may help to evaluate the service. We have also specified Functional Properties like IOPE (Input/Output/Precondition/Effects) that help with the specification of what the service provides. The hasParameter property relates Profile instances to process:Parameter instances. In addition, the following properties relate Profile to expr:Condition and process:Result: hasPrecondition and hasResult as follows:


126 Semantics in Action – Applications and Scenarios

OWL-S Service profile generated through Jena and OWL-API is as follows:

<owl:versionInfo rdf:datatype="http://www.w3.org/2001/XMLSchema#string"> -----OWL ontology for all parameters (input, output, are subclasses of parameters) ----

<rdfs:Class rdf:ID="Output"/><rdfs:subClassOf rdf:resource="#Parameter"/>


Profile is a subclass of OWLs Service Profile defined below. It is used to acknowledge that there may be different ways to profile services that are different from the way we expressed it so far (HSSP). OWL Profile ontology has no classes for modeling IOPE's. Profile instances will be able to define IOPE's using the schema offered by the Process.owl ontology defined by OWL. Additional Classes, needed to specify details of the OWL-S service profile, are also specified for publication and discovery purpose. These are Service Category, Service Parameters and Quality Rating. We have also specified the definition of Profile that provides a definition of the Profile class. Non-Functional Properties are also defined those provide a definition of properties such as name of the service, contact information, quality of the service, and additional information that may help to evaluate the service. We have also specified Functional Properties like IOPE (Input/Output/Precondition/Effects) that help with the specification of what the service provides. The hasParameter property relates

xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:xsd="http://www.w3.org/2001/XMLSchema#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"

xmlns:owl="http://www.w3.org/2002/07/owl#"

<owl:Class rdf:about="#Parameter"> </owl:Class>

<owl:ObjectProperty rdf:ID="hasPrecondition"> <rdfs:domain rdf:resource="#Process"/> <rdfs:range rdf:resource="&expr;#Condition"/>

<owl:ObjectProperty rdf:ID="inCondition"> <rdfs:label>inCondition</rdfs:label> <rdfs:domain rdf:resource="#Result"/>

<owl:ObjectProperty rdf:ID="hasEffect"> <rdfs:label>hasEffect</rdfs:label> <rdfs:domain rdf:resource="#Result"/>

<rdfs:range rdf:resource="&expr;#Condition"/>

<rdfs:range rdf:resource="&expr;#Expression"/>

<?xml version="1.0"?>

</owl:Ontology>

<rdfs:Class rdf:ID="Parameter"/>

<owl:Class rdf:ID="ServiceCategory">

<rdfs:Class rdf:ID="Input"/>

<owl:Class rdf:ID="Result"> <rdfs:label>Result</rdfs:label>


</owl:ObjectProperty>

</owl:ObjectProperty>

</owl:ObjectProperty>

<rdf:RDF

The working of the system consists of following phases:


OWL-S based semantic web service is consists of three modules: Service Profile, Process Model, and Service Grounding. Service profile is used for advertisement purpose and provides data semantics. In paper (Iftikhar et al., 2010) HL7 compliant health service capabilities were provided for publication to Semantic Registry. In order to provide functional semantics in the service description, process model is also defined for functional description of services. Figure 8 explains information publication.

Fig. 8. Healthcare information publication.

The Physician and patients can now query the Semantic Registry by providing service inputs, output, preconditions or effects. As service discovery will use OWL-S service profile and service process model, the service requester have to provide the required service criteria on the basis of service inputs, output, preconditions or effects. Figure 9 explains the working of the demand oriented information provision.

Semantic Interoperability in E-Health for Improved Healthcare 129

Precondition: The service should be the member of ControlActProcess class of HL7

Effects: The service provider mentions that the service should receive an

Process Model of FindLabReportResult described below is the functional description of the service where the service functionality is termed as a process. One atomic process is defined for the service Inputs, output, preconditions and result, where result contains condition,

rdf:resource="#resultStatus ">

The service provider publishes service profile and functional service descriptions of health service in OWL-S Profile Repository of Semantic Registry of Figure 2. Service metadata is stored in database for permanent storage purpose. Process model is used in this work to describe the atomic process of service profile. This model will be used further in our future

The service requester can now query the Semantic Registry by providing service inputs, output, preconditions or effects. As service discovery will use OWL-S service profile and

Inputs: The service provider enters findResult and labReport as inputs.

Outputs: The service provider enters resultStatus as output.

output constraints and effects to come true for the result outcome.

acknowledgement of type ApplicationLevelAck.

correctfindResultInfo(labReport,findResult)

</expr:SWRL-Condition>

Service Category: Health\_application\_services

ServiceName : FindLabReportResult

process:AtomicProcess rdf:ID="findResult"> <process:hasInput rdf:resource="#labRepot"/> <process:hasInput rdf:resource="#findResult"/> <process:hasOutput rdf:resource="#resultStatus"/> <process:hasPrecondition isMember(ControlActProcess)/>

artifacts.

 <process:hasResult> <process:Result> <process:inCondition> <expr:SWRL-Condition>

<valueType

 </process:Result> </process:hasResult> </process:AtomicProcess>

**Service discovery** 

 </process:inCondition> <process:withOutput

 rdr:resource="#findResultMsg"> </process:withOutput> <process:hasEffect>

<expr:SWRL-Condition>

 </expr:SWRL-Condition> </process:hasEffect>

work for service invocation process.

ApplicationLevelAck(labReport,findResult)

Fig. 9. Healthcare information provision.

## **2.3.4 Results**

### **A. FindLabReportResult HL7 compliant healthcare web service scenario**

We described a "FindLabReportResult" HL7 compliant healthcare web service scenario where our Semantic Registry allows a service provider to give service descriptions of his health service in terms of service inputs, output, preconditions and effects. Service descriptions published with data as well as functional semantics in the Semantic Registry. Our Semantic Registry also allows a service requester to query the HL7 compliant semantic web health service on the bases of service inputs, output, preconditions or effects. We implemented this scenario in our Semantic Registry.

### **Service publishing**

OWL-S based semantic web service is consists of three modules: Service Profile, Process Model, and Service Grounding. Service profile is used for advertisement purpose and provides data semantics. In this paper HL7 compliant health service capabilities are provided for publication to Semantic Registry. In order to provide functional semantics in the service description, process model is also defined for functional description of services. FindLabReportResult scenario is published in the Semantic Registry by defining service profile and service model.

Service profile of FindLabReportResult described below is the capabilities of web services in terms of its inputs, outputs, preconditions and effects. The service profile is described using OWL protégé APIs (protégé API). This service profile is used for publishing health service in the Semantic Registry.


Fig. 9. Healthcare information provision.

implemented this scenario in our Semantic Registry.

**A. FindLabReportResult HL7 compliant healthcare web service scenario** 

We described a "FindLabReportResult" HL7 compliant healthcare web service scenario where our Semantic Registry allows a service provider to give service descriptions of his health service in terms of service inputs, output, preconditions and effects. Service descriptions published with data as well as functional semantics in the Semantic Registry. Our Semantic Registry also allows a service requester to query the HL7 compliant semantic web health service on the bases of service inputs, output, preconditions or effects. We

OWL-S based semantic web service is consists of three modules: Service Profile, Process Model, and Service Grounding. Service profile is used for advertisement purpose and provides data semantics. In this paper HL7 compliant health service capabilities are provided for publication to Semantic Registry. In order to provide functional semantics in the service description, process model is also defined for functional description of services. FindLabReportResult scenario is published in the Semantic Registry by defining service

Service profile of FindLabReportResult described below is the capabilities of web services in terms of its inputs, outputs, preconditions and effects. The service profile is described using OWL protégé APIs (protégé API). This service profile is used for publishing health service

**2.3.4 Results** 

**Service publishing** 

profile and service model.

in the Semantic Registry.


Process Model of FindLabReportResult described below is the functional description of the service where the service functionality is termed as a process. One atomic process is defined for the service Inputs, output, preconditions and result, where result contains condition, output constraints and effects to come true for the result outcome.

```
process:AtomicProcess rdf:ID="findResult"> 
 <process:hasInput rdf:resource="#labRepot"/> 
 <process:hasInput rdf:resource="#findResult"/> 
 <process:hasOutput rdf:resource="#resultStatus"/> 
 <process:hasPrecondition isMember(ControlActProcess)/> 
 <process:hasResult> 
 <process:Result> 
 <process:inCondition> 
 <expr:SWRL-Condition> 
 correctfindResultInfo(labReport,findResult) 
 </expr:SWRL-Condition> 
 </process:inCondition> 
 <process:withOutput 
rdf:resource="#resultStatus "> 
 <valueType 
 rdr:resource="#findResultMsg"> 
 </process:withOutput> 
 <process:hasEffect> 
 <expr:SWRL-Condition> 
 ApplicationLevelAck(labReport,findResult) 
 </expr:SWRL-Condition> 
 </process:hasEffect> 
 </process:Result> 
 </process:hasResult> 
</process:AtomicProcess>
```
The service provider publishes service profile and functional service descriptions of health service in OWL-S Profile Repository of Semantic Registry of Figure 2. Service metadata is stored in database for permanent storage purpose. Process model is used in this work to describe the atomic process of service profile. This model will be used further in our future work for service invocation process.

#### **Service discovery**

The service requester can now query the Semantic Registry by providing service inputs, output, preconditions or effects. As service discovery will use OWL-S service profile and

Semantic Interoperability in E-Health for Improved Healthcare 131

<binding name="addPatientSoapBinding" type="addPatient:addPatientPortType"> <soap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/>

The detailed OWL-S annotations for patient registration web service are as follows:

regPatient.com/regPatient.wsdl/regPatient\_Grounding#regPatient\_Grounding"/>

<profile:hasInput rdf:resource=" #regPatientPortType \_patientReg\_Name\_IN"/> <profile:hasInput rdf:resource="#regPatientPortType \_ patientReg\_ Location\_IN"/> <profile:hasInput rdf:resource=" #regPatientPortType \_ patientReg\_isCritical\_IN"/> <profile:hasOutput rdf:resource="#regPatientPortType \_ patientReg\_patientId\_OUT"/> <profile:hasOutput rdf:resource="#regPatientPortType \_

<process:AtomicProcess rdf:ID="#regPatientPortType \_ patientReg">

<profile:hasOutput rdf:resource="#regPatientPortType \_ patientReg\_contactNo\_OUT"/>

<process:hasOutput rdf:resource="="#regPatientPortType \_ patientReg\_patientId\_OUT"/> <process:hasOutput rdf:resource="="#regPatientPortType \_ patientReg

<process:hasInput rdf:resource=" #regPatientPortType \_ patientReg \_Name\_IN"/> <process:hasInput rdf:resource="#regPatientPortType \_ patientReg \_Location\_IN"/> <process:hasInput rdf:resource="#regPatientPortType \_ patientReg\_isCritical\_ IN"/>

<service:describedBy rdf:resource="http://www. regPatient.com/regPatient.wsdl/regPatient\_ProcessModel#regPatient\_ProcessModel"/> <service:supports rdf:resource="http://www.

regPatient.com/regPatient.wsdl/regPatient\_Profile#regPatient\_Profile"/>

</message>

</operation> </portType>

</operation> </binding>

1. Service

<service:presents

</service:Service> 2. Service Profile

 </profile:Profile> 3. Process Model

<profile:textDescription/>

patientReg\_hospitalName\_OUT"/>

<process:hasResult> <process:Result>

\_hospitalName\_OUT"/>

rdf:resource="http://www.

<portType name="addPatientPortType">

<service:Service rdf:ID="regPatient\_Service">

<profile:serviceName>patientReg</profile:serviceName>

 <input message="addPatient: addPatient\_Request "/> <output message="addPatient:addPatient\_Response"/>

<operation name="addPatient">

<operation name="addPatient">

service process model, the service requester have to provide the required service criteria on the basis of service inputs, output, preconditions or effects. After service publication in OWL-S Profile Repository as described in Figure 2, service requester can query the Semantic Registry. The Capability Matching Module searches the UDDI registry and OWL-S Profile Repository for the requested service parameters. The Capability Matching Module then executes the matchmaking algorithm with OWL-S Service Profile defined through Jena APIs as described below and with healthcare Service Ontology. Healthcare Service ontology is the upper ontology used by our Semantic Registry for implementing service profile publishing and discovery phases. The searched results and matching levels are provided based on scoring and ranking (degree of match: exact match, plugin match, subsume match, no match). The OWL-S Service Profile used for Service Discovery is as follows:

<profile:Profile> <profile:serviceName>FindLabReportResult</profile:serviceName> <profile:textDescription> An HL7 compliant semantic web healthcare service </profile:textDescription>………

How a Physician and a Patient bound on OWL-S Semantic Registry as shown in Figure 6 is better explained through a scenario. We implemented a scenario where a patient registered in a hospital through our semantic registry. The required steps for publishing and discovery phases included as follows:

	- 2 Web services (WSDL)
	- WSDL 2 OWL-S conversion
	- OWL-S Annotations for these 2 WSDL
	- Service, Profile, Process Model, Grounding
	- Stored in Repositories

### **Discovery phase**


#### **B. Publishing phase implemented scenario**

The web services description in WSDL for Web service name addPatient is as follows:

<message name="addPatient\_Request"> <part name="Name" type="xsd:string"> <part name="location" type="xsd:string"> </message> <message name="addPatient\_Response"> <part name="patientId" type="xs:string">

```
</message> 
 <portType name="addPatientPortType"> 
<operation name="addPatient"> 
 <input message="addPatient: addPatient_Request "/> 
 <output message="addPatient:addPatient_Response"/> 
</operation> 
</portType> 
<binding name="addPatientSoapBinding" type="addPatient:addPatientPortType"> 
<soap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/> 
<operation name="addPatient"> 
</operation> 
</binding>
```
The detailed OWL-S annotations for patient registration web service are as follows:

1. Service

130 Semantics in Action – Applications and Scenarios

service process model, the service requester have to provide the required service criteria on the basis of service inputs, output, preconditions or effects. After service publication in OWL-S Profile Repository as described in Figure 2, service requester can query the Semantic Registry. The Capability Matching Module searches the UDDI registry and OWL-S Profile Repository for the requested service parameters. The Capability Matching Module then executes the matchmaking algorithm with OWL-S Service Profile defined through Jena APIs as described below and with healthcare Service Ontology. Healthcare Service ontology is the upper ontology used by our Semantic Registry for implementing service profile publishing and discovery phases. The searched results and matching levels are provided based on scoring and ranking (degree of match: exact match, plugin match, subsume match, no

How a Physician and a Patient bound on OWL-S Semantic Registry as shown in Figure 6 is better explained through a scenario. We implemented a scenario where a patient registered in a hospital through our semantic registry. The required steps for publishing and discovery

The web services description in WSDL for Web service name addPatient is as follows:

match). The OWL-S Service Profile used for Service Discovery is as follows:

<profile:serviceName>FindLabReportResult</profile:serviceName>

An HL7 compliant semantic web healthcare service

 OWL-S Annotations for these 2 WSDL Service, Profile, Process Model, Grounding

 Map user request, OWL-S Annotations Semantic matchmaking , Selection, Ranking

 WSDL information from UDDI Registry Service invocation and execution Information provided to End user **B. Publishing phase implemented scenario** 

<profile:Profile>

<profile:textDescription>

phases included as follows:

**Publishing phase** 

 **Discovery phase**  User request

</message>

</profile:textDescription>………

 2 Web services (WSDL) WSDL 2 OWL-S conversion

Stored in Repositories

Convert into OWL-S

OWL-S 2 WSDL conversion

<message name="addPatient\_Request"> <part name="Name" type="xsd:string"> <part name="location" type="xsd:string">

<message name="addPatient\_Response"> <part name="patientId" type="xs:string">

```
<service:Service rdf:ID="regPatient_Service"> 
<service:presents 
rdf:resource="http://www. 
regPatient.com/regPatient.wsdl/regPatient_Profile#regPatient_Profile"/> 
<service:describedBy rdf:resource="http://www. 
regPatient.com/regPatient.wsdl/regPatient_ProcessModel#regPatient_ProcessModel"/> 
<service:supports rdf:resource="http://www. 
regPatient.com/regPatient.wsdl/regPatient_Grounding#regPatient_Grounding"/> 
</service:Service>
```
2. Service Profile

<profile:serviceName>patientReg</profile:serviceName>

```
<profile:textDescription/>
```

```
<profile:hasInput rdf:resource=" #regPatientPortType _patientReg_Name_IN"/> 
<profile:hasInput rdf:resource="#regPatientPortType _ patientReg_ Location_IN"/> 
<profile:hasInput rdf:resource=" #regPatientPortType _ patientReg_isCritical_IN"/> 
<profile:hasOutput rdf:resource="#regPatientPortType _ patientReg_patientId_OUT"/> 
<profile:hasOutput rdf:resource="#regPatientPortType _ 
patientReg_hospitalName_OUT"/>
```
<profile:hasOutput rdf:resource="#regPatientPortType \_ patientReg\_contactNo\_OUT"/> </profile:Profile>

3. Process Model

```
<process:AtomicProcess rdf:ID="#regPatientPortType _ patientReg"> 
<process:hasInput rdf:resource=" #regPatientPortType _ patientReg _Name_IN"/> 
<process:hasInput rdf:resource="#regPatientPortType _ patientReg _Location_IN"/> 
<process:hasInput rdf:resource="#regPatientPortType _ patientReg_isCritical_ IN"/> 
<process:hasResult> <process:Result>
```
<process:hasOutput rdf:resource="="#regPatientPortType \_ patientReg\_patientId\_OUT"/> <process:hasOutput rdf:resource="="#regPatientPortType \_ patientReg \_hospitalName\_OUT"/>

Semantic Interoperability in E-Health for Improved Healthcare 133

advertisement have direct parent-child relationship. (3) Enclosure match is applicable when request and advertisement is not direct parent-child but still match in the ontology hierarchy. (4) Unknown matches or Fail when there is no concept in the ontology. Table 2 shows the ranking and degree of matches for the requested service with the advertised services. Similarly the same results achieved for other capabilities of services such as service

The performance analysis of the system is represented in form of time taken for ontology to be loaded into the memory. Analysis also contains the results of the system in terms of number of relevant results produced by our system comparing with the results of the syntax

The Figure 10 shows the performance of the parsing based approach with the protégé-OWL API. We used protégé-OWL API approach for creation of OWL-S profiles. Though it takes more time as compared to parsing based approach but it captures all required semantics for

outputs, preconditions and effects.

Table 2. Ranking of Found Services

based systems without ontologies.

**Profile Name Rank Degree of Match**  FindLabReportResult 5 Exact Match

…………………. ….. ………………...

Fig. 10. Performance Analysis of Publishing OWL-S Profile.

OWL-S Profiles.

Laboratory Service 4 Subsume Match (Parent Match) Health-application-service 4 Subsume Match (Parent Match)

<process:hasOutput rdf:resource="="#regPatientPortType \_ patientReg \_contactNo\_OUT"/> </process:Result> </process:hasResult> </process:AtomicProcess> 4. Grounding <grounding:WsdlGrounding rdf:ID="regPatient\_Grounding"> …………. rdf:ID="WSDLGrounding\_regPatient\_ patientReg "> <grounding:owlsProcess rdf:resource="http://www. regPatient.com/regPatient.wsdl/regPatient\_ProcessModel#regPatientPortType\_ patientReg "/> <xsd:uriReference rdf:value="http://www.regPatient.com/regPatient.wsdl# patientReg"/> <grounding:wsdlInputMessage> // inputs <xsd:uriReference rdf:value="http://www. regPatient.com/regPatient.wsdl#patientReg\_Request"/>…………………………… <xsd:uriReference rdf:value="http://www. regPatient.com/regPatient.wsdl#Name"/> …………………………….. <xsd:uriReference rdf:value="http://www. regPatient.com/regPatient.wsdl#Location"/> ……………….. <xsd:uriReference rdf:value="http://www. regPatient.com/regPatient.wsdl#isCritical"/> // Outputs ……………. <xsd:uriReference rdf:value="http://www. regPatient.com/regPatient.wsdl#hospitalName"/> ………………….. <xsd:uriReference rdf:value="http://www. regPatient.com/regPatient.wsdl#contactNo"/> </grounding:wsdlOutputMessageParts>

## **C. Discovery phase implemented scenario**

	- Inputs: saman, seecs, true
	- Output: patient id, hospital name, contact no

<profile:serviceName> </profile:serviceName>

<profile:textDescription/> <profile:hasInput rdf:resource=" saman"/>

<profile:hasInput rdf:resource=seecs"/>

```
<profile:hasInput rdf:resource=" true"/>
```
<profile:hasOutput rdf:resource="patient Id"/>

```
 <profile:hasOutput rdf:resource="hospital name"/>
```
<profile:hasOutput rdf:resource="contact no"/> </profile:Profile>

### **2.3.5 Discussion**

We ranked the web services based on service level matching and it varies from 5 as the highest and 0 as the lowest. Ranking helps in displaying the best matching results on top of the list. The default lower bound has the value 3 which filters all the results and displays only those services which have Ranking of 3 or above. We also described concept level matching with these possible degrees of match. (1) Exact Match is applicable when concepts mach exactly. (2) Plug-in Match and Subsume Match are applicable when both request and advertisement have direct parent-child relationship. (3) Enclosure match is applicable when request and advertisement is not direct parent-child but still match in the ontology hierarchy. (4) Unknown matches or Fail when there is no concept in the ontology. Table 2 shows the ranking and degree of matches for the requested service with the advertised services. Similarly the same results achieved for other capabilities of services such as service outputs, preconditions and effects.


Table 2. Ranking of Found Services

132 Semantics in Action – Applications and Scenarios

<process:hasOutput rdf:resource="="#regPatientPortType \_ patientReg

<grounding:owlsProcess rdf:resource="http://www. regPatient.com/regPatient.wsdl/regPatient\_ProcessModel#regPatientPortType\_ patientReg

<xsd:uriReference rdf:value="http://www.regPatient.com/regPatient.wsdl# patientReg"/>

<xsd:uriReference rdf:value="http://www.

 <xsd:uriReference rdf:value="http://www. regPatient.com/regPatient.wsdl#Name"/> …………………………….. <xsd:uriReference rdf:value="http://www. regPatient.com/regPatient.wsdl#Location"/> ……………….. <xsd:uriReference rdf:value="http://www.

<xsd:uriReference rdf:value="http://www.

………………….. <xsd:uriReference rdf:value="http://www. regPatient.com/regPatient.wsdl#contactNo"/> </grounding:wsdlOutputMessageParts>

We ranked the web services based on service level matching and it varies from 5 as the highest and 0 as the lowest. Ranking helps in displaying the best matching results on top of the list. The default lower bound has the value 3 which filters all the results and displays only those services which have Ranking of 3 or above. We also described concept level matching with these possible degrees of match. (1) Exact Match is applicable when concepts mach exactly. (2) Plug-in Match and Subsume Match are applicable when both request and

regPatient.com/regPatient.wsdl#patientReg\_Request"/>……………………………

regPatient.com/regPatient.wsdl#isCritical"/> // Outputs …………….

<profile:textDescription/> <profile:hasInput rdf:resource=" saman"/>

<profile:hasOutput rdf:resource="contact no"/> </profile:Profile>

\_contactNo\_OUT"/>

4. Grounding

………….

"/>

</process:AtomicProcess>

</process:Result> </process:hasResult>

<grounding:wsdlInputMessage> // inputs

<grounding:WsdlGrounding rdf:ID="regPatient\_Grounding">

rdf:ID="WSDLGrounding\_regPatient\_ patientReg ">

regPatient.com/regPatient.wsdl#hospitalName"/>

Output: patient id, hospital name, contact no

**C. Discovery phase implemented scenario**  User request (goal oriented request) Inputs: saman, seecs, true

<profile:serviceName> </profile:serviceName>

 <profile:hasOutput rdf:resource="patient Id"/> <profile:hasOutput rdf:resource="hospital name"/>

 <profile:hasInput rdf:resource=seecs"/> <profile:hasInput rdf:resource=" true"/>

Discovered OWL-S Profile

**2.3.5 Discussion** 

The performance analysis of the system is represented in form of time taken for ontology to be loaded into the memory. Analysis also contains the results of the system in terms of number of relevant results produced by our system comparing with the results of the syntax based systems without ontologies.

Fig. 10. Performance Analysis of Publishing OWL-S Profile.

The Figure 10 shows the performance of the parsing based approach with the protégé-OWL API. We used protégé-OWL API approach for creation of OWL-S profiles. Though it takes more time as compared to parsing based approach but it captures all required semantics for OWL-S Profiles.

Semantic Interoperability in E-Health for Improved Healthcare 135

through semantic interoperability. The SEMR system will develop a semantic based SaaS service model on top of cloud for healthcare domain to resolve the above mentioned problems. This will result in efficient healthcare provision to patients in a timely manner

A framework for semantic registry based on OWL-S - an ontology web language for web services is used for semantic composition. We implemented a scenario where we bound a physician and a patient for registering to a hospital. We also provide service advertisement publication and discovery of service profiles and process model of HL7 compliant healthcare web services. The service description capabilities of Semantic Registry incorporated functional semantics where we also defined preconditions and effects. The service discovery is also more efficient as matchmaking algorithm is also considering service preconditions and effects for fulfilling the user requests. The whole working of the service publication and discovery is described through FindLabReportResult HL7 compliant healthcare semantic web service scenario. The results are evaluated through implementing the UDDI Publish APIs and Inquire APIs as these are without semantics and do not provide discovery on the basis of preconditions

This work is part of Health Life Horizon project initiated at NUST SEECS (http://hl7.niit.edu.pk/index.htm) and is funded by National ICT R&D Funds, Pakistan,

Dogac A., Laleci G., Kirbas S., Kabak Y., Sinir S., Yildiz A., and Gurcan, Y.: "Artemis:

Dogac, Asuman., Gokce, B. Laleci., Kabak, Yildiray., Unal, Seda., Beale, Thomas., Heard,

Papazoglou, M.P. and W-J. van den Heuvel, "Service-Oriented Architectures: Approaches,

Beeler, George W., Stan Huff, Wesley Rishel, Abdul-Malik Shakir, Mead Walker, Charlie

CASCOM: Nadine Fröhlich, Heikki Helin, Heimo Laamanen, Thorsten Möller, Thomas

December 1999, Copyright 1999 by Health Level Seven, Inc.

deploying semantically enriched web services in the healthcare domain". *Proceedings of Information systems journal (Elsevier),* 2006, 31, (4–5), pp. 321–

Sam., Elkin, Peter., Najmi, Farrukh., Mattocks, Carl., Webber, David. (2008). Exploiting ebXML Registry Semantic Constructs for Handling Archetype Metadata in Healthcare Informatics. *Proceedings of International Journal of Metadata, Semantics* 

Technologies and Research Issues. *Proceedings of VLDB J.*, vol. 16, no. 3, 2007, pp.

Mead, Gunther Schadow. (1999) Message Development Framework. Version 3.3,

Schabetsberger, Heiko Schuldt, and Christian Stark. (2007). Semantic Service Co-Ordination for Emergency Assistance in Mobile e-Health Environments. *Proceedings* 

(Iftikhar et al., 2011).

and effects (Iftikhar et al., 2011).

**4. Acknowledgment** 

http://ictrdf.org.pk.

339

389-415.

*and Ontologies* (IJMSO-08).

**5. References** 

Fig. 11. Performance Analysis of Service Discovery.

For service discovery analysis the system was tested on 100 service profiles. We tested our semantic based approach with the syntax based approach of UDDI. The figure shows the average result of different queries executed on both the systems under same conditions. The relevant result of the syntax based approach is much lower than that of the semantic based approach. With syntax based approach 95 service profiles retrieved out of which 50 were relevant. With our semantic based approach 70 profiles retrieved out of which 62 were relevant. The result is based on the exact matches of IOPE of service capabilities for both systems as shown in Figure 11.

Our semantic based registry or OWL-S framework that provides services and metadata to manage healthcare information and processes in a consistent way that would be compliant with emerging international standards. Our Solution would provide collaboration and give hospitals and clinics the ability to share healthcare semantic information. Semantic web technologies can provide extensible, flexible and efficient information. Semantics can provide interoperable, automated and seamlessly meaningful communication in healthcare domain.

## **3. Conclusion**

This chapter is based on discussion of two major problems of healthcare industry: interoperability and integration. We presented two designing architecture to handle the two common problems and manage large scale medical data, patient records and the technological infrastructure. Healthcare domain is facing challenges of information sharing, interoperability and efficient discovery. These challenges can be handled with the help of semantic web technologies, service oriented architecture and cloud computing by providing automated semantic web health services. This will lead to extensible and flexible data storage, retrieval and sharing among physicians and patients and efficient discovery of information related to diseases and clinical processes.

Cloud computing will change the rules of healthcare service provision globally with adding values to existing platforms as SaaS and will automate processes and knowledge networks through semantic interoperability. The SEMR system will develop a semantic based SaaS service model on top of cloud for healthcare domain to resolve the above mentioned problems. This will result in efficient healthcare provision to patients in a timely manner (Iftikhar et al., 2011).

A framework for semantic registry based on OWL-S - an ontology web language for web services is used for semantic composition. We implemented a scenario where we bound a physician and a patient for registering to a hospital. We also provide service advertisement publication and discovery of service profiles and process model of HL7 compliant healthcare web services. The service description capabilities of Semantic Registry incorporated functional semantics where we also defined preconditions and effects. The service discovery is also more efficient as matchmaking algorithm is also considering service preconditions and effects for fulfilling the user requests. The whole working of the service publication and discovery is described through FindLabReportResult HL7 compliant healthcare semantic web service scenario. The results are evaluated through implementing the UDDI Publish APIs and Inquire APIs as these are without semantics and do not provide discovery on the basis of preconditions and effects (Iftikhar et al., 2011).

## **4. Acknowledgment**

This work is part of Health Life Horizon project initiated at NUST SEECS (http://hl7.niit.edu.pk/index.htm) and is funded by National ICT R&D Funds, Pakistan, http://ictrdf.org.pk.

## **5. References**

134 Semantics in Action – Applications and Scenarios

For service discovery analysis the system was tested on 100 service profiles. We tested our semantic based approach with the syntax based approach of UDDI. The figure shows the average result of different queries executed on both the systems under same conditions. The relevant result of the syntax based approach is much lower than that of the semantic based approach. With syntax based approach 95 service profiles retrieved out of which 50 were relevant. With our semantic based approach 70 profiles retrieved out of which 62 were relevant. The result is based on the exact matches of IOPE of service capabilities for both

Our semantic based registry or OWL-S framework that provides services and metadata to manage healthcare information and processes in a consistent way that would be compliant with emerging international standards. Our Solution would provide collaboration and give hospitals and clinics the ability to share healthcare semantic information. Semantic web technologies can provide extensible, flexible and efficient information. Semantics can provide interoperable, automated and seamlessly meaningful communication in healthcare

This chapter is based on discussion of two major problems of healthcare industry: interoperability and integration. We presented two designing architecture to handle the two common problems and manage large scale medical data, patient records and the technological infrastructure. Healthcare domain is facing challenges of information sharing, interoperability and efficient discovery. These challenges can be handled with the help of semantic web technologies, service oriented architecture and cloud computing by providing automated semantic web health services. This will lead to extensible and flexible data storage, retrieval and sharing among physicians and patients and efficient discovery of

Cloud computing will change the rules of healthcare service provision globally with adding values to existing platforms as SaaS and will automate processes and knowledge networks

**Irrelevant Results Relevant Results**

**Semantic Based Syntax Based**

systems as shown in Figure 11.

domain.

**3. Conclusion** 

Fig. 11. Performance Analysis of Service Discovery.

information related to diseases and clinical processes.


Semantic Interoperability in E-Health for Improved Healthcare 137

CCHIT. http://en.wikipedia.org/wiki/Certification\_Commission\_for\_Healthcare\_

Health Services Specifications Project (HSSP) http://hssp.wikispaces.com/ [(Last Visited

HIPPA, http://en.wikipedia.org/wiki/Health\_Insurance\_Portabity\_and\_Accountability\_

http://www.himss.org/content/files/selectingemr\_flyer2.pdf [(Last Visited

Healthcare Services Specification Project (HSSP) HL7 Services Oriented Architecture SIG;

Neotool "The HL7 Evolution, Comparing HL7 Version 2 to Version 3, Including a History of

OpenEMR Virtual Appliance, http://en.wikipedia.org/wiki/OpenMRS/ [(Last Visited

SequelMed EMR.

http://www.sequelmed.com/Products/electronic\_medical\_records.aspx/. [(Last

Object Management Group (OMG), http://www.omg.org/ [(Last Visited march 2011)] OpenEMR, http://en.wikipedia.org/wiki/OpenEMR/ [(Last Visited march 2011)]

OpenMRS, http://en.wikipedia.org/wiki/OpenMRS. [(Last Visited march 2011)]

http://microarray.wordpress.com/2009/01/24/saas-and-healthcare-internet-

SmartCare. http://en.wikipedia.org/wiki/SmartCare/ [(Last Visited march 2011)]

http://msdn.microsoft.com/en-us/library/aa905332.aspx#enterprisertw\_topic4/

Understanding HIPPA: Health Insurance Portability And Accountability Act. http://www.googobits.com/articles/p0-2899-understanding-hippa-healthinsurance-portability-and-accountability-act.html. [(Last Visited march 2011)]

Information\_Technology/. [(Last Visited march 2011)]

Cocoon. 2011. Available from: http://www.cocoon-health.com Health Level Seven, http://www.hl7.org/ [(Last Visited March 2011)]

HL7 V3 Interoperability Standard, Normative Edition 2009

HL7 V3 Interoperability Standard, Normative Edition 2009.

2011. Available from: http://hssp.wikispaces.com.

Protégé API. Jena, 2011. Available from: http://jena.sourceforge.net/.

business-models/ [(Last Visited march 2011)]

SNOMED Clinical Terms User Guide January 2007 Release

Web Services Modeling Ontology. http://www.wsmo.org/

Xchart. http://www.openhealth.org/XChart/ [(Last Visited march 2011)]

Act/ [(Last Visited march 2011)]

HIMSS "Selecting the Right EMR Vendor,"

LOINC. 2011. Accessed from http://loinc.org/

march 2011)]

March 2011)]

Version 2"

march 2011)]

Software as a Service (SaaS)

WSMO. http://www.wsmo.org/

Visited march 2011)]

SaaS and Healthcare Internet Business Models

[(Last Visited march 2011)]

CCHIT EMR. http://www.mtbc.com/cchit-emr.aspx/ [(Last Visited march 2011)]

Health Life Horizon (HLH), http://hl7.seecs.nust.edu.pk/ [(Last Visited march 2011)]

*of Workshop on Semantic Web in Ubiquitous Healthcare, collocated with the 6th International Semantic Web Conference* (ISWC2007), 2007.


World Wide Web Sites and Other Electronic Sources


Cesar Cáceresc, Alberto Fernández, Sascha Ossowski, Carlos Matteo Vasirani. (2006). Agent-

Clement, L. Hately, A. Riegen, C. and Rogers, T. "UDDI Version 3.0.2". OASIS, 2004. UDDI Spec Technical Committee Draft. Available at http://www.uddi.org. Cocoon: Emanuele Della Valle, and Dario Cerizza. "The mediators centric approach to

Ginsburg, Mark. Interface Considerations in a Pediatric EMR. *Proceedings of 12th Americans Conference on Information Systems,* Acapulco, Mexico, August 4-6, 2006. Iftikhar, Saman., Khan, Wajahat Ali., Hussain, Maqbool., Afzal, Muhammad., Ahmad,

Iftikhar, Saman., Ahmad, Farooq., Fatima, Kiran. (2011). A framework based on OWL-S for

Iftikhar, Saman., Nawaz, Falak., Ahmad, Farooq., Fatima, Kiran. (2011). Introducing

Nawaz, F., Pasha, M., Ahmad, F., Suguri, H., (2007). Pushing Semantic Web Service Profiles

N. Srinivasan, M. Paolucci, K. Sycara. (2004). An Efficient Algorithm for OWL-S based

Paolucci, M., T. Kawamura, T. Payne, K. Sycara, "Semantic Matching of Web Services

Willmott, S., Willmott, H. Ronsdorf, K. Krempels. (2005)Publish and search versus registries for semantic web service discovery. *Proceedings of Web Intelligence*, 2005.

Benefits of SOA, Website 2010, http://www.devshed.com/c/a/Web-

Benefits of Semantic SOA, http://members.sti2.at/~jacekk/education.sti2.org/slides/1-

Clear-health-improving practice management. http://www.clear-health.com/ [(Last

*Emerging Technologies,* Islamabad, Pakistan, September, 2011.

*Services and Web Process Composition*, SWSWPC 2004 96-110.

*Semantic Web*, p.333-347, June 09-12, 2002.

graham-soa.pdf [(Last Visited march 2011)]

World Wide Web Sites and Other Electronic Sources

*International Semantic Web Conference* (ISWC2007), 2007.

IEEE Intelligent Systems, vol. 21, no. 6, 2006, pp. 11-20.

*Interoperability Conference,* March, 2011.

2005, pp. 35–50.

2010

2007.

march 2011)]

Visited march 2011)]

*of Workshop on Semantic Web in Ubiquitous Healthcare, collocated with the 6th* 

Based Semantic Service Discovery for Healthcare: An Organizational Approach",

automatic Web Service discovery of Glue". *CEUR Workshop Proceedings*, vol. 168,

Farooq. (2011). Design of Semantic Electronic Medical Record (SEMR) system as SaaS service model for Efficient Healthcare. *Proceedings of International Health* 

health care information provision, *Proceedings of 7th IEEE International Conference on* 

Semantics in DHTs for Grid Services in a Semantic Registry. *Proceedings of 6th IEEE International Conference on Emerging Technologies,* pp 382 - 387, 18-19 Oct.

to Subscribers for Efficient Service Discovery. *Proceedings of the 3rd International Conference on Semantics, Knowledge and Grid* (SKG '07) Xi'an, China, October

Semantic Search in UDDI. *Proceedings of first International Workshop on Semantic Web* 

Capabilities. *Proceedings of the First International Semantic Web Conference on The* 

Services/Introduction-to-Service-Oriented-Architecture-SOA/2/ [(Last Visited

CCHIT. http://en.wikipedia.org/wiki/Certification\_Commission\_for\_Healthcare\_ Information\_Technology/. [(Last Visited march 2011)] CCHIT EMR. http://www.mtbc.com/cchit-emr.aspx/ [(Last Visited march 2011)] Cocoon. 2011. Available from: http://www.cocoon-health.com Health Level Seven, http://www.hl7.org/ [(Last Visited March 2011)] Health Life Horizon (HLH), http://hl7.seecs.nust.edu.pk/ [(Last Visited march 2011)] HL7 V3 Interoperability Standard, Normative Edition 2009 Health Services Specifications Project (HSSP) http://hssp.wikispaces.com/ [(Last Visited march 2011)] HIPPA, http://en.wikipedia.org/wiki/Health\_Insurance\_Portabity\_and\_Accountability\_ Act/ [(Last Visited march 2011)] HL7 V3 Interoperability Standard, Normative Edition 2009. HIMSS "Selecting the Right EMR Vendor," http://www.himss.org/content/files/selectingemr\_flyer2.pdf [(Last Visited March 2011)] Healthcare Services Specification Project (HSSP) HL7 Services Oriented Architecture SIG; 2011. Available from: http://hssp.wikispaces.com. LOINC. 2011. Accessed from http://loinc.org/ Neotool "The HL7 Evolution, Comparing HL7 Version 2 to Version 3, Including a History of Version 2" Object Management Group (OMG), http://www.omg.org/ [(Last Visited march 2011)] OpenEMR, http://en.wikipedia.org/wiki/OpenEMR/ [(Last Visited march 2011)] OpenEMR Virtual Appliance, http://en.wikipedia.org/wiki/OpenMRS/ [(Last Visited march 2011)] OpenMRS, http://en.wikipedia.org/wiki/OpenMRS. [(Last Visited march 2011)] Protégé API. Jena, 2011. Available from: http://jena.sourceforge.net/. SequelMed EMR. http://www.sequelmed.com/Products/electronic\_medical\_records.aspx/. [(Last Visited march 2011)] SaaS and Healthcare Internet Business Models http://microarray.wordpress.com/2009/01/24/saas-and-healthcare-internetbusiness-models/ [(Last Visited march 2011)] Software as a Service (SaaS) http://msdn.microsoft.com/en-us/library/aa905332.aspx#enterprisertw\_topic4/ [(Last Visited march 2011)] SmartCare. http://en.wikipedia.org/wiki/SmartCare/ [(Last Visited march 2011)] SNOMED Clinical Terms User Guide January 2007 Release Understanding HIPPA: Health Insurance Portability And Accountability Act. http://www.googobits.com/articles/p0-2899-understanding-hippa-healthinsurance-portability-and-accountability-act.html. [(Last Visited march 2011)] WSMO. http://www.wsmo.org/ Web Services Modeling Ontology. http://www.wsmo.org/ Xchart. http://www.openhealth.org/XChart/ [(Last Visited march 2011)]

**6** 

*China* 

Xueming Qian

**Semantic Based Sport Video Browsing** 

*School of Electronic and Information Engineering, Xi'an Jiaotong University, Xi'an,* 

In this chapter, we focus our attention on semantic based sport video highlights detection and semantic based sport video browsing. Users are more interested in sport video highlights than the normal kicks. They site before their TV sets to enjoy the exciting moments that their favorite teams shooting goals. As audiences enjoy the highlight of the video content, usually they have patient to the inserted video Advertisements. For the web based video services, the click rates of sport video highlights are far more than the whole video sequences [56]. Detecting highlights effectively is not only of interest to users/audiences (they are willing to view the highlights) but also of interest to commercial companies. They are interested in inserting their advertisement around the highlights to get more attention from consumers and expand the influences of their products/services. In many online video services websites, the sport video highlights are manually labeled which is very time-consuming and expensive. Thus automatic sport video highlight detection is very urgent [1]-[29]. It is a fundamental step for semantic based video browsing [1,7,8]. However, due to the semantic gaps between computer and human beings, highlights

Two types of approaches have been widely utilized in sport video highlight detection, as shown in Fig.1. The first type of highlights detection approaches are carried out by mapping from low-level features directly or using model-based approaches. In the second type of approaches a mid-level semantic layer is introduced between the low-level features and high-level semantics. Highlights are detected from mid-level semantics rather than from low-level features directly. Thus it is robust against the divergences of low-level features. The second type of sport video highlight detection approaches is more effective than the first type of approaches. Based on the detected high-level semantics, semantics based video

**1. Introduction** 

detection is not a trivial.

browsing can be performed.

Fig. 1. Block diagram of semantic based sport video browsing.


## **Semantic Based Sport Video Browsing**

## Xueming Qian

*School of Electronic and Information Engineering, Xi'an Jiaotong University, Xi'an, China* 

### **1. Introduction**

138 Semantics in Action – Applications and Scenarios

ZIMBRA- open source email and collaboration. http://www.zimbra.com/ [(Last Visited

Erl, T., (2005). "Service-Oriented Architecture (SOA). Concepts, Technology, and Design".

Kourtesis D. and Paraskakis I. (2008). Combining SAWSDL, OWL-DL and UDDI for

Semantically Enhanced Web Service Discovery. In Bechhofer S. et al.(Eds.): ESWC 2008, Lecture Notes in Computer Science 5021, Springer-Verlag Berlin Heidelberg

march 2011)]

Prentice Hall PTR.

2008, pp. 614-628.

In this chapter, we focus our attention on semantic based sport video highlights detection and semantic based sport video browsing. Users are more interested in sport video highlights than the normal kicks. They site before their TV sets to enjoy the exciting moments that their favorite teams shooting goals. As audiences enjoy the highlight of the video content, usually they have patient to the inserted video Advertisements. For the web based video services, the click rates of sport video highlights are far more than the whole video sequences [56]. Detecting highlights effectively is not only of interest to users/audiences (they are willing to view the highlights) but also of interest to commercial companies. They are interested in inserting their advertisement around the highlights to get more attention from consumers and expand the influences of their products/services. In many online video services websites, the sport video highlights are manually labeled which is very time-consuming and expensive. Thus automatic sport video highlight detection is very urgent [1]-[29]. It is a fundamental step for semantic based video browsing [1,7,8]. However, due to the semantic gaps between computer and human beings, highlights detection is not a trivial.

Two types of approaches have been widely utilized in sport video highlight detection, as shown in Fig.1. The first type of highlights detection approaches are carried out by mapping from low-level features directly or using model-based approaches. In the second type of approaches a mid-level semantic layer is introduced between the low-level features and high-level semantics. Highlights are detected from mid-level semantics rather than from low-level features directly. Thus it is robust against the divergences of low-level features. The second type of sport video highlight detection approaches is more effective than the first type of approaches. Based on the detected high-level semantics, semantics based video browsing can be performed.

Fig. 1. Block diagram of semantic based sport video browsing.

Semantic Based Sport Video Browsing 141

metric [36] are then utilized to classify each pixel of current frame into dominant color or non-dominant color. 3) Adaptive dominant color refinement. For the pixels labeled as initial dominant color, their accumulative histograms of HSI are reconstructed again and the same operations as in initial dominant color modeling step are utilized to obtain the refined dominant color *HSI* 0 00 , , of the current frame. Let's give a brief overview of this approach.

The distance of a pixel located at coordinate (*i*, *j*) with color *H i*( , ), ( , ), ( , ) *j S i j I i j* to the

where *D ij* (, ) *int* and *D ij* (, ) *chr* denote the distance in intensity and chrominance components

2 2 0 0 ( , ) ( , ) 2 ( , )cos( ( , )) *D i chr j S i j S SSi j*

0 (, ) ( , ) 180 (, ) 360 ( , ) *H i j H if H i j H i j H i j H otherwise*

0 (, ) (, ) 1 (, )

where *Dth* is learned from several soccer video clips, *DCRM*(*i*, *j*)=1 indicates that the pixel (*i*, *j*) belongs to the field region. The dominant color ratio (*DCR*) of a frame is obtained as

1 1

 

where *H* and *W* are the height and width of a frame. Fig.2 shows the corresponding dominant color region extraction results. The dominant color region of Fig.2(a) is shown in Fig.2 (b) and the field region is extracted as shown in Fig. 2 (c). The dominant color

Based on *DCRM*, both global or grid based dominant color distributions can be utilized. In [56], the *DCRM* is parsed into 8 equal-sized regions in both vertical and horizontal direction. By calculating the dominant color pixel ratio of each region, a 16-bin dominant color distribution vector is constructed. In addition to the dominant color distribution, a 255

*i j*

*DCR*

distribution is represented by the dominant color projection vectors of *DCRM*.

*H W*

*Di j D DCRM i j Di j D* 

2 2 (, ) (, ) (, ) *D i chr j D i j D i j int* (1)

<sup>0</sup> *D i j Ii j I int*(, ) (, ) (2)

*i j* (3)

(5)

(4)

(6)

*th th*

(, )

*DCRM i j*

*H W*

0 0

dominant color *HSI* 0 00 , , is measured by the cylindrical metric [36] as follows

**2.1.2 Coarse to fine dominant color extraction approach** 

Similar to [36], the dominant color region map *DCRM*(*i*, *j*) as follows:

respectively.

follows:

The conventional video content browsing is based on the summarization of the whole video content. However viewers may be interested to the highlights of sport video. In this chapter, a semantic based sport video browsing approach is introduced. It is a novel book style based approach. Sport video summarization structure is similar to the table of content (ToC) of books. The semantic based video browsing approach has several advantages. Firstly, it is a hierarchical video summarization framework. Secondly, it provides seamless connections with sport video high-level semantics extraction. Thirdly, it is very convenient for users to find their interested content in a large scale database.

In this chapter, firstly learning based soccer video highlight detection approaches are expressed and then semantic based soccer video browsing approach is introduced. The main content of this chapter is organized as follows. Effective low level features representation for sports video is given in Section 2. Middle level semantic classification approaches for soccer video is provided in Section 3. High level semantic detection approaches are introduced in Section 4. Semantic based video browsing is given in Section 5 and conclusions are drawn in Section 6.

## **2. Low-level feature representation**

In this section, several effective low-level visual features for sport video content analysis are introduced. The features are dominant color, motion, texture and overlaid text information. From the low-level visual feature detection results, mid-level semantics classification and high-level events can be determined by using either direct mapping or statistical learning based approaches.

## **2.1 Dominant color**

Dominant color is an effective feature for soccer video analysis. The dominant color is essentially the color of the grass field. The distribution and percentage of grass field region offer significant cues for mid-level semantics categorization and highlights detection.

## **2.1.1 Related work on dominant color extraction**

In [14], Duan et al. classified video shots into eight categories by fusing the global motion pattern, color, texture, shape, and shot length information in a supervised learning framework. Motion and dominant color information of a video shot are fused by multi-layer hidden Markov models (HMM) to determine its categorization [6]. Dominant color ratio and the dominant color projection histogram [12] are utilized for semantic soccer video shot classification. The dominant color extraction approach consists of two stages: dominant color modeling and adaptive dominant color refinement [12]. The dominant color modeling can be viewed as determining a coarse dominant color, which can be utilized for various soccer videos. While the dominant color refinement can be viewed as getting more accurate dominant color for a specific video.

In [56], dominant color detection scheme consists of three steps: 1) Initial dominant color modeling in HSI color space. The accumulative histograms of the HSI components are constructed from training frames of global views randomly selected form wide range of soccer videos and the initial dominant color are determined from the color histogram [36]. 2) Initial dominant color region determination. The initial dominant color and the cylindrical

The conventional video content browsing is based on the summarization of the whole video content. However viewers may be interested to the highlights of sport video. In this chapter, a semantic based sport video browsing approach is introduced. It is a novel book style based approach. Sport video summarization structure is similar to the table of content (ToC) of books. The semantic based video browsing approach has several advantages. Firstly, it is a hierarchical video summarization framework. Secondly, it provides seamless connections with sport video high-level semantics extraction. Thirdly, it is very convenient for users to

In this chapter, firstly learning based soccer video highlight detection approaches are expressed and then semantic based soccer video browsing approach is introduced. The main content of this chapter is organized as follows. Effective low level features representation for sports video is given in Section 2. Middle level semantic classification approaches for soccer video is provided in Section 3. High level semantic detection approaches are introduced in Section 4. Semantic based video browsing is given in Section 5 and conclusions are drawn in

In this section, several effective low-level visual features for sport video content analysis are introduced. The features are dominant color, motion, texture and overlaid text information. From the low-level visual feature detection results, mid-level semantics classification and high-level events can be determined by using either direct mapping or statistical learning

Dominant color is an effective feature for soccer video analysis. The dominant color is essentially the color of the grass field. The distribution and percentage of grass field region

In [14], Duan et al. classified video shots into eight categories by fusing the global motion pattern, color, texture, shape, and shot length information in a supervised learning framework. Motion and dominant color information of a video shot are fused by multi-layer hidden Markov models (HMM) to determine its categorization [6]. Dominant color ratio and the dominant color projection histogram [12] are utilized for semantic soccer video shot classification. The dominant color extraction approach consists of two stages: dominant color modeling and adaptive dominant color refinement [12]. The dominant color modeling can be viewed as determining a coarse dominant color, which can be utilized for various soccer videos. While the dominant color refinement can be viewed as getting more accurate

In [56], dominant color detection scheme consists of three steps: 1) Initial dominant color modeling in HSI color space. The accumulative histograms of the HSI components are constructed from training frames of global views randomly selected form wide range of soccer videos and the initial dominant color are determined from the color histogram [36]. 2) Initial dominant color region determination. The initial dominant color and the cylindrical

offer significant cues for mid-level semantics categorization and highlights detection.

find their interested content in a large scale database.

**2. Low-level feature representation** 

**2.1.1 Related work on dominant color extraction** 

dominant color for a specific video.

Section 6.

based approaches.

**2.1 Dominant color** 

metric [36] are then utilized to classify each pixel of current frame into dominant color or non-dominant color. 3) Adaptive dominant color refinement. For the pixels labeled as initial dominant color, their accumulative histograms of HSI are reconstructed again and the same operations as in initial dominant color modeling step are utilized to obtain the refined dominant color *HSI* 0 00 , , of the current frame. Let's give a brief overview of this approach.

#### **2.1.2 Coarse to fine dominant color extraction approach**

The distance of a pixel located at coordinate (*i*, *j*) with color *H i*( , ), ( , ), ( , ) *j S i j I i j* to the dominant color *HSI* 0 00 , , is measured by the cylindrical metric [36] as follows

$$D(i,j) = \sqrt{D\_{int}(i,j)^2 + D\_{clr}(i,j)^2} \tag{1}$$

where *D ij* (, ) *int* and *D ij* (, ) *chr* denote the distance in intensity and chrominance components respectively.

$$D\_{\rm int}(i, j) = I(i, j) - I\_0 \tag{2}$$

$$D\_{\rm dir}(i,j) = \sqrt{S(i,j)^2 + S\_0^2 - 2S\_0 S(i,j) \cos(\theta(i,j))}\tag{3}$$

$$\theta(i,j) = \begin{cases} \left| H(i,j) - H\_0 \right| & \text{if } \left| H(i,j) - H\_0 \right| \le 180^\circ \\ 360^\circ - \left| H(i,j) - H\_0 \right| & \text{otherwise} \end{cases} \tag{4}$$

Similar to [36], the dominant color region map *DCRM*(*i*, *j*) as follows:

$$DCRM(i,j) = \begin{cases} 0 & D(i,j) > D\_{\text{th}} \\ 1 & D(i,j) \le D\_{\text{th}} \end{cases} \tag{5}$$

where *Dth* is learned from several soccer video clips, *DCRM*(*i*, *j*)=1 indicates that the pixel (*i*, *j*) belongs to the field region. The dominant color ratio (*DCR*) of a frame is obtained as follows:

$$\begin{aligned} \sum\_{i=1}^{H} \sum\_{j=1}^{W} DCRM(i,j) \\ DCR = \frac{i \times W \times W}{H \times W} \end{aligned} \tag{6}$$

where *H* and *W* are the height and width of a frame. Fig.2 shows the corresponding dominant color region extraction results. The dominant color region of Fig.2(a) is shown in Fig.2 (b) and the field region is extracted as shown in Fig. 2 (c). The dominant color distribution is represented by the dominant color projection vectors of *DCRM*.

Based on *DCRM*, both global or grid based dominant color distributions can be utilized. In [56], the *DCRM* is parsed into 8 equal-sized regions in both vertical and horizontal direction. By calculating the dominant color pixel ratio of each region, a 16-bin dominant color distribution vector is constructed. In addition to the dominant color distribution, a 255

Semantic Based Sport Video Browsing 143

text information is incorporated with audio-visual features to improve highlights detection

In soccer video the title texts usually appeared in the bottom-center of the image. The icon, scoreboard and time tables are appeared in the top-left and top-right parts of an image respectively. This type text always appeared during the whole video sequence, for example, the extracted local text lines of Fig. 4(a), as shown in Fig.4(c) are appeared in the whole video. However, the title text as appeared in Fig.4(b), as shown in Fig. 4(d) usually existed in a limited time range. The title text is usually provides more information on its corresponding highlight event inference than a long term text. They are purposively added during soccer video editing, and they are aimed at providing the audience some indicative

Fig. 3. Overlaid texts (including bulletin texts and title texts) and replays of a soccer video.

(a) (b)

The corresponding text detection approaches can be classified into following 3 types. The first is the connection characteristics of video texts [63], [64]. The text detection methods based on this characteristic assume that text regions have uniform colors and satisfy certain constraints on size, shape, and spatial layout. The second is the texture alike characteristic of the text regions [65, 33, 66, 67]. The text detection methods based on texture information usually assume that the text regions have special texture patterns. And the third is the edge density information [34], [68]. These methods make full use of the fact that the edge densities of background are comparatively sparser than those of the text regions [69], [70]. Usually the corner point number in the text region is larger than that in the background

(d)

(c)

information about the video content, such as a card or a goal [56], [62].

Fig. 4. The types of texts appeared in soccer video frames.

regions.

**2.3.1 Related works on video text detection and tracking** 

performances [28,29].

dimensional block wise color moment generated from 5-by-5 grids of the images is also utilized.

 (a) Original image (b) dominant color region (c) field and non-field region Fig. 2. Dominant color region, field region and field lines detection results.

## **2.2 Texture feature**

Texture features are insensitive to lighting, rotation, and other conditions. Two kinds of texture features are utilized for coarse shot classification in [56]. The first is hierarchical wavelet packet texture descriptor (HWVP) [47]. The second is 24 dimensional histogram of oriented gradient (HOG) [46]. HWVP descriptor is a 210 dimensional feature which is extracted under local partitioning pattern Local5 (the image is partitioned into 2×2 grids and a centralized grid), by setting the wavelet packet basis to be db2, with hierarchical wavelet packet transform level 2.

## **2.3 Title text detection, localization, and tracking**

Title texts provide valuable information for event detection in soccer video content analysis. Usually the overlaid text about goal, foul, yellow/red card are followed the corresponding highlight events. In Fig.3, overlaid texts (including bulletin texts and title texts) and replays of a soccer video are plotted. There are 20 replays and 7 overlaid texts. The first text is a bulletin text that shows the players' name of the two teams. The second text shows the initial score of two teams. The 3rd and the 6th texts show the names of the players who got goals. The 4th and 7th texts show the updated scores just after a goal. Moreover, the 5th text shows the score of two teams. From Fig.3, the co-occurrences of the title texts and replays always indicate the appearing of highlights. So, it is reasonable to fuse the title text detection results to improve the highlight detection performance. In [56], the detected title text information is utilized for improving highlight detection performances. Thus from the production knowledge overlaid text information is one of the effective clues for high-level semantics inference.

There are four types of texts in the sport video sequences: 1) long term texts, such as icons of the TV channels, the score-boards which appear at the four corners of video frame, as shown in Fig.4(a) and (b) respectively. This type of texts exists in a fixed location in a frame with long duration. 2) Title texts, e.g. showing the score of two teams after getting a goal, as shown in Fig.4(b). 3) Scene texts, such as the texts appearing in signboard and cloth. 4) Bulletin texts, e.g. showing the name list of a team. This type of text usually appears at the beginning of a soccer video. Except utilizing the text detected in sports video, web-casting

dimensional block wise color moment generated from 5-by-5 grids of the images is also

(a) Original image (b) dominant color region (c) field and non-field region

Texture features are insensitive to lighting, rotation, and other conditions. Two kinds of texture features are utilized for coarse shot classification in [56]. The first is hierarchical wavelet packet texture descriptor (HWVP) [47]. The second is 24 dimensional histogram of oriented gradient (HOG) [46]. HWVP descriptor is a 210 dimensional feature which is extracted under local partitioning pattern Local5 (the image is partitioned into 2×2 grids and a centralized grid), by setting the wavelet packet basis to be db2, with hierarchical wavelet

Title texts provide valuable information for event detection in soccer video content analysis. Usually the overlaid text about goal, foul, yellow/red card are followed the corresponding highlight events. In Fig.3, overlaid texts (including bulletin texts and title texts) and replays of a soccer video are plotted. There are 20 replays and 7 overlaid texts. The first text is a bulletin text that shows the players' name of the two teams. The second text shows the initial score of two teams. The 3rd and the 6th texts show the names of the players who got goals. The 4th and 7th texts show the updated scores just after a goal. Moreover, the 5th text shows the score of two teams. From Fig.3, the co-occurrences of the title texts and replays always indicate the appearing of highlights. So, it is reasonable to fuse the title text detection results to improve the highlight detection performance. In [56], the detected title text information is utilized for improving highlight detection performances. Thus from the production knowledge overlaid text information is one of the effective clues for high-level

There are four types of texts in the sport video sequences: 1) long term texts, such as icons of the TV channels, the score-boards which appear at the four corners of video frame, as shown in Fig.4(a) and (b) respectively. This type of texts exists in a fixed location in a frame with long duration. 2) Title texts, e.g. showing the score of two teams after getting a goal, as shown in Fig.4(b). 3) Scene texts, such as the texts appearing in signboard and cloth. 4) Bulletin texts, e.g. showing the name list of a team. This type of text usually appears at the beginning of a soccer video. Except utilizing the text detected in sports video, web-casting

Fig. 2. Dominant color region, field region and field lines detection results.

utilized.

**2.2 Texture feature** 

packet transform level 2.

semantics inference.

**2.3 Title text detection, localization, and tracking** 

text information is incorporated with audio-visual features to improve highlights detection performances [28,29].

In soccer video the title texts usually appeared in the bottom-center of the image. The icon, scoreboard and time tables are appeared in the top-left and top-right parts of an image respectively. This type text always appeared during the whole video sequence, for example, the extracted local text lines of Fig. 4(a), as shown in Fig.4(c) are appeared in the whole video. However, the title text as appeared in Fig.4(b), as shown in Fig. 4(d) usually existed in a limited time range. The title text is usually provides more information on its corresponding highlight event inference than a long term text. They are purposively added during soccer video editing, and they are aimed at providing the audience some indicative information about the video content, such as a card or a goal [56], [62].

Fig. 3. Overlaid texts (including bulletin texts and title texts) and replays of a soccer video.

Fig. 4. The types of texts appeared in soccer video frames.

## **2.3.1 Related works on video text detection and tracking**

The corresponding text detection approaches can be classified into following 3 types. The first is the connection characteristics of video texts [63], [64]. The text detection methods based on this characteristic assume that text regions have uniform colors and satisfy certain constraints on size, shape, and spatial layout. The second is the texture alike characteristic of the text regions [65, 33, 66, 67]. The text detection methods based on texture information usually assume that the text regions have special texture patterns. And the third is the edge density information [34], [68]. These methods make full use of the fact that the edge densities of background are comparatively sparser than those of the text regions [69], [70]. Usually the corner point number in the text region is larger than that in the background regions.

Semantic Based Sport Video Browsing 145

Global motions in a video sequence are caused by camera motion, which can be modeled by parametric transforms [30, 32]. The process of estimating the transform parameters is called global motion estimation. The widely used global motion model is perspective model with 8

> 012 6 7 345 6 7

*mx m y m*

*mx my mx m y m*

*mx my*

where (,) *x y* and (,) *x y* are the coordinates in the current and the reference image respectively, with the set of parameters 0 7 **m** [ ,, ] *m m* denoting the global motion

Average motion intensities of global motion and local motion are utilized for coarse semantic

where (,) *GMVx GMV j j y* is the global motion vector of the block at (,) *xj j y* , M is the total block number. The global motion vector (,) *GMVx GMVy t t* at the coordinates (,) *t t x y* is

> *GMVx x x GMVy y y*

where (,) *t t x y* are the warped coordinates in the reference frame by the global motion

The local motion information is represented by average motion intensity (*AMV* ) which is

where (*MVxj*, *MVyj*) is the motion vector (MV) of the block with its coordinates (*xj*, *yj*) and *j* 

Parsing the sequential video sequences into shots is helpful for video content analysis. This is often called scene change detection or shot boundary detection. Thee sport video sequences are different with other video sequences, e.g. the highlights are often replayed. Usually, logos are utilized to connect the live broadcasted clips with the replayed segments. Thus, scene changes and logos are the basis of sport video semantics detection and semantic based sport video content browsing. Scene change detection and logo detection approach

1

1 *<sup>M</sup>*

*j AMV MVx MVy <sup>M</sup>*

*ttt ttt*

refinement [56]. The average global motion intensity (*AGMV* ) is calculated as follows:

1 *<sup>M</sup>*

*j AGMV GMVx GMVy <sup>M</sup>*

1

*x*

*y*

1

(7)

(9)

1

2 2

(8)

*j j*

2 2

(10)

*j j*

parameters, which is expressed as follows

parameters to be estimated.

determined as follows

expressed as follows

is the block index.

parameters from the coordinate (*xt*, *yt*).

**2.5 Scene change detection and logo detection** 

Video text detection and localization can be carried out both in pixel and compressed domains [71,34,68,65,72]. In order to eliminate false detections, the edge [69], texture [33], and shape information [57] are often utilized in text verifications. In addition, the available redundant temporal information is often used in candidate text region verification and falsely detected text region elimination [71,34,68,65,72,73]. Lyu et al. proposed a multiresolution based text detection method [34]. Firstly, original edge map is generated for each of target video frames. Multi-resolution text maps of a target frame are generated by downsampling the original text map. Then text detection, verification and localization are carried out on multi-resolution text maps. Finally, the text detection texts in various resolutions are integrated.

## **2.3.2 Text detection and tracking**

In this chapter, we utilize the corresponding text detection and tracking approaches [56]. The block diagram of the title text detection, localization and tracking is shown in Fig.5. Text line number, spatial location, and temporal duration constraints are utilized during title text detection, localization and tracking. Lyu et al' method [34] is used to detect and localize title texts for a given frame.

It is not necessary to carry out text detection for each of the video frames. In our observations, we find that title texts usually exist for about 5 seconds in soccer video. Detecting text with an interval of one or half second is enough, e.g. detecting text in intraframes of sport video. Only the texts appeared at the bottom-center of the images and with sufficient duration are determined as candidate title texts. The starting and ending frames of each title text are determined by text line matching and tracking [33].

### **2.4 Motion feature representation**

Motion information is also very important for sport video content analysis [14,15,17]. Typically the camera men operate cameras, by fast track, or zoom in to provide audiences with clearer views of the games. Based on the camera motion (i.e. global motion) patterns and domain related knowledge, high level semantics can be inferred.

In [18], global views of soccer video are further refined into the following three types: stationary, zoom and track in terms of camera motion information using a set of empirical rules with respect to domain and production knowledge. The key-frames of a shot with stationary by means of average motion intensity and average motion intensities of global motion.

Video text detection and localization can be carried out both in pixel and compressed domains [71,34,68,65,72]. In order to eliminate false detections, the edge [69], texture [33], and shape information [57] are often utilized in text verifications. In addition, the available redundant temporal information is often used in candidate text region verification and falsely detected text region elimination [71,34,68,65,72,73]. Lyu et al. proposed a multiresolution based text detection method [34]. Firstly, original edge map is generated for each of target video frames. Multi-resolution text maps of a target frame are generated by downsampling the original text map. Then text detection, verification and localization are carried out on multi-resolution text maps. Finally, the text detection texts in various resolutions are

In this chapter, we utilize the corresponding text detection and tracking approaches [56]. The block diagram of the title text detection, localization and tracking is shown in Fig.5. Text line number, spatial location, and temporal duration constraints are utilized during title text detection, localization and tracking. Lyu et al' method [34] is used to detect and localize title

It is not necessary to carry out text detection for each of the video frames. In our observations, we find that title texts usually exist for about 5 seconds in soccer video. Detecting text with an interval of one or half second is enough, e.g. detecting text in intraframes of sport video. Only the texts appeared at the bottom-center of the images and with sufficient duration are determined as candidate title texts. The starting and ending frames of

Fig. 5. Title text detection, localization and tracking with text line number, localization and

Motion information is also very important for sport video content analysis [14,15,17]. Typically the camera men operate cameras, by fast track, or zoom in to provide audiences with clearer views of the games. Based on the camera motion (i.e. global motion) patterns

In [18], global views of soccer video are further refined into the following three types: stationary, zoom and track in terms of camera motion information using a set of empirical rules with respect to domain and production knowledge. The key-frames of a shot with stationary by means of average motion intensity and average motion intensities of global

each title text are determined by text line matching and tracking [33].

and domain related knowledge, high level semantics can be inferred.

integrated.

**2.3.2 Text detection and tracking** 

texts for a given frame.

temporal duration constraints.

motion.

**2.4 Motion feature representation** 

Global motions in a video sequence are caused by camera motion, which can be modeled by parametric transforms [30, 32]. The process of estimating the transform parameters is called global motion estimation. The widely used global motion model is perspective model with 8 parameters, which is expressed as follows

$$\begin{cases} \mathbf{x}' = \frac{m\_0 \mathbf{x} + m\_1 \mathbf{y} + m\_2}{m\_6 \mathbf{x} + m\_7 \mathbf{y} + \mathbf{1}}\\ \mathbf{y}' = \frac{m\_3 \mathbf{x} + m\_4 \mathbf{y} + m\_5}{m\_6 \mathbf{x} + m\_7 \mathbf{y} + \mathbf{1}} \end{cases} \tag{7}$$

where (,) *x y* and (,) *x y* are the coordinates in the current and the reference image respectively, with the set of parameters 0 7 **m** [ ,, ] *m m* denoting the global motion parameters to be estimated.

Average motion intensities of global motion and local motion are utilized for coarse semantic refinement [56]. The average global motion intensity (*AGMV* ) is calculated as follows:

$$AGMV = \frac{1}{M} \sum\_{j=1}^{M} \sqrt{\text{GMVx}\_{j}^{2} + \text{GMVy}\_{j}^{2}} \tag{8}$$

where (,) *GMVx GMV j j y* is the global motion vector of the block at (,) *xj j y* , M is the total block number. The global motion vector (,) *GMVx GMVy t t* at the coordinates (,) *t t x y* is determined as follows

$$\begin{cases} \mathbf{G}MVx\_t = \mathbf{x}'\_t - \mathbf{x}\_t \\ \mathbf{G}MVy\_t = y'\_t - y\_t \end{cases} \tag{9}$$

where (,) *t t x y* are the warped coordinates in the reference frame by the global motion parameters from the coordinate (*xt*, *yt*).

The local motion information is represented by average motion intensity (*AMV* ) which is expressed as follows

$$AMV = \frac{1}{M} \sum\_{j=1}^{M} \sqrt{MV\mathbf{x}\_j^2 + MV\mathbf{y}\_j^{-2}} \tag{10}$$

where (*MVxj*, *MVyj*) is the motion vector (MV) of the block with its coordinates (*xj*, *yj*) and *j*  is the block index.

#### **2.5 Scene change detection and logo detection**

Parsing the sequential video sequences into shots is helpful for video content analysis. This is often called scene change detection or shot boundary detection. Thee sport video sequences are different with other video sequences, e.g. the highlights are often replayed. Usually, logos are utilized to connect the live broadcasted clips with the replayed segments. Thus, scene changes and logos are the basis of sport video semantics detection and semantic based sport video content browsing. Scene change detection and logo detection approach

Semantic Based Sport Video Browsing 147

Fig.6 shows four coarse mid-level semantics for soccer video. They are global view, medium

(a) (b)

Fig. 7. Coarse to fine soccer video shot categorization. (a) Coarse soccer video shot

refinement based on camera motion information and field line detection results.

classification using four one-versus-all SVM classifiers. Replay is refined into replay-global, replay-median and replay-close-up using the first three SVM classifiers. (b) Global view

In coarse mid-level semantic categorization, soccer video shots are parsed into four coarse views of global, median, close-up, and audience using four one-versus-all SVM classifiers as shown in Fig.7 (a). The kernels of SVM classifiers are RBF (radius-basis-function). The input of the four SVM classifiers is a 505 dimensional low-level feature, including 255d color moment, 16d dominant color distribution, 24d HOG feature, and 210d HWVP feature.

As median, audience, and close-up views are related to the details, which do not need further refinement. Moreover, the audience shots are rarely replayed, the replay into three types: replay-global, replay-median and replay-close-up using the trained SVM classifiers for coarse semantics classification [56]. The block diagram of global view refinement is

Global view is further refined into one of the following three types: stationary, zoom and track with respect to camera motion information using a set of empirical rules. A shot is determined as with stationary if *AMV*<0.5, otherwise non-stationary. The non-stationary shot is further refined into track if *m*0=*m*5 =1, otherwise zoom. A zoom is a zoom-in if *m*0=*m*<sup>5</sup> >1 and a zoom-out if *m*0=*m*5 <1. The track is a slow-track if 2 *AGMV* , otherwise a fast-

A stationary shot is further refined into stationary-with-goal-post and normal stationary according to the field lines detection results. When the valid field line number is larger than four, then the frame is a stationary with goal-post (SG), otherwise a normal stationary [56].

**3.2 Coarse mid-level semantic classification** 

**3.3 Refinement for coarse mid-level semantics** 

shown in Fig.7 (b) respectively.

track.

view, close-up and audience respectively.

consists of the following three steps [3,56]: (1) logo template detection based on the fact that different starting or ending logo transitions, (2) logo transitions detection in the video program using the logo template based on the probability models of pixel-wise intensitybased mean-square difference and color histogram-based mean-square difference, (3) the replay segments identification.

## **3. Mid-level semantic classification**

In this section, the corresponding middle level semantics for sport video are defined and detected. Related work on middle level semantic classification is reviewed.

## **3.1 Related work on mid-level semantic classification**

Xu *et al.* classified each soccer video shot into one of the following 3 views : global, zoom-in and close-up [16]. From the view labels, soccer video sequences are further parsed into play and break. Duan *et al.* classified video shots into predefined mid-level semantics [18]. Based on which, soccer video is coarsely parsed into two type in-play and out-of-play [14]. In [43], global motion and visual features are fused to carry out shot categorization for basketball video. Tan *et al*. also segmented a basketball sequence into wide angle, close-up, fast break and possible shoot at the basket [61]. Motion and dominant color information of a video shot are fused by multi-layer hidden Markov models (HMM) to determine its category [2]. In [37], each shot of a baseball video is classified into predefined semantic scenes by fusing features extracted from visual content of the image, object level feature representations, and camera motion.

Fig. 6. Four coarse views for soccer video.

consists of the following three steps [3,56]: (1) logo template detection based on the fact that different starting or ending logo transitions, (2) logo transitions detection in the video program using the logo template based on the probability models of pixel-wise intensitybased mean-square difference and color histogram-based mean-square difference, (3) the

In this section, the corresponding middle level semantics for sport video are defined and

Xu *et al.* classified each soccer video shot into one of the following 3 views : global, zoom-in and close-up [16]. From the view labels, soccer video sequences are further parsed into play and break. Duan *et al.* classified video shots into predefined mid-level semantics [18]. Based on which, soccer video is coarsely parsed into two type in-play and out-of-play [14]. In [43], global motion and visual features are fused to carry out shot categorization for basketball video. Tan *et al*. also segmented a basketball sequence into wide angle, close-up, fast break and possible shoot at the basket [61]. Motion and dominant color information of a video shot are fused by multi-layer hidden Markov models (HMM) to determine its category [2]. In [37], each shot of a baseball video is classified into predefined semantic scenes by fusing features extracted from visual content of the image, object level feature representations, and

(a) Global view (b) Medium view

(c) Close-up (d) Audience

detected. Related work on middle level semantic classification is reviewed.

**3.1 Related work on mid-level semantic classification** 

replay segments identification.

camera motion.

**3. Mid-level semantic classification** 

Fig. 6. Four coarse views for soccer video.

## **3.2 Coarse mid-level semantic classification**

Fig.6 shows four coarse mid-level semantics for soccer video. They are global view, medium view, close-up and audience respectively.

Fig. 7. Coarse to fine soccer video shot categorization. (a) Coarse soccer video shot classification using four one-versus-all SVM classifiers. Replay is refined into replay-global, replay-median and replay-close-up using the first three SVM classifiers. (b) Global view refinement based on camera motion information and field line detection results.

In coarse mid-level semantic categorization, soccer video shots are parsed into four coarse views of global, median, close-up, and audience using four one-versus-all SVM classifiers as shown in Fig.7 (a). The kernels of SVM classifiers are RBF (radius-basis-function). The input of the four SVM classifiers is a 505 dimensional low-level feature, including 255d color moment, 16d dominant color distribution, 24d HOG feature, and 210d HWVP feature.

#### **3.3 Refinement for coarse mid-level semantics**

As median, audience, and close-up views are related to the details, which do not need further refinement. Moreover, the audience shots are rarely replayed, the replay into three types: replay-global, replay-median and replay-close-up using the trained SVM classifiers for coarse semantics classification [56]. The block diagram of global view refinement is shown in Fig.7 (b) respectively.

Global view is further refined into one of the following three types: stationary, zoom and track with respect to camera motion information using a set of empirical rules. A shot is determined as with stationary if *AMV*<0.5, otherwise non-stationary. The non-stationary shot is further refined into track if *m*0=*m*5 =1, otherwise zoom. A zoom is a zoom-in if *m*0=*m*<sup>5</sup> >1 and a zoom-out if *m*0=*m*5 <1. The track is a slow-track if 2 *AGMV* , otherwise a fasttrack.

A stationary shot is further refined into stationary-with-goal-post and normal stationary according to the field lines detection results. When the valid field line number is larger than four, then the frame is a stationary with goal-post (SG), otherwise a normal stationary [56].

Semantic Based Sport Video Browsing 149

Wang *et al*. proposed conditional random fields (CRF) based soccer video event detection method [11]. Firstly, mid-level semantics are determined by mapping from low-level audiovisual features. Then, highlights are detected by fusing the mid-level semantic using CRF [11]. In [27], dynamic Bayesian networks (DBN) are utilized to model goal, corner kick, penalty, and foul events. Low-level audio-visual features are mapped into mid-level nodes of Bayesian networks directly. Bayesian networks model the dependency of the observations of each shot for event type inference. The shot-based event recognition results

HCRF is utilized to model highlight events in golf and bowling videos [60]. Different from the existing event detection approaches, the transformed low-level features are utilized to perform event detection. Independent component analysis (ICA) is utilized to determine two main components of the input low-level feature. The HCRF is utilized to fuse the ICA

In most existing works, events are determined from shot level. It is well known that, a single shot separated from its context does poorly in conveying semantics [22], [56]. That is to say the contextual information among the shots of an event clip is not fully disclosed in event inference [11], [27]. Grouping the sequential shots with the same semantics into unified event clips and then recognizing their event types is an optimal way in event detection [56],

Let *VS* denote a video sequence. Assuming that it consists of *K* sequential events, the whole

where ( 1, , ) *Et K <sup>t</sup>* belongs to one of the predefined events. Each event clip *Et* is composed

<sup>1</sup> ( ) ( ,, ) *N t EM M tt t* (12)

where *<sup>q</sup> Mt* ( *q Nt* 1, , ( ) ) corresponds to the predefined mid-level semantics, which can be represented by key-frames of video shots. *N*(*t*) is the total number of mid-level semantics of

> 111 (1) ( ) ( ) <sup>1</sup> <sup>1</sup> ,, ,, ,, ,, ,,

*EEE*

We aimed at segmenting *VS* into event clips by finding event boundaries according to the consistence of semantics. According to domain rules and production knowledge of soccer video, the starting and ending frames of an event clip are the frames at the boundaries where non-global views (including median, close-up, audience, replay and logo) transition

*VS M M M M M M* 

*N N t N K tt K K*

*t K*

(13)

<sup>1</sup> (,, ) *VS E E <sup>K</sup>* (11)

are fused by DBN to accumulate evidence in the temporal domain [27].

components for event type determination.

**4.2 Event clip segmentation** 

to global views [56].

video sequence can modeled as follows

of several sequential mid-level semantics, which is expressed as

1

the *t*-th events. From Eq.(12) and Eq.(11), we have

[21].

After mid-level semantic classification and refinement, each segment of a soccer video or event clip is classified into one of 13 mid-level semantics: logo, audience, median, close-up, replay-global, replay-median, replay-close-up, fast-track, slow-track, zoom-in, zoom-out, normal stationary, and stationary-with-goalmouth.

## **4. High-level semantic detection**

In this section, the high level sport video semantics detection approaches are illustrated in details. Statistical learning models such as hidden Markov model (HMM) as shown in Fig.8(a), enhanced hidden Markov model (EHMM), and hidden conditional random field (HCRF) as shown in Fig.8 (b), based soccer video event detection approaches are described. Correspondingly, soccer video highlight detection results are given.

The main content of this section is as follows. Firstly the related works on soccer video highlight detection are briefly overviewed in Section 4.1. Secondly, event clip segmentation in Section 4.2. Thirdly, HMM, EHMM and HCRF based soccer video event detection approaches are provided in Section 4.3 and Section 4.4 respectively. And finally, highlight detection performances are evaluated in Section 4.5.

Fig. 8. HMM and HCRF models. (a) HMM and (b) HCRF.

#### **4.1 Related work on soccer video highlight detection**

As shown in Fig.1, one type of highlights detection approaches are carried out by mapping from low-level audio-visual features [1], [2], [14], [23], mid-level semantics, coarse events [23], [25], scenes with overlaid text-lines [31] and replays [3]-[5], [31]. Another type of highlight detection approaches are carried out by statistical learning based models. The learning based approaches have been proved to be effective in fusing multi-modal features to improve highlights detection performance [6], [11], [15]-[21], [23]-[27], [35]-[37], [39]-[42], [44], [45].

Xie *et al.* proposed multi-HMMs based method to improve the performance of play-break detection [6]. The HMMs are different structures and with the same input. Dynamic programming is utilized to fuse the outputs of multi-HMMs to determine the shot type (play or break). In [26], soccer video is segmented into sequential event clips: forward pass, shot on goal, placed kick, turnover, counter-attack, and kick-off using a set of domain rules and sport video production knowledge. The placed kicks are further refined into corner kick, free kick and penalty according to the distribution of players' position.

After mid-level semantic classification and refinement, each segment of a soccer video or event clip is classified into one of 13 mid-level semantics: logo, audience, median, close-up, replay-global, replay-median, replay-close-up, fast-track, slow-track, zoom-in, zoom-out,

In this section, the high level sport video semantics detection approaches are illustrated in details. Statistical learning models such as hidden Markov model (HMM) as shown in Fig.8(a), enhanced hidden Markov model (EHMM), and hidden conditional random field (HCRF) as shown in Fig.8 (b), based soccer video event detection approaches are described.

The main content of this section is as follows. Firstly the related works on soccer video highlight detection are briefly overviewed in Section 4.1. Secondly, event clip segmentation in Section 4.2. Thirdly, HMM, EHMM and HCRF based soccer video event detection approaches are provided in Section 4.3 and Section 4.4 respectively. And finally, highlight

As shown in Fig.1, one type of highlights detection approaches are carried out by mapping from low-level audio-visual features [1], [2], [14], [23], mid-level semantics, coarse events [23], [25], scenes with overlaid text-lines [31] and replays [3]-[5], [31]. Another type of highlight detection approaches are carried out by statistical learning based models. The learning based approaches have been proved to be effective in fusing multi-modal features to improve highlights detection performance [6], [11], [15]-[21], [23]-[27], [35]-[37], [39]-[42],

Xie *et al.* proposed multi-HMMs based method to improve the performance of play-break detection [6]. The HMMs are different structures and with the same input. Dynamic programming is utilized to fuse the outputs of multi-HMMs to determine the shot type (play or break). In [26], soccer video is segmented into sequential event clips: forward pass, shot on goal, placed kick, turnover, counter-attack, and kick-off using a set of domain rules and sport video production knowledge. The placed kicks are further refined into corner

kick, free kick and penalty according to the distribution of players' position.

normal stationary, and stationary-with-goalmouth.

detection performances are evaluated in Section 4.5.

 (a) (b) Fig. 8. HMM and HCRF models. (a) HMM and (b) HCRF.

**4.1 Related work on soccer video highlight detection** 

[44], [45].

Correspondingly, soccer video highlight detection results are given.

**4. High-level semantic detection** 

Wang *et al*. proposed conditional random fields (CRF) based soccer video event detection method [11]. Firstly, mid-level semantics are determined by mapping from low-level audiovisual features. Then, highlights are detected by fusing the mid-level semantic using CRF [11]. In [27], dynamic Bayesian networks (DBN) are utilized to model goal, corner kick, penalty, and foul events. Low-level audio-visual features are mapped into mid-level nodes of Bayesian networks directly. Bayesian networks model the dependency of the observations of each shot for event type inference. The shot-based event recognition results are fused by DBN to accumulate evidence in the temporal domain [27].

HCRF is utilized to model highlight events in golf and bowling videos [60]. Different from the existing event detection approaches, the transformed low-level features are utilized to perform event detection. Independent component analysis (ICA) is utilized to determine two main components of the input low-level feature. The HCRF is utilized to fuse the ICA components for event type determination.

In most existing works, events are determined from shot level. It is well known that, a single shot separated from its context does poorly in conveying semantics [22], [56]. That is to say the contextual information among the shots of an event clip is not fully disclosed in event inference [11], [27]. Grouping the sequential shots with the same semantics into unified event clips and then recognizing their event types is an optimal way in event detection [56], [21].

#### **4.2 Event clip segmentation**

Let *VS* denote a video sequence. Assuming that it consists of *K* sequential events, the whole video sequence can modeled as follows

$$VS = (E\_1, \dots, E\_K) \tag{11}$$

where ( 1, , ) *Et K <sup>t</sup>* belongs to one of the predefined events. Each event clip *Et* is composed of several sequential mid-level semantics, which is expressed as

$$E\_t = (M\_t^1, \dots, M\_t^{N(t)}) \tag{12}$$

where *<sup>q</sup> Mt* ( *q Nt* 1, , ( ) ) corresponds to the predefined mid-level semantics, which can be represented by key-frames of video shots. *N*(*t*) is the total number of mid-level semantics of the *t*-th events. From Eq.(12) and Eq.(11), we have

$$VS = \left( \underbrace{M^1\_1, \dots, M^{N(1)}\_1}\_{E\_1}, \dots, \underbrace{M^1\_{t'}, \dots, M^{N(t)}\_t}\_{E\_t}, \dots, \underbrace{M^1\_{\mathbb{K}'}, \dots, M^{N(\mathbb{K})}\_{E\_{\mathbb{K}}}}\_{E\_{\mathbb{K}}} \right) \tag{13}$$

We aimed at segmenting *VS* into event clips by finding event boundaries according to the consistence of semantics. According to domain rules and production knowledge of soccer video, the starting and ending frames of an event clip are the frames at the boundaries where non-global views (including median, close-up, audience, replay and logo) transition to global views [56].

Semantic Based Sport Video Browsing 151

HMM models the state sequence as being Markov, and each observation being independent of all others given the corresponding state [52]. HMM models the joint distribution under

In this section the hidden Markov and enhanced Markov models based event detection approaches are illustrated in details. The HMM based approach is carried out event inference using the normal observations of an event clip. While, the EHMM based approach

HMM is a generative model. It defines a joint probability distribution for the observations and their corresponding labels. HMM models a sequence of observations 1 (,,) *<sup>n</sup>* **x** *x x* with the corresponding label *y* under the assumption that there is an underlying sequence of state 1 (, ,) *<sup>n</sup>* **s** *s s* drawn from a finite state set. HMM carries out inference under two basic independence assumptions [43]. The first assumption is that each state *sj* depends only on its immediate predecessor *sj*-1, and independent of its previous states (i.e. 1 2 , , *<sup>j</sup> s s* ). The second assumption is that each *xj* depends only on the corresponding state *sj*. Let

distribution { } *A a jk* , where *jk a* is the state transition probability from state *j* to state *k*  ( 0 *jk a* ) . *B* denotes the observation probability distribution in state *j* with its observations

> 1 (x|s, ) ( | , ) *n*

<sup>1</sup> {(x , )} *k kN D y <sup>k</sup>* denote the training data. *N* is the total number of training event

*<sup>T</sup> xx x* and its label is ( ) *<sup>k</sup> <sup>y</sup>* . ( ) *<sup>k</sup> xm* corresponds to the *m*-th mid-level semantics in the *k*-th training clip, and ( ) *<sup>k</sup> <sup>y</sup>* is its

The flowchart of the training of HMMs is as follows. Firstly, the mid-level semantics of each event clip are determined from the extracted low-level features. Then, the Expectation-

algorithm consists of E-steps and M-steps [43]. The E-step computes the forward and backward probability for the given model, and the M-step re-estimates the model

maximization (EM) is utilized to estimate the model parameters (,,)

*j P Px s* 

By assuming that observations are statistical independent, then we have

clips. For the *k-*th training clip, its input is ( ) () () ()

denote the model parameters. *A* denotes the state transition probability

*<sup>j</sup>* is the initial state distribution. Thus the joint probability of a state

*j j*

1 2 1 2 (x|s, ) ( ) ( ) ( ) *<sup>n</sup> P bx bx bx s s s n λ* (15)

1 2 x { , ,, } *<sup>k</sup> kk k*

, the EM algorithm can find a model *<sup>t</sup>*

can be modeled as follows

*λ λ* (14)

*i iii A B* 

. The EM

at *t*-th iteration

two basic independence assumptions: Markov property and independent property.

carries out event inference using the enhanced observations of an event clip.

**4.3 HMM based event detection approach** 

**4.3.1 Overview of HMM** 

 (,,) *A B* 

*xk*, { ( )} *B bk <sup>j</sup>* . { } 

**4.3.2 The training of HMM** 

parameters. Given an initial model

Let () ()

event label.

sequence **s** and an observation sequence x under a model

Fig. 9 shows a diagram of event clip (EC) segmentation. The event clips #1 - #4 are composed of global views and several close-up or median views. The event clip #4 is composed of the following mid-level semantics: global, close-up, median, close-up, logo, replays, and logo.

Fig. 9. An example of event boundary determination.

## **4.2.1 Observations of an event clip**

Assuming that an event clip is composed of *n* mid-level semantics, *xj* ( *j n* 1, , ), the temporal transitions of the mid-level semantics of an event clip can be viewed as observations of statistical learning models. Moreover, in [56], the overall feature extracted from each event clip is combined with the temporal observations for event type inference.

The normal observations of an event clip are the temporal transitions of mid-level semantics (i.e. 1 (,,) *<sup>n</sup>* **x** *x x* ) as shown in Fig.10. In [56], the enhanced observations <sup>1</sup> (,,,, ) *n nk xxx* **x** consist of the normal observations and the overall features are utilized to accumulate the semantics for soccer video highlights detection.

The enhanced observation is composed of *k* overall features ( 3 *k* ). They correspond to the title text (TLT), long time break (LTB), and replay (REP) information respectively. All of them are binary. TLT=0 means that there is no title text in event clip. REP =0 denotes that there is no replay in this event clip. LTB=0 represents that the non-global view segment number is very small.

Fig. 10. The temporal and overall features of an event clip. k is the overall feature number.

Fig. 9 shows a diagram of event clip (EC) segmentation. The event clips #1 - #4 are composed of global views and several close-up or median views. The event clip #4 is composed of the following mid-level semantics: global, close-up, median, close-up, logo,

Assuming that an event clip is composed of *n* mid-level semantics, *xj* ( *j n* 1, , ), the temporal transitions of the mid-level semantics of an event clip can be viewed as observations of statistical learning models. Moreover, in [56], the overall feature extracted from each event clip is combined with the temporal observations for event type inference.

The normal observations of an event clip are the temporal transitions of mid-level semantics (i.e. 1 (,,) *<sup>n</sup>* **x** *x x* ) as shown in Fig.10. In [56], the enhanced observations <sup>1</sup> (,,,, ) *n nk xxx* **x** consist of the normal observations and the overall features are utilized

The enhanced observation is composed of *k* overall features ( 3 *k* ). They correspond to the title text (TLT), long time break (LTB), and replay (REP) information respectively. All of them are binary. TLT=0 means that there is no title text in event clip. REP =0 denotes that there is no replay in this event clip. LTB=0 represents that the non-global view segment

Fig. 10. The temporal and overall features of an event clip. k is the overall feature number.

replays, and logo.

Fig. 9. An example of event boundary determination.

to accumulate the semantics for soccer video highlights detection.

**4.2.1 Observations of an event clip** 

number is very small.

#### **4.3 HMM based event detection approach**

HMM models the state sequence as being Markov, and each observation being independent of all others given the corresponding state [52]. HMM models the joint distribution under two basic independence assumptions: Markov property and independent property.

In this section the hidden Markov and enhanced Markov models based event detection approaches are illustrated in details. The HMM based approach is carried out event inference using the normal observations of an event clip. While, the EHMM based approach carries out event inference using the enhanced observations of an event clip.

#### **4.3.1 Overview of HMM**

HMM is a generative model. It defines a joint probability distribution for the observations and their corresponding labels. HMM models a sequence of observations 1 (,,) *<sup>n</sup>* **x** *x x* with the corresponding label *y* under the assumption that there is an underlying sequence of state 1 (, ,) *<sup>n</sup>* **s** *s s* drawn from a finite state set. HMM carries out inference under two basic independence assumptions [43]. The first assumption is that each state *sj* depends only on its immediate predecessor *sj*-1, and independent of its previous states (i.e. 1 2 , , *<sup>j</sup> s s* ). The second assumption is that each *xj* depends only on the corresponding state *sj*. Let (,,) *A B* denote the model parameters. *A* denotes the state transition probability distribution { } *A a jk* , where *jk a* is the state transition probability from state *j* to state *k*  ( 0 *jk a* ) . *B* denotes the observation probability distribution in state *j* with its observations *xk*, { ( )} *B bk <sup>j</sup>* . { } *<sup>j</sup>* is the initial state distribution. Thus the joint probability of a state sequence **s** and an observation sequence x under a model can be modeled as follows

$$P(\mathbf{x} \mid \mathbf{s}, \boldsymbol{\lambda}) = \prod\_{j=1}^{n} P(\mathbf{x}\_{j} \mid \mathbf{s}\_{j}, \boldsymbol{\lambda}) \tag{14}$$

By assuming that observations are statistical independent, then we have

$$P(\mathbf{x} \mid \mathbf{s}, \boldsymbol{\lambda}) = b\_{s\_1}(\mathbf{x}\_1) \times b\_{s\_2}(\mathbf{x}\_2) \times \cdots \times b\_{s\_n}(\mathbf{x}\_n) \tag{15}$$

#### **4.3.2 The training of HMM**

Let () () <sup>1</sup> {(x , )} *k kN D y <sup>k</sup>* denote the training data. *N* is the total number of training event clips. For the *k-*th training clip, its input is ( ) () () () 1 2 x { , ,, } *<sup>k</sup> kk k <sup>T</sup> xx x* and its label is ( ) *<sup>k</sup> <sup>y</sup>* . ( ) *<sup>k</sup> xm* corresponds to the *m*-th mid-level semantics in the *k*-th training clip, and ( ) *<sup>k</sup> <sup>y</sup>* is its event label.

The flowchart of the training of HMMs is as follows. Firstly, the mid-level semantics of each event clip are determined from the extracted low-level features. Then, the Expectationmaximization (EM) is utilized to estimate the model parameters (,,) *i iii A B* . The EM algorithm consists of E-steps and M-steps [43]. The E-step computes the forward and backward probability for the given model, and the M-step re-estimates the model parameters. Given an initial model , the EM algorithm can find a model *<sup>t</sup>* at *t*-th iteration

Semantic Based Sport Video Browsing 153

s,x;

The parameters of HCRF are trained on the training data <sup>1</sup> {(x , )}*<sup>N</sup> D y i ii* , where each observation x*i i i in xx x* ,1 ,2 , , ,, is a sequence of inputs, and each *yi* is the label index of the input sequence x*i*. The parameters estimations in HCRF is carried out by using the following

> <sup>1</sup> ( ) log ( | ; ) <sup>2</sup> *L t t*

objective function in optimization. It is the log of a Gaussian prior with variance <sup>2</sup>

Given a test event clip with its observations x = <sup>1</sup> {,,} *<sup>n</sup> x x* and trained parameters

\* argmax ( ) *L* 

\* argmax ( | , ) <sup>x</sup> *<sup>e</sup> <sup>e</sup>*

where *l<sup>e</sup>* is the parameter vector of the event *e*. In this chapter, we have *e*{Goal, Shoot,

In this section, high-level semantics detection performances of HMM, EHMM, and HCRF based approaches are evaluated. The training set for the models HMM, EHMM and HCRF is the same which consists of *N* manually labeled examples ( , ) *i i* **x** *y* ( *i N* 1, , ) where each

{Goal, Shoot, Normal Kick, Foul, Placed Kick}is the event label, and each observations

( {1, , } *k n* ) of the *i*-th training event clip. 12 soccer sequences and some highlight event clips which are downloaded from Internet are utilized to train the model parameters of

The training steps are as follows: 1) manually label the event clips for the test soccer video sequences. 2) manually label the mid-level semantics. 3) manually label the event

boundaries; 4) get the corresponding mid-level semantics for each event clip.

x <sup>2</sup> 2

is used for regularization which can reduce the over-fitting of the

 

*e Pe l* (25)

*<sup>k</sup> x* corresponds to the observation of the *k*-th segment

(22)

(23)

as follows [50]-

of the

[48, 49].

(24)

s *Z y* ( ; ) exp{ ( ', )} x

*y'*

{1 }

The limited memory BFGS can be utilized to find the optimal parameters \*

*t , ,N*

*p y*

HCRFs, the label *e*\* of this event clip is recognized as follows

**4.5 Highlight detection performances evaluation** 

*<sup>n</sup> x x* . The component *<sup>i</sup>*

where y' is a possible label for the observations x.

**4.4.2 The training of HCRF** 

objective function

The last term <sup>2</sup>

[52].

*yi*

are x = <sup>1</sup> {,,} *ii i*

HMM, EHMM and HCRF.

2 1 2

**4.4.3 The classification of HCRF** 

Normal Kick, Foul, Placed Kick}.

such that its performance is better than the (*t-*1)-th iteration. That is to say for the *K* training samples, the EM algorithm makes the following equation hold.

$$\sum\_{k} P(\mathbf{x}^{(k)} \Big| \mathcal{X}^{t}) \ge \sum\_{k} P(\mathbf{x}^{(k)} \Big| \mathcal{X}^{t-1}) \tag{16}$$

#### **4.3.3 The classification of HMM**

In event recognition, the type of the event clip with observations **x** is determined as follow

$$\log y = \arg\max\_{i} P(\mathbf{x}|\mathcal{Z}\_{i}) = \arg\max\_{i} \sum\_{s \in \mathbf{S}} P(\mathbf{x}|\mathbf{S}\_{i}, \mathcal{Z}\_{i}) \times P(\mathbf{S}|\mathcal{Z}\_{i}),\tag{17}$$

The probability indicates how well the model *<sup>i</sup>* matches the given observations **x**. The probability of the hidden state sequence S can be expressed as

$$P(\mathbf{s} \mid \lambda) = \pi\_{s\_1} a\_{s\_1 s\_2} \times a\_{s\_2 s\_3} \times \cdots \times a\_{s\_{n-1} s\_n} \tag{18}$$

Thus, Eq.(17) can be rewritten as

$$y = \underset{\text{i}}{\text{arg}\,\text{max}} \sum\_{\text{i}} \pi\_{s\_1}^{i} b\_{s\_1}^{i}(\mathbf{x}\_1) a\_{s\_1 s\_2}^{i} \times b\_{s\_2}^{i}(\mathbf{x}\_2) a\_{s\_2 s\_3}^{i} \times \dots \times b\_{s\_n}^{i}(\mathbf{x}\_n) a\_{s\_{n-1} s\_n}^{i} \tag{19}$$

#### **4.4 HCRF based event detection**

Hidden conditional random fields (HCRF) are discriminative models that generalize the hidden Markov models (HMM) and the conditional random fields (CRF) [52]. HCRF models the state sequence as being conditionally Markov given the observations. Unlike HMM, HCRF is also capable of modeling long range dependencies of the observation.

#### **4.4.1 Overview of HCRF**

HCRF uses intermediate hidden variables to model the latent structure of the observations as shown in Fig.8 (b). A HCRF models the conditional probability of a label *y* given the observations x <sup>1</sup> {,,} *<sup>n</sup> x x* with latent states 1 s {, ,} *<sup>n</sup> s s* as follows:

$$p(\mathbf{y} \mid \mathbf{x}; \mathcal{A}) = \frac{1}{Z(\mathbf{x}; \mathcal{A})} \sum\_{\mathbf{s}} p(\mathbf{y}; \mathbf{s} \mid \mathbf{x}; \mathcal{A}) \tag{20}$$

where *p y*( ,| ; ) s x is a conditional probability, given the labels y, observations x, and hidden states s under the HCRF model parameters .

$$p(\mathbf{y}, \mathbf{s} \mid \mathbf{x}; \mathcal{X}) = \exp\{\Psi(\mathbf{y}, \mathbf{s}; \mathbf{x}; \mathcal{X})\}\tag{21}$$

where (, ;) *y* s,x is a potential function parameterized by the model parameters . *Z*(;) x is a normalization factor. It ensures that the model be a properly normalized probability over all labels. It is defined as follows

such that its performance is better than the (*t-*1)-th iteration. That is to say for the *K* training

( ) ( ) <sup>1</sup> ()( ) *k k t t*

argmax ( ) argmax ( ,) ( ,) *<sup>i</sup> s*

**S**

1 1 12 2 23 1

*ii i i i i i s s ss s ss s n s s*

In event recognition, the type of the event clip with observations **x** is determined as follow

*i i i i yP P*

1 12 23 <sup>1</sup> (s| ) *n n P aa a*

1 2

*b xa b xa b xa*

Hidden conditional random fields (HCRF) are discriminative models that generalize the hidden Markov models (HMM) and the conditional random fields (CRF) [52]. HCRF models the state sequence as being conditionally Markov given the observations. Unlike HMM,

HCRF uses intermediate hidden variables to model the latent structure of the observations as shown in Fig.8 (b). A HCRF models the conditional probability of a label *y* given the

> s <sup>x</sup> s x <sup>x</sup> <sup>1</sup> (|; ) ( ,| ; ) (;) *p y p y <sup>Z</sup>*

*p y*( , | ; ) exp{ ( , ; )} s x

*y* s,x

is a normalization factor. It ensures that the model be a properly normalized

is a conditional probability, given the labels y, observations x, and

is a potential function parameterized by the model parameters

.

(20)

argmax () () () , *n nn*

 

**x x** (16)

*P*

**x xS S** (17)

*s ss ss s s λ* (18)

*<sup>i</sup>* matches the given observations **x**. The

(19)

(21)

*k k P P* 

HCRF is also capable of modeling long range dependencies of the observation.

observations x <sup>1</sup> {,,} *<sup>n</sup> x x* with latent states 1 s {, ,} *<sup>n</sup> s s* as follows:

samples, the EM algorithm makes the following equation hold.

**4.3.3 The classification of HMM** 

Thus, Eq.(17) can be rewritten as

**4.4 HCRF based event detection** 

**4.4.1 Overview of HCRF** 

where *p y*( ,| ; ) s x

where (, ;) *y* s,x

 . *Z*(;) x 

hidden states s under the HCRF model parameters

probability over all labels. It is defined as follows

The probability indicates how well the model

*s s <sup>i</sup> <sup>y</sup>*

probability of the hidden state sequence S can be expressed as

1

, ,

*n*

$$Z(\mathbf{x}; \mathcal{A}) = \sum\_{\mathbf{y}'} \sum\_{\mathbf{s}} \exp\{\Psi(\mathbf{y}'; \mathbf{s}, \mathbf{x}; \mathcal{A})\} \tag{22}$$

where y' is a possible label for the observations x.

#### **4.4.2 The training of HCRF**

The parameters of HCRF are trained on the training data <sup>1</sup> {(x , )}*<sup>N</sup> D y i ii* , where each observation x*i i i in xx x* ,1 ,2 , , ,, is a sequence of inputs, and each *yi* is the label index of the input sequence x*i*. The parameters estimations in HCRF is carried out by using the following objective function

$$L(\mathcal{A}) = \sum\_{t \in \{1, \cdots, N\}} \log p(y\_t \mid \mathbf{x}\_t; \mathcal{A}) - \frac{1}{2\sigma^2} \left\| \mathcal{A} \right\|^2 \tag{23}$$

The last term <sup>2</sup> 2 1 2 is used for regularization which can reduce the over-fitting of the objective function in optimization. It is the log of a Gaussian prior with variance <sup>2</sup> [48, 49]. The limited memory BFGS can be utilized to find the optimal parameters \* as follows [50]- [52].

$$\mathcal{X}^\* = \arg\max\_{\mathcal{X}} L(\mathcal{X}) \tag{24}$$

#### **4.4.3 The classification of HCRF**

Given a test event clip with its observations x = <sup>1</sup> {,,} *<sup>n</sup> x x* and trained parameters of the HCRFs, the label *e*\* of this event clip is recognized as follows

$$e^\* = \arg\max\_e P(e \mid \mathbf{x}\_\prime \lambda\_\varepsilon) \tag{25}$$

where *l<sup>e</sup>* is the parameter vector of the event *e*. In this chapter, we have *e*{Goal, Shoot, Normal Kick, Foul, Placed Kick}.

#### **4.5 Highlight detection performances evaluation**

In this section, high-level semantics detection performances of HMM, EHMM, and HCRF based approaches are evaluated. The training set for the models HMM, EHMM and HCRF is the same which consists of *N* manually labeled examples ( , ) *i i* **x** *y* ( *i N* 1, , ) where each *yi* {Goal, Shoot, Normal Kick, Foul, Placed Kick}is the event label, and each observations are x = <sup>1</sup> {,,} *ii i <sup>n</sup> x x* . The component *<sup>i</sup> <sup>k</sup> x* corresponds to the observation of the *k*-th segment ( {1, , } *k n* ) of the *i*-th training event clip. 12 soccer sequences and some highlight event clips which are downloaded from Internet are utilized to train the model parameters of HMM, EHMM and HCRF.

The training steps are as follows: 1) manually label the event clips for the test soccer video sequences. 2) manually label the mid-level semantics. 3) manually label the event boundaries; 4) get the corresponding mid-level semantics for each event clip.

Semantic Based Sport Video Browsing 155

assumptions of HMM. However, HCRF achieves highest performances for detecting shoot

In this section, a semantic based video browsing framework is proposed. Users can view the video content freely like reading books. The semantic based video content browsing approach is organized as the table-of-content (ToC) of a book. Let's give a brief overlook of the ToC of a book, before illustrating the proposed video browsing method. Fig.11 shows the ToC structure of a book which can be departed into following seven layers: (1) book title (BT); (2) chapter (CH); (3) subchapter (SC); (4) paragraph (PH); (5) page (PG); (6) sentence (ST); and (7) words (WD). With respect to the ToC, readers can know its content very well. They can go straight to the interested content by skipping the irrelevant parts. In order to let readers know the overall content of book, the **preface**, **introduction** and **summary** of a book

If sport video content is organized as the ToC of a book, it will be convenient for us to browsing its content. In this chapter, a novel book style based semantic video browsing approach is expressed. TABLE II gives the correspondences of ToC of book and soccer video. In the first layer the book title corresponds to the video category, such as baseball and soccer video. In the second layer, the chapter of a book is similar to a set of events. In the third layer, the sub-chapter corresponds to the detailed information of an event. In the fourth layer, the paragraph of a book is corresponding to the shot of an event. In the fifth layer, the page of a book is similar to the frame of a video. In the sixth layer, the sentence of a book is corresponding to the object in a video sequence. And in the seventh layer, the word of a book corresponds to the pixel. Fig.12 shows the ToC of soccer videos. Soccer video is parsed into following seven events: normal kick, foul, free kick (FK), penalty, corner kick, shoot and goal. The free kick, penalty and corner kick are refined from placed kick using the distributions of players' positions [7]. The fifth layer of the video provides the original video

and goal events.

sequences of event.

Fig. 11. Table-of-Content of a Book.

**5. Semantic based sport video browsing** 

give the brief illustrations of the book, a chapter and a subchapter.

The observations of HMM and HCRF are the temporal transitions of mid-level semantics. The observations of EHMM consist of two parts. The first part is temporal transitions of mid-level semantics which is identical to the observations of HCRF. The second is enhanced observations, including the title texts, long term breaks, and replay information of an event clip [56]. Three overall features are utilized in EHMM. The EHMM and HCRF based event detection approaches are on the basis of event clips and the observation of EHMM and HCRF are mid-level semantics of an event clip.

In order to show the performances of model based event detection approach, in this Section the HMM, EHMM and HCRF based soccer video highlights detection performances are evaluated on seven soccer video sequences. These video sequences are captured from a variety of sources. The total duration of the test video sequences is about 625 minutes. Totally, there are 26 goals, 137 placed kicks, 85 fouls, and 109 shoots. Recall *NR*, precision *NP* and F-measure *F* are used to evaluate objective event detection performances, which are defined as follows:

$$\text{NR} = \frac{\text{NC}}{\text{NC} + \text{NM}} \times 100\% \tag{26}$$

$$NP = \frac{NC}{NC + NF} \times 100\,\%\tag{27}$$

$$F = \frac{2 \times \text{NR} \times \text{NP}}{\text{NR} + \text{NP}} \% \tag{28}$$

where *NC*, *NM*, and *NF* denote the correctly, missed, and falsely detected event numbers. Moreover, confusion matrix is utilized to show the discrimination of HCRF based event detection approach for the five events.

The recall, precision and F-measure values of the four highlights of HMM, EHMM and HCRF are shown in TABLE I. The average recall values of the four highlight events are79.55%, 89.08%, and 86.55%. The average precision values of the four highlight events are76.96%, 87.60%, and 88.03%. The average F-measure values of the highlights are 78.24%, 88.33% and 87.29% respectively.


Table 1. Comparisons of highlights detection performances of HMM, EHMM and HCRF.

EHMM and HCRF based highlight detection approaches outperform that of HMM based approaches. EHMM is a little better than the HCRF based approach for the events: placed kick and foul. The main reasons are due to the following two aspects: 1) overall features of an event clip are utilized in EHMM. The overall features served as global features which are helpful for highlight event discrimination. 2) The overall features compensate the interior of HMM in modeling the dependence of long term observations and ease the two basic

The observations of HMM and HCRF are the temporal transitions of mid-level semantics. The observations of EHMM consist of two parts. The first part is temporal transitions of mid-level semantics which is identical to the observations of HCRF. The second is enhanced observations, including the title texts, long term breaks, and replay information of an event clip [56]. Three overall features are utilized in EHMM. The EHMM and HCRF based event detection approaches are on the basis of event clips and the observation of EHMM and

In order to show the performances of model based event detection approach, in this Section the HMM, EHMM and HCRF based soccer video highlights detection performances are evaluated on seven soccer video sequences. These video sequences are captured from a variety of sources. The total duration of the test video sequences is about 625 minutes. Totally, there are 26 goals, 137 placed kicks, 85 fouls, and 109 shoots. Recall *NR*, precision *NP* and F-measure *F* are used to evaluate objective event detection performances, which are

100% *NC NR*

100% *NC NP*

<sup>2</sup> % *NR NP <sup>F</sup> NR NP*

where *NC*, *NM*, and *NF* denote the correctly, missed, and falsely detected event numbers. Moreover, confusion matrix is utilized to show the discrimination of HCRF based event

The recall, precision and F-measure values of the four highlights of HMM, EHMM and HCRF are shown in TABLE I. The average recall values of the four highlight events are79.55%, 89.08%, and 86.55%. The average precision values of the four highlight events are76.96%, 87.60%, and 88.03%. The average F-measure values of the highlights are 78.24%,

 Placed Kick Foul Shoot Goal method NC NM NF NC NM NF NC NM NF NC NM NF HMM 113 24 18 69 16 25 81 28 28 21 5 14 EHMM 130 7 2 76 9 11 88 21 22 24 2 10 HCRF 120 17 3 73 12 13 90 19 14 26 0 12

Table 1. Comparisons of highlights detection performances of HMM, EHMM and HCRF.

EHMM and HCRF based highlight detection approaches outperform that of HMM based approaches. EHMM is a little better than the HCRF based approach for the events: placed kick and foul. The main reasons are due to the following two aspects: 1) overall features of an event clip are utilized in EHMM. The overall features served as global features which are helpful for highlight event discrimination. 2) The overall features compensate the interior of HMM in modeling the dependence of long term observations and ease the two basic

*NC NM* (26)

*NC NF* (27)

(28)

HCRF are mid-level semantics of an event clip.

detection approach for the five events.

88.33% and 87.29% respectively.

defined as follows:

assumptions of HMM. However, HCRF achieves highest performances for detecting shoot and goal events.

## **5. Semantic based sport video browsing**

In this section, a semantic based video browsing framework is proposed. Users can view the video content freely like reading books. The semantic based video content browsing approach is organized as the table-of-content (ToC) of a book. Let's give a brief overlook of the ToC of a book, before illustrating the proposed video browsing method. Fig.11 shows the ToC structure of a book which can be departed into following seven layers: (1) book title (BT); (2) chapter (CH); (3) subchapter (SC); (4) paragraph (PH); (5) page (PG); (6) sentence (ST); and (7) words (WD). With respect to the ToC, readers can know its content very well. They can go straight to the interested content by skipping the irrelevant parts. In order to let readers know the overall content of book, the **preface**, **introduction** and **summary** of a book give the brief illustrations of the book, a chapter and a subchapter.

If sport video content is organized as the ToC of a book, it will be convenient for us to browsing its content. In this chapter, a novel book style based semantic video browsing approach is expressed. TABLE II gives the correspondences of ToC of book and soccer video. In the first layer the book title corresponds to the video category, such as baseball and soccer video. In the second layer, the chapter of a book is similar to a set of events. In the third layer, the sub-chapter corresponds to the detailed information of an event. In the fourth layer, the paragraph of a book is corresponding to the shot of an event. In the fifth layer, the page of a book is similar to the frame of a video. In the sixth layer, the sentence of a book is corresponding to the object in a video sequence. And in the seventh layer, the word of a book corresponds to the pixel. Fig.12 shows the ToC of soccer videos. Soccer video is parsed into following seven events: normal kick, foul, free kick (FK), penalty, corner kick, shoot and goal. The free kick, penalty and corner kick are refined from placed kick using the distributions of players' positions [7]. The fifth layer of the video provides the original video sequences of event.

Fig. 11. Table-of-Content of a Book.

Semantic Based Sport Video Browsing 157

Semantics

Layer Book Sports Video 1 Book Title Soccer video 2 Chapter A set of Events 3 Sub-Chapter An Event 4 Paragraph Mid-level

5 Page Frame 6 Sentence Objects 7 Word Pixel Table 2. The Correspondence of ToC structure of different types of video sequences.

(a) The third layer abstraction of an event (summary of ToC)

(b) The second layer abstraction of an event (introduction of ToC)

(c) The first layer abstraction of an event (preface of ToC)

Fig. 13. Video ToC for a soccer event.

Fig. 12. ToC of soccer video.

The target of us is how to carry out ToC based sport video summarization. According to the correspondence of ToC of Book and sport video, a four layer summarization framework is utilized in this chapter for soccer video summarization. In the fourth layer, the abstraction is carried out for the semantic shots by extracting several key-frames. The third layer abstraction (i.e. **summary**) of a video is the shortened video frames of a specified story unit or event. The summary of the ToC of a video sequences in the third layer is the combination of the summarizations extracted in the fourth layer for the semantic shots. In the sports video, there are also two kinds of semantic shots: global shots and non-global shots. The second layer abstraction (i.e. **introduction**) of a video is the skimmed video frames of story units or events of a certain class. The first layer abstraction (i.e. **preface**) of a video is providing the corresponding abstractions of the video sequences.

Fig.13 shows an example of the proposed sematic based soccer video browsing approach for a goal event. In the fourth layer, it is the continuous video clips of this event. The third layer shows the extracted 11 key-fames of this event clip, which corresponds to the summary of this event. The second layer shows two representative key-frames, which corresponds to the introduction of this event. The first layer is the most representative frames of this event, namely the preface of this event.

The target of us is how to carry out ToC based sport video summarization. According to the correspondence of ToC of Book and sport video, a four layer summarization framework is utilized in this chapter for soccer video summarization. In the fourth layer, the abstraction is carried out for the semantic shots by extracting several key-frames. The third layer abstraction (i.e. **summary**) of a video is the shortened video frames of a specified story unit or event. The summary of the ToC of a video sequences in the third layer is the combination of the summarizations extracted in the fourth layer for the semantic shots. In the sports video, there are also two kinds of semantic shots: global shots and non-global shots. The second layer abstraction (i.e. **introduction**) of a video is the skimmed video frames of story units or events of a certain class. The first layer abstraction (i.e. **preface**) of a video is

Fig.13 shows an example of the proposed sematic based soccer video browsing approach for a goal event. In the fourth layer, it is the continuous video clips of this event. The third layer shows the extracted 11 key-fames of this event clip, which corresponds to the summary of this event. The second layer shows two representative key-frames, which corresponds to the introduction of this event. The first layer is the most representative frames of this event,

providing the corresponding abstractions of the video sequences.

Fig. 12. ToC of soccer video.

namely the preface of this event.


Table 2. The Correspondence of ToC structure of different types of video sequences.

(a) The third layer abstraction of an event (summary of ToC)

(b) The second layer abstraction of an event (introduction of ToC)

(c) The first layer abstraction of an event (preface of ToC)

Fig. 13. Video ToC for a soccer event.

Semantic Based Sport Video Browsing 159

[12] N. Nan, G. Liu, X. Qian, and C. Wang, "An SVM-based soccer video shot classification

[13] A. Hanjialic, "Generic approach to highlights extraction from a sports video," in *Proc.* 

[14] L. Duan, M. Xu, T. Chua, Q. Tian, and C. Xu, "A mid-level representation framework for semantic sports video analysis," in *Proc. ACM Multimedia*, pp. 29-32,2003. [15] F. Wang, Y. Ma, H. Zhang, and J. Li, "A generic framework for semantic sports video

[16] P. Xu, L. Xie, and S. Chang, "Algorithms and systems for segmentation and structure analysis in soccer video," in *Proc. Int. Conf. Multimedia & Expo*, 2001, pp. 184-187. [17] C. Xu, J. Wang, H. Lu, and Y. Zhang, "A Novel Framework for Semantic Annotation

[18] L. Duan, M. Xu, Q. Tian, C. Xu, and J. S. Jin, "A unified framework for semantic shot

[19] X. Qian, H. Wang, G.Liu, Z. Li, and Z. Wang, "Soccer Video Event Detection by Fusing Middle Level Visual Semantics of an Event Clip", in Proc. PCM 2010, pp.439-451. [20] L. Duan, M. Xu, and Q. Tian, "Semantic shot classification in sports video," in *Proc. SPIE* 

[21] X. Qian, Guizhong Liu, Zhe Wang, Zhi Li, Huan Wang, "Highlight Events Detection in

[22] X. Zhu, X. Wu, A. Elmagarmid, Z. Feng, and L. Wu, "Video data mining semantic

[23] Z. Xiong, R. Radhakrishnan, A. Divakaran, and T. Huang, "Highlights extraction from

[24] C. Xu, Y. Zhang, G. Zhu, Y. Rui, H. Lu, and Q. Huang, "Using Webcast Text for

[25] Y. Wang, Z. Liu, and J. Huang, "Multimedia content analysis using both audio and

[26] J. Assfalg, M. Bertini, C. Colombo, A. Bimbo, and W. Nunziati, "Semantic annotation of

[27] C. Huang, H. Shih, and C. Chao, "Semantic analysis of soccer video using dynamic Bayesian network," IEEE Trans. Multimedia, vol.8, No.4, pp.749-760, Aug. 2006. [28] M. Dao, and N. Babaguchi, "Mining temporal information and web-casting text for automatic sports event detection," in Proc. MMSP, 2008, pp.616-621. [29] M. Dao, and N. Babaguchi, "Sports event detection using temporal patterns mining and

[30] X. Qian, and G. Liu, "Global motion estimation from randomly selected motion vector

groups and GM/LM based applications," Signal, Image and Video Processing,

indexing and event detection from the association perspective," IEEE Trans.

sports video based on an audio-visual marker detection framework, " in Proc. Int.

Semantic Event Detection in Broadcast Sports Video," IEEE Trans. Multimedia,

soccer videos: automatic highlight identification," Computer Vision and Image

*Storage and Retrieval for Media Database*, vol. 5021, 2003, pp. 300-313.

Knowledge and Data Engineering., vol.17, no.5, pp.665-677, 2005.

Soccer Video using HCRF," in Proc. ICIMCS, 2010.

video clues," IEEE Signal Processing Magazine, 2000.

Understanding, vol.6, No.4, pp.285-305, Aug.2003.

web-casting text," in Proc. ACM AREA, 2008, pp.33-40.

Conf. Multimedia & Expo, pp. 29-32,2005.

vol.10, no.7, 2008, pp.1342-1325.

analysis using dynamic Bayesian networks," in *Proc. Int. Conf. Multimedia* 

and Personalized Retrieval of Sports Video," IEEE Transactions on Multimedia,

classification in sports video," *IEEE Trans. Multimedia*, vol.7, No.6, 2005, pp.1066-

scheme using projection histograms," PCM 2008.

*Int. Conf. Image Processing*, vol.1, pp. 1-4, 2003.

*Modelling*, pp. 29-32,2005.

vol. 10, no. 3, 2008, pp.421-436.

1083.

2007.

## **6. Conclusion**

In this chapter, semantic based sport video browsing is introduced. Soccer video high-level semantics detection approaches using hidden Markov models, enhanced Markov models, and hidden conditional fields are expressed in detail and their performances are evaluated. From the detected highlights, a semantic based soccer video browsing approach is proposed. The proposed semantic based soccer video browsing approach carries out video content browsing using a book-like structure.

## **7. Acknowledgement**

This work is supported in part by National Natural Science Foundations of China (NSFC) No.60903121, No.61173109, and Foundation of Microsoft Research Asia.

## **8. References**


In this chapter, semantic based sport video browsing is introduced. Soccer video high-level semantics detection approaches using hidden Markov models, enhanced Markov models, and hidden conditional fields are expressed in detail and their performances are evaluated. From the detected highlights, a semantic based soccer video browsing approach is proposed. The proposed semantic based soccer video browsing approach carries out video

This work is supported in part by National Natural Science Foundations of China (NSFC)

[1] B. Li, J. Errico, H. Pan, and M. Sezan, "Bridging the semantic gap in sports video retrieval and summarization," *J. Vis. Commun. Image R.* vol.17, pp.393-424, 2004. [2] G. Xu, Y. Ma, H. Zhang, and S. Yang, "An HMM-based framework for video semantic

[3] H. Pan, B. Li, and M. Sezan, "Automatic detection of replay segments in broadcast sports

*Speech*, *and Signal Processing*, vol.4, pp. 3385-3388, Orlando, FL, May 2002. [4] Z. Zhao, S. Jiang, Q. Huang, and G. Zhu, "Highlight summarization in sports video

[5] H. Pan, P. Beek, and M. Sezan, "Detection of slow-motion replay segments in sports

[6] L. Xie, S. Chang, A. Divakaran, and H. Sun, "Structure analysis of soccer video with

[7] A. Ekin, and A. Tekalp, "Generic play-break event detection for summarization and

[8] D. W. Tjondronegoro, Y. Chen, and B. Pham, "Classification of self-consumable

[9] G. Zhu, C. Xu, Q. Huang, Y. Rui, S. Jiang, W. Gao, and H. Yao, "Event Tactic Analysis

[10] S. Chen, M. Chen, C. Zhang, and M. Shyu, "Exciting event detection using multi-level

[11] T. Wang, J. Li, Q. Diao, W. Hu, Y. Zhang, and C. Dulong, "Semantic event detection

multimodal descriptors and data classification," in Proc. ISM, 2006.

*Processing*, vol.3, pp.1649-1652, Salt Lake City, USA, May, 2001.

analysis," *IEEE Trans. Circuits and Systems for Video Technology*, vol.15, no.11, Nov.

programs by detecting of logos in scene transitions," in *Proc. Int. Conf. Acoustics*,

based on replay detection," in *Proc. Int. Conf. Mulmedia and Expo*., pp. 1613-1616,

video for highlights generation," in *Proc. Int. Conf. Acoustics*, *Speech*, *and Signal* 

hidden Markov models", in *Proc. Int. Conf. Acoustics*, *Speech*, *and Signal* 

hierarchical sports video analysis," in *Proc. Int. Conf. Mulmedia and Expo*, vol.1,

highlights for soccer video summaries," in *Proc. Int. Conf. Mulmedia and Expo*, 2004,

Based on Broadcast Sport Video," IEEE Trans. Multimedia, vol.11, no.1, 2009,

using conditional random fields," in *Proc. Computer Vision and Pattern Recognition* 

No.60903121, No.61173109, and Foundation of Microsoft Research Asia.

**6. Conclusion** 

**7. Acknowledgement** 

**8. References** 

content browsing using a book-like structure.

2005, pp.1422-1433.

Toronto, Ontario, Canada, July 2006.

*Processing*,2002, pp. 4096-4099.

*Workshop*, 2006, pp.109-115.

2003, pp. 169-172.

pp. 579-582.

pp.49-67.


Semantic Based Sport Video Browsing 161

[48] C. Sutton, and A. McCallum, "An introduction to conditional random fields for

[49] J. Lafferty, A. McCallum, and F. Pereira, "Conditional random fields: Probabilistic

[50] A. McCallum, "Efficiently inducing features of conditional random fields," in Proc.

[51] F. Sha, and F. Pereira, "Shallow parsing with conditional random fields," in Proc.

[52] M. Mahajan, A. Gunawardana, and A. Acero, "Training algorithms for hidden

[53] A. Gunawardana, M. Mahajan, A. Acero, and J. Platt, "Hidden conditional random

[54] Y. Sung, C. Boulis, C. Manning, and D. Jurafsky, "Regularization, adaptation, and non-

[55] R. Malouf, "A comparison of algorithms for maximum entropy parameter estimation,"

[56] X. Qian, H. Wang, G. Liu, and X. Hou, "HMM Based Soccer Video Event Detection

[57] Z. Liu, and S. Sarkar, "Robust outdoor text detection using text intensity and shape

[58] N. Dalal, and B Triggs, "Histogram of oriented gradients for human detection," in Proc.

[59] X. Qian, X. Hua, P. Chen, and L. Ke, "PLBP: An Effective Local Binary Patterns Texture

[60] X. Wang, and X. Zhang, "ICA mixture hidden conditional random field model for

[61] Y. Tan, D. Saur, S. Kulkarni, and P. Ramadge, "Rapid estimation of camera motion from

[62] N. Babaguchi, Y. Kawai, and T. Kitashi, "Event based indexing of broadcasted sports

[63] A. Jain, and B. Yu, "Automatic text location in images and video frames," in Proc. ICPR,

[64] V. Mariano, and R. Kasturi, "Locating uniform-colored text in video frames," in Proc.

[65] X. Qian, and G. Liu, "Text detection, localization and segmentation in compressed

in Proc. Int. Conf. National Language Learning, 2002, pp. 49-55.

Int. Conf. Computer Vision and Pattern Recognition, 2005.

sports event classification," in Proc. ICCV, 2009, pp.562-569.

15th Int. Conf. Pattern Recognit., vol. 4, 2000, pp. 539-542.

videos," in Proc. ICASSP2006., vol. 2, 2006, pp. II385-II388.

Video Technol., vol. 10, no. 1, pp. 133-146, Feb. 2000.

fields for phone classification," in Proc. Int. Conf. Speech Communication and

independent features improve hidden conditional random fields for phone classification," in Workshop of Automatic Speech Recognition and Understanding,

Using Enhanced Mid-Level Semantic", Multimedia Tools and Applications, 2011.

Descriptor with Pyramid Representation", Pattern Recognition, 2011, vol.44, pp.

compressed video with application to video annotation," IEEE Trans. Circuits Syst.

video by intermodal collaboration," IEEE Trans. Multimedia, vol.4, no.1, pp.68-75,

Uncertainty in Articifical Intelligence, 2003, pp.403-410.

conditional random fields," ICASSP 2007, vol.1, pp.273-276.

Human Language Technology, NAACL, 2003.

Technology, pp.1117-1120, Sept. 2005.

2007.

Learning, 2001, pp.282-289.

pp.347-352, 2007.

features," in Proc. ICPR 2008

(Accepted)

2502-2515.

Mar. 2002.

1998, pp. 1497-1499.

relational learning," Introduction to Statistical Relational Learning. MIT Press.

models for segmenting and labeling sequence data," in Proc. Int. Conf. Machine


[31] D. Zhang, and S. Chang, "Event detection in baseball video using superimposed caption

[32] Y. Su, M. Sun, and V. Hsu, "Global motion estimation from coarsely sampled motion

[33] X. Qian, G. Liu, H. Wang, and R. Su, "Text detection, localization and tracking in

[34] M. Lyu, J. Song, and M. Cai, "A comprehensive method for multilingual video text

[35] G. Jin, L. Tao, and G. Xu, "Hidden markov model based events detection in soccer

[36] A. Ekin, A. Tekalp, and R. Mehrotra, "Automatic soccer video analysis and summarization," IEEE Trans. Image Processing, vol.12, no.7, 2003, pp. 796-807. [37] C. Lien, C. Chiang, and C. Lee, "Scene-based event detection for baseball videos," J. Vis.

[38] C. Snoek, and M. Worring, "Multimedia event-based video indexing using time intervals," IEEE Trans. Multimedia, vol.7, No.4, pp.638-647, Aug. 2005. [39] G. Papadopoulos, V. Mezaris, I. Kompatsiaris, and M. Strintzis, "Accumulated motion

[40] F. Wang, Y. Ma, H. Zhang, and J. Li, "Dynamic Bayesian network based event detection

[41] D. Sadlier, and N. O'Connor, "Event detection in field sports video using audio-visual

[42] K. Wickramaratna, M. Chen, S. Chen, and M. Shyu, "Neural network based framework

[43] L. Rabiner, "A tutorial on hidden markov models and selected applications in speech recognition," Proceedings of the IEEE, vol. 77, no. 2, pp. 257-285, 1989. [44] C. Cheng, and C. Hsu, "Fusion of audio and motion information on HMM-based

[45] A. Mittal, L. Cheong, and T. Leung, "Dynamic bayesian framework for extracting

[46] N. Dalal, and B Triggs, "Histogram of oriented gradients for human detection," in Proc.

[47] X. Qian, G. Liu, D. Guo, Z. Li, Z. Wang, and H. Wang, "Object categorization using hierarchical wavelet packet texture descriptors," in Proc. ISM 2009, pp.44-51.

Int. Conf. Computer Vision and Pattern Recognition, 2005.

energy fields estimation and representation for semantic event detection," in Proc.

for soccer highlight extraction," in Proc. Int. Conf. Image Processing, 2004, pp. 633-

features and a support vector Machine," IEEE Trans. Circuits Syst. Video Technol.,

for goal event detection in soccer videos," in Proc. Int. Symposium on Multimedia.

highlight extraction for baseball games," IEEE Trans. Multimedia, vol.8, No.3,

temporal structure in video," in Proc. Int. Conf. Computer Vision and Pattern

315-318.

pp.752-768.

15, No. 2, Feb. 2005, pp. 232-242.

Technology, vol.15, no.2,2005, pp.243-255.

Commun. Image R. 18 (2007) 1–14.

vol. 15, no. 10, pp. 602-615, Oct. 2005.

CIVR, 2008, pp.221-230.

Dec. 2005, pp.21-28.

pp.585-599, June. 2006.

Recognition, 2001, pp. 110-115.

636.

video," ICIAR 2004, LNCS 3221, pp.605-612, 2004.

recognition," in Proc. ACM Multimedia, Juan-les- Pins, France, Nov. 1, 2002, pp.

vector field and the applications", IEEE Trans. Circuits Syst. Video Technol., Vol.

compressed videos," Signal Processing: Image Communication, vol.22 , 2007,

detection, localization, and extraction," IEEE Trans. Circuits and Systems for Video


**7** 

*1FGH e.V., 2TECNALIA 1Germany 2Spain* 

**Intelligent Self-Describing Power Grids** 

The liberalization of the electrical energy market increases the complexity of power grid operation and pushes companies to build cheaper electrical power generation and distribution systems. Automated network management systems based on massively distributed information help to maintain the expected quality performance, manageability and reliability. In this context the European Technology Platform (ETP) Smart Grids was set up in 2005 to create a joint vision for the European networks of 2020 and beyond. Its vision builds both upon an efficient and intelligent integration of distributed resources with other network components and upon two basic concepts upon which the integration of huge amounts of distributed resources is going to be done: "Virtual Power plants" and "Micro Grids" both combined with an extensive use of Information and Communication Technologies (ICT). The Internet and the Web are identified as promising approaches for the

integration of large amounts of distributed resources in a dynamic and efficient way.

other kind of action, such as giving best practice support to the operator.

The EC funded S-TEN (Intelligent Self-describing Technical and Environmental Networks) [1] project running from April 2006 to September 2008 makes a substantial step to let this vision come true by providing decision support in a complex and potentially continuously changing networks based upon the use of semantic web technology [2]. E.g. ontologies are used to represent knowledge associated with the devices, ranging from simple sensors to complex plants such as combined heat and power plants or wind power plants, so that they can register themselves at a registry exposing their capabilities and interfaces to applications searching for a matching service. Besides, semantics describing the equipment design information and measurement data can be used together with the respective data by decision support systems to detect any malfunctioning of the equipment and to trigger corresponding alarms or any

The following chapters present the work being done to develop and use technologies designed to provide functionalities required by the grid of the future. A clear focus is on the defined ontologies as they form the backbone of intelligent and autonomously acting applications.

In future intelligent grids with lots of distributed resources, such as sensors, photovoltaic plants (PV), wind power plants, combined heat and power plants (CHP) and electrical

**1. Introduction** 

**2. S-TEN system** 

Andrea Schröder1 and Inaki Laresgoiti2


## **Intelligent Self-Describing Power Grids**

Andrea Schröder1 and Inaki Laresgoiti2

*1FGH e.V., 2TECNALIA 1Germany 2Spain* 

## **1. Introduction**

162 Semantics in Action – Applications and Scenarios

[66] T. Sato and T. Kanade, "Video OCR: Indexing digital news libraries by recognition of superimposed caption," ICCV Workshop on Image and Video retrieval. 1998. [67] Y. Zhong, H. Zhang, and A. Jain, "Automatic Caption Localization in Compressed

[68] C. Ngo, C. Chan, "Video text detection and segmentation for optical character

[69] J. Zhang, D. Goldgof, and R. Kasturi, "A New Edge-Based Text Verification Approach

[70] L. Sun, G. Liu, X. Qian, D. Guo, "A Novel Text Detection and Localization Method

[71] U.Gargi, S.Antani, and R. Kasturi, "Indexing text events in digital video databases," in

[72] X. Tang, B. Gao, J. Liu, and H. Zhang, "A spatial-temporal approach for video caption

[73] H. Jiang, G. Liu, X. Qian, et al., "A Fast and Efficient Text Tracking in Compressed

detection and recognition," IEEE Trans. Neural Networks.,vol. 13, no.4, 2002, pp.

recognition,". Multimedia Systems, vol.10, no.3, 2005, pp.261-272.

pp.385-392.

961-971.

for Video," in Proc. ICPR, 2008.

Video," in Proc. ISM 2008.

Based on Corner Response," in Proc. ICME 2009.

Proc. Int. Conf. Pattern Recognit., vol. 1, 1998, pp.916-918.

Video," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.22, no.4, 2000,

The liberalization of the electrical energy market increases the complexity of power grid operation and pushes companies to build cheaper electrical power generation and distribution systems. Automated network management systems based on massively distributed information help to maintain the expected quality performance, manageability and reliability. In this context the European Technology Platform (ETP) Smart Grids was set up in 2005 to create a joint vision for the European networks of 2020 and beyond. Its vision builds both upon an efficient and intelligent integration of distributed resources with other network components and upon two basic concepts upon which the integration of huge amounts of distributed resources is going to be done: "Virtual Power plants" and "Micro Grids" both combined with an extensive use of Information and Communication Technologies (ICT). The Internet and the Web are identified as promising approaches for the integration of large amounts of distributed resources in a dynamic and efficient way.

The EC funded S-TEN (Intelligent Self-describing Technical and Environmental Networks) [1] project running from April 2006 to September 2008 makes a substantial step to let this vision come true by providing decision support in a complex and potentially continuously changing networks based upon the use of semantic web technology [2]. E.g. ontologies are used to represent knowledge associated with the devices, ranging from simple sensors to complex plants such as combined heat and power plants or wind power plants, so that they can register themselves at a registry exposing their capabilities and interfaces to applications searching for a matching service. Besides, semantics describing the equipment design information and measurement data can be used together with the respective data by decision support systems to detect any malfunctioning of the equipment and to trigger corresponding alarms or any other kind of action, such as giving best practice support to the operator.

The following chapters present the work being done to develop and use technologies designed to provide functionalities required by the grid of the future. A clear focus is on the defined ontologies as they form the backbone of intelligent and autonomously acting applications.

## **2. S-TEN system**

In future intelligent grids with lots of distributed resources, such as sensors, photovoltaic plants (PV), wind power plants, combined heat and power plants (CHP) and electrical

Intelligent Self-Describing Power Grids 165

For physical devices it is necessary to specify the way in which they announce their presence in a network, so that they can be found when it is necessary to carry out a control or monitoring activity. In the following example, an implementation of the register function is shown: The Register message is used to register an XML-File with the registry. The request to activate this function comprises a HTTP header, a SOAP header with authentication information according to WS-security and an XML-File in the SOAP body which complies

<wsse:Password Type="...#PasswordDigest"> 1mFoYgXbWQ72/6bvnWNEnxtatmU=

In reaction to the register request, the registry sends a HTTP response back to the client:

If the registration request was successful, a HTTP response with an empty result is sent back to the client. Otherwise an error message will be returned in the SOAP body of the HTTP

**3.1 Self-description and registry** 

with the semantically annotated S-TEN registry schema:

POST /STENRegistry/services/Registry HTTP/1.1

<wsse:Username>XXX</wsse:Username>

Content-Type: text/xml; charset=UTF-8

<?xml version='1.0' encoding='UTF-8'?>

SOAPAction: "urn:register" User-Agent: Axis2

 <soapenv:Envelope> <soapenv:Header> <wsse:Security> <wsse:UsernameToken>

</wsse:Password>

...

</wsse:Security>

 </soapenv:Header> <soapenv:Body> <ns1:register>

 </ns1:register> </soapenv:Body> </soapenv:Envelope>

HTTP/1.1 200 OK

<soapenv:Envelope>

 <soapenv:Body> <ns:registerResponse>

 <ns:return> <result></result> </ns:return>

 </ns:registerResponse> </soapenv:Body> </soapenv:Envelope>

....

</wsse:UsernameToken>

<xmlfile> the xml file </xmlfile>

Content-Type: text/xml;charset=UTF-8

<?xml version='1.0' encoding='UTF-8'?>

<soapenv:Header> ... </soapenv:Header>

....

vehicles, each node has its own intelligence, is able to register in the network autonomously and publishes information about its position, services and data.

This is done by giving devices that may be sensors or plants a semantically interpretable self-description. This self-description is then uploaded into the S-TEN registry. The S-TEN registry is also based on semantically annotated Web Services complemented by a semantically annotated registry schema describing the semantics of data stored in the registry. Authorized control systems can access and query for a particular service (Figure 1).

The registry response contains relevant information to access the Web services offered by the sensor/plant, e.g. monitor current data, browse data for a specific instance in time or a period, control capabilities, subscribe to event notification such as alarms and warnings etc.. In addition, semantic annotation of technical documentation and definition and application of rules support best-practice advice to decision-makers enabling them to react more precisely and faster to unusual situations.

Fig. 1. Overview of the S-TEN system.

## **3. S-TEN technology**

The S-TEN system is based on the S-TEN technology consisting of the following main components:


The following section describes the above listed main components of the S-TEN technology in more detail.

vehicles, each node has its own intelligence, is able to register in the network autonomously

This is done by giving devices that may be sensors or plants a semantically interpretable self-description. This self-description is then uploaded into the S-TEN registry. The S-TEN registry is also based on semantically annotated Web Services complemented by a semantically annotated registry schema describing the semantics of data stored in the registry. Authorized control systems can access and query for a particular service (Figure 1). The registry response contains relevant information to access the Web services offered by the sensor/plant, e.g. monitor current data, browse data for a specific instance in time or a period, control capabilities, subscribe to event notification such as alarms and warnings etc.. In addition, semantic annotation of technical documentation and definition and application of rules support best-practice advice to decision-makers enabling them to react more

> Service Type Query

Control Systems

Matching

Registry (Un)Reg

ister Device

and relevant information of the registered resources.

concepts of the S-TEN top ontologies have been defined.

Service Coordinates

The S-TEN system is based on the S-TEN technology consisting of the following main

 **Self-description and registry.** S-TEN systems publish their **self-description** at the S-TEN registry exposing their capabilities and interfaces to applications searching for a matching service. An application first accesses the registry in order to get the endpoints

**Ontologies. S-TEN top ontologies** and **demonstrator specific ontologies** referencing

 **Semantically annotated web services.** The semantics used refer to concepts of the S-TEN ontology for self-describing networks and the S-TEN ontologies for physical quantity space and scale. The web service based implementation of S-TEN services is

The following section describes the above listed main components of the S-TEN technology

Service Request

Service Response

and publishes information about its position, services and data.

precisely and faster to unusual situations.

1

2

N

Devices

Fig. 1. Overview of the S-TEN system.

**3. S-TEN technology** 

based on Axis2 [3].

in more detail.

components:

## **3.1 Self-description and registry**

For physical devices it is necessary to specify the way in which they announce their presence in a network, so that they can be found when it is necessary to carry out a control or monitoring activity. In the following example, an implementation of the register function is shown: The Register message is used to register an XML-File with the registry. The request to activate this function comprises a HTTP header, a SOAP header with authentication information according to WS-security and an XML-File in the SOAP body which complies with the semantically annotated S-TEN registry schema:

```
POST /STENRegistry/services/Registry HTTP/1.1 
Content-Type: text/xml; charset=UTF-8 
SOAPAction: "urn:register" 
User-Agent: Axis2 
.... 
<?xml version='1.0' encoding='UTF-8'?> 
 <soapenv:Envelope> 
 <soapenv:Header> 
 <wsse:Security> 
 <wsse:UsernameToken> 
 <wsse:Username>XXX</wsse:Username> 
 <wsse:Password Type="...#PasswordDigest"> 1mFoYgXbWQ72/6bvnWNEnxtatmU= 
</wsse:Password> 
 </wsse:UsernameToken> 
 </wsse:Security> 
 ... 
 </soapenv:Header> 
 <soapenv:Body> 
 <ns1:register> 
 <xmlfile> the xml file </xmlfile> 
 </ns1:register> 
 </soapenv:Body>
```

```
 </soapenv:Envelope>
```
In reaction to the register request, the registry sends a HTTP response back to the client:

```
HTTP/1.1 200 OK
```

```
Content-Type: text/xml;charset=UTF-8 
.... 
<?xml version='1.0' encoding='UTF-8'?> 
 <soapenv:Envelope> 
 <soapenv:Header> ... </soapenv:Header> 
 <soapenv:Body> 
 <ns:registerResponse> 
 <ns:return> 
 <result></result> 
 </ns:return> 
 </ns:registerResponse> 
 </soapenv:Body> 
 </soapenv:Envelope>
```
If the registration request was successful, a HTTP response with an empty result is sent back to the client. Otherwise an error message will be returned in the SOAP body of the HTTP

Intelligent Self-Describing Power Grids 167

maxOccurs="unbounded" sawsdl:modelReference="http://www.s-ten.eu/sten-web-

<xsd:element name="MonitoredDevices" type="xsd:string" minOccurs="0" maxOccurs="unbounded"

<xsd:element name="OfferedServices" type="xsd:string" minOccurs="0" maxOccurs="unbounded"

 <xsd:element name="LegacySystemSpecificInformation" minOccurs="0" maxOccurs="unbounded" sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#LegacySystemInformation">

Ontologies are an important part of the Semantic Web and they are thought to be one of the most important advances in software development. They define the terms used to describe and represent an area of knowledge and have the advantage of being a way for a much

They are composed of a formal description of concepts (classes) in a domain of discourse, properties of each concept describing various features of the concept, and restrictions on properties. If we add a set of individual instances of classes to the ontology we obtain a

There are different languages that can be used to express ontologies such as XML, RDF(S), SHOE, OWL, etc. In the S-TEN project the Web Ontology Language (OWL) [6] is employed for the definition of the case study because it provides the most expressive language for

<xsd:element name="ObservedPhysicalQuantitySpace" type="xsd:string" minOccurs="0"

sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#InventaryNumber"/>

sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Manufacturer"/>

sawsdl:modelReference="http://www.labein.es/Microgrid.xml#Device"/>

 sawsdl:modelReference="http://www.labein.es/Microgrid.xml#Service"/> <xsd:element name="InventaryNumber" type="xsd:string" minOccurs="0"

sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Owner"/>

sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Possessor"/>

sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Operator"/>

<xsd:element name="TextualDescription" type="xsd:string" minOccurs="0"/>

more formal and less ambiguous definition of the meaning of the concepts.

<xsd:element name="Producer" type="xsd:string" minOccurs="0"

<xsd:element name="Owner" type="xsd:string" minOccurs="0"

<xsd:element name="Possessor" type="xsd:string" minOccurs="0"

<xsd:element name="Operator" type="xsd:string" minOccurs="0"

<xsd:element name="Name" type="xsd:string" minOccurs="0"/>

<xsd:element name="URN" type="xsd:anyURI"/>

<xsd:element name="Value" type="xsd:string"/>

<xsd:element name="System" type="SystemType"/>

 </xsd:sequence> </xsd:complexType> </xsd:element>

 <xsd:complexType> <xsd:sequence>

 </xsd:sequence> </xsd:complexType> </xsd:element> </xsd:sequence> </xsd:complexType>

</xsd:schema>

**3.2 Ontologies** 

knowledge base.

ontology description.

services#PhysicalQuantitySpace"/>

response. To enable the invoking client to semantically interpret the register service offered by the registry server, the WSDL [4] specification of the registry service has to be semantically annotated according to SAWSDL [5]. As at this time Axis2 did not fully support WSDL 2.0 it was necessary to base the work on WSDL 1.1, and annotate the respective S-TEN registry WSDL specification accordingly. The data-type associated with the Register Message looks in its semantically annotated form the following way:

<xs:element name="register" sawsdl:modelReference="http://www.s-ten.eu/sten-webservices#AddRequestMessage">

```
 <xs:complexType> 
 <xs:sequence> 
 <xs:element name="xmlfile" nillable="true" 
 sawsdl:modelReference=" 
 http://www.s-ten.eu/sten-web-services#XMLObject 
 http://www.w3.org/2000/01/rdf-schema#isDefinedBy 
 http://www.s-ten.eu/sten-web-services#RegistrySchema"/> 
 </xs:sequence> 
 </xs:complexType> 
</xs:element>
```
The referenced registry schema is then again semantically annotated which enables the client using the GetSchema message to arrive at a complete, semantically correct interpretation of the expected registry content.

```
<xsd:schema xmlns:Ontology0="http://www.s-ten.eu/sten#"
xmlns:sawsdl="http://www.w3.org/ns/sawsdl" 
 xmlns:xsd="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified"
attributeFormDefault="unqualified"> 
 <xsd:complexType name="SystemType" sawsdl:modelReference="http://www.s-ten.eu/sten-web-
services#S-TENSystem"> 
 <xsd:sequence> 
 <xsd:element name="SystemID" type="xsd:anyURI" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#ReferenceDesignation"/> 
 <xsd:element name="Name" type="xsd:string" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Designation"/> 
 <xsd:element name="S-TENServiceDescriptionEndpoint" type="xsd:anyURI" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#WSDLDescription"/> 
 <xsd:element name="TextualDescription" type="xsd:string" minOccurs="0" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Description"/> 
 <xsd:element name="SymbolicLocation" type="xsd:string" minOccurs="0" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#LocationDesignation"/> 
 <xsd:element name="S-TENDescriptionEndpoint" type="xsd:anyURI" minOccurs="0" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#OWLDescription"/> 
 <xsd:element name="GISLocation" minOccurs="0" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#PointInSpace"> 
 <xsd:complexType> 
 <xsd:sequence> 
 <xsd:element name="Longitude" type="xsd:decimal" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Longitude"/> 
 <xsd:element name="Latitude" type="xsd:decimal" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Latitude"/> 
 <xsd:element name="HeightOverSeaLevel" type="xsd:decimal" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#HeightOverSeaLevel"/>
```
response. To enable the invoking client to semantically interpret the register service offered by the registry server, the WSDL [4] specification of the registry service has to be semantically annotated according to SAWSDL [5]. As at this time Axis2 did not fully support WSDL 2.0 it was necessary to base the work on WSDL 1.1, and annotate the respective S-TEN registry WSDL specification accordingly. The data-type associated with

The referenced registry schema is then again semantically annotated which enables the client using the GetSchema message to arrive at a complete, semantically correct

<xsd:schema xmlns:Ontology0="http://www.s-ten.eu/sten#"

xmlns:xsd="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified"

<xsd:complexType name="SystemType" sawsdl:modelReference="http://www.s-ten.eu/sten-web-

sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#ReferenceDesignation"/>

sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#WSDLDescription"/>

 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#LocationDesignation"/> <xsd:element name="S-TENDescriptionEndpoint" type="xsd:anyURI" minOccurs="0" sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#OWLDescription"/>

 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Designation"/> <xsd:element name="S-TENServiceDescriptionEndpoint" type="xsd:anyURI"

sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#PointInSpace">

sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Longitude"/>

sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Latitude"/>

sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#HeightOverSeaLevel"/>

 <xsd:element name="TextualDescription" type="xsd:string" minOccurs="0" sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Description"/>

<xsd:element name="SymbolicLocation" type="xsd:string" minOccurs="0"

the Register Message looks in its semantically annotated form the following way: <xs:element name="register" sawsdl:modelReference="http://www.s-ten.eu/sten-web-

services#AddRequestMessage">

sawsdl:modelReference="

<xs:element name="xmlfile" nillable="true"

interpretation of the expected registry content.

xmlns:sawsdl="http://www.w3.org/ns/sawsdl"

<xsd:element name="SystemID" type="xsd:anyURI"

<xsd:element name="GISLocation" minOccurs="0"

<xsd:element name="Longitude" type="xsd:decimal"

<xsd:element name="Latitude" type="xsd:decimal"

<xsd:element name="HeightOverSeaLevel" type="xsd:decimal"

<xsd:element name="Name" type="xsd:string"

attributeFormDefault="unqualified">

services#S-TENSystem"> <xsd:sequence>

 <xsd:complexType> <xsd:sequence>

 http://www.s-ten.eu/sten-web-services#XMLObject http://www.w3.org/2000/01/rdf-schema#isDefinedBy http://www.s-ten.eu/sten-web-services#RegistrySchema"/>

 <xs:complexType> <xs:sequence>

 </xs:sequence> </xs:complexType>

</xs:element>

```
 </xsd:sequence> 
 </xsd:complexType> 
 </xsd:element> 
 <xsd:element name="ObservedPhysicalQuantitySpace" type="xsd:string" minOccurs="0" 
 maxOccurs="unbounded" sawsdl:modelReference="http://www.s-ten.eu/sten-web-
services#PhysicalQuantitySpace"/> 
 <xsd:element name="MonitoredDevices" type="xsd:string" minOccurs="0" maxOccurs="unbounded" 
 sawsdl:modelReference="http://www.labein.es/Microgrid.xml#Device"/> 
 <xsd:element name="OfferedServices" type="xsd:string" minOccurs="0" maxOccurs="unbounded" 
 sawsdl:modelReference="http://www.labein.es/Microgrid.xml#Service"/> 
 <xsd:element name="InventaryNumber" type="xsd:string" minOccurs="0" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#InventaryNumber"/> 
 <xsd:element name="Producer" type="xsd:string" minOccurs="0" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Manufacturer"/> 
 <xsd:element name="Owner" type="xsd:string" minOccurs="0" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Owner"/> 
 <xsd:element name="Possessor" type="xsd:string" minOccurs="0" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Possessor"/> 
 <xsd:element name="Operator" type="xsd:string" minOccurs="0" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#Operator"/> 
 <xsd:element name="LegacySystemSpecificInformation" minOccurs="0" maxOccurs="unbounded" 
 sawsdl:modelReference="http://www.s-ten.eu/sten-web-services#LegacySystemInformation"> 
 <xsd:complexType> 
 <xsd:sequence> 
 <xsd:element name="URN" type="xsd:anyURI"/> 
 <xsd:element name="Name" type="xsd:string" minOccurs="0"/> 
 <xsd:element name="TextualDescription" type="xsd:string" minOccurs="0"/> 
 <xsd:element name="Value" type="xsd:string"/> 
 </xsd:sequence> 
 </xsd:complexType> 
 </xsd:element> 
 </xsd:sequence> 
 </xsd:complexType> 
 <xsd:element name="System" type="SystemType"/> 
</xsd:schema>
```
#### **3.2 Ontologies**

Ontologies are an important part of the Semantic Web and they are thought to be one of the most important advances in software development. They define the terms used to describe and represent an area of knowledge and have the advantage of being a way for a much more formal and less ambiguous definition of the meaning of the concepts.

They are composed of a formal description of concepts (classes) in a domain of discourse, properties of each concept describing various features of the concept, and restrictions on properties. If we add a set of individual instances of classes to the ontology we obtain a knowledge base.

There are different languages that can be used to express ontologies such as XML, RDF(S), SHOE, OWL, etc. In the S-TEN project the Web Ontology Language (OWL) [6] is employed for the definition of the case study because it provides the most expressive language for ontology description.

Intelligent Self-Describing Power Grids 169

developed and applied to information available in the Web, e.g. measurements. Rules trigger corresponding alarms and generate Best Practice advice for the system operator based on the currently valid state of the network. Different organisations usually have different views on the same data. Therefore each organisation can have their own rule-bases to generate appropriate Best Practice Advice according to the nature of the system status enabling decision-makers to react more precise and faster to unusual situations and

A network is made up of physical objects. Information about these objects includes:

a manufacturing or processing activity performed by a physical object within a

what the quality of the measurement is (if it is a measurement device).

unbalanced systems.

Fig. 2. S-TEN ontology.

what it is;

what it does;

An activity can be:

**activities** 

The scope of the S-TEN ontology includes:

**physical objects (physical thing)** 

what it is connected to;

where it is (e.g. input from GPS);

what it measures (if it is a measurement device);

network as part of its normal operation;

OWL was the W3C recommended standard for ontologies when the S-TEN project started. It is an extension of RDF(S) [7] that provides additional vocabulary in order to represent more characteristics between concepts. It distinguishes three types of entities: individuals, classes and properties:


There are three different OWL sublanguages:


For S-TEN OWL-DL was chosen because the existent reasoners were based on OWL-DL and although they would work also in OWL-Full they could enter into endless loops.

The S-TEN ontology [8] describes all S-TEN system components in a computer understandable way and provides the semantics necessary for exchanging messages between the different components of the S-TEN system including those coming from external users and the ones used internally by the system. Key features of the S-TEN ontology are as follows:


A top level ontology for sensor related information and Web Services has been developed taking into account existing standards such as ISO 15926, IEEE SUMO, IEC 61850 and ISO 10303 (Figure 3). Based on the top S-TEN ontologies, application specific ontologies referencing concepts of the S-TEN top level ontology have been defined for the prototype applications.

The advantage of having an ontology is that automated reasoning can be applied to the data of the network components. Within S-TEN, rules supporting system operations are

OWL was the W3C recommended standard for ontologies when the S-TEN project started. It is an extension of RDF(S) [7] that provides additional vocabulary in order to represent more characteristics between concepts. It distinguishes three types of entities: individuals,

Individuals (or instances) represent objects in the domain of interest (e.g.

 Classes are groups of individuals (e.g. Equipment, Transformer). Classes can be organised into a superclass-subclass hierarchy, which is also known as taxonomy. Properties are binary relations between individuals. There are two main types of properties: object properties that link two individuals and datatype properties that link an individual to an XML Schema Datatype value or an rdf literal (e.g. hasActivePower,

 OWL-Lite: It is the syntactically simplest sublanguage. It is intended to be used in situations where only a simple class hierarchy and simple constraints are needed. OWL-DL: It is much more expressive than OWL-Lite and it is based on Description Logics. It can be used when a maximum expressiveness is required but the computational completeness (all conclusions are guaranteed to be computable) and

OWL-Full: It provides the maximum expressiveness and the syntactic freedom of RDF

For S-TEN OWL-DL was chosen because the existent reasoners were based on OWL-DL and

The S-TEN ontology [8] describes all S-TEN system components in a computer understandable way and provides the semantics necessary for exchanging messages between the different components of the S-TEN system including those coming from external users and the ones used internally by the system. Key features of the S-TEN

it can be used by intelligent devices to publish information about their existence and

it encompasses measurements made by sensors within a network and the storage and

it enables the use of inferencing to support activities carried out upon a network,

A top level ontology for sensor related information and Web Services has been developed taking into account existing standards such as ISO 15926, IEEE SUMO, IEC 61850 and ISO 10303 (Figure 3). Based on the top S-TEN ontologies, application specific ontologies referencing concepts of the S-TEN top level ontology have been defined for the prototype

The advantage of having an ontology is that automated reasoning can be applied to the data of the network components. Within S-TEN, rules supporting system operations are

decidability (all computations will finish in finite time) must be ensured.

although they would work also in OWL-Full they could enter into endless loops.

it specifies an ontology for describing a physical network;

classes and properties:

Transformer\_1)

ontology are as follows:

state;

applications.

There are three different OWL sublanguages:

with no computational guaranties.

processing of measurements and

including operations and maintenance.

name).

developed and applied to information available in the Web, e.g. measurements. Rules trigger corresponding alarms and generate Best Practice advice for the system operator based on the currently valid state of the network. Different organisations usually have different views on the same data. Therefore each organisation can have their own rule-bases to generate appropriate Best Practice Advice according to the nature of the system status enabling decision-makers to react more precise and faster to unusual situations and unbalanced systems.

### Fig. 2. S-TEN ontology.

The scope of the S-TEN ontology includes:

#### **physical objects (physical thing)**

A network is made up of physical objects. Information about these objects includes:


#### An activity can be:

 a manufacturing or processing activity performed by a physical object within a network as part of its normal operation;

Intelligent Self-Describing Power Grids 171

sten:facility, which is an object defined by the role it performs, and which is of

sten:physical\_asset, which is an object defined by its material, and which is of

computer file, which is an information bearing object defined by a computer

EXAMPLE: Consider a pump in a process plant. The switch in the control room which operates this pump is labeled "P\_101". On 2007-03-19, the pump with serial number "98- 1234" is installed as "P\_101". A maintenance activity is carried out overnight, so that on

The object "P\_101" is a facility, and is defined by the role it performs. The object "98-

The qualification *more or less* is important, because a major refurbishment may change many of the parts of asset "98-1234". In this case, whether or not a new asset is created is

The justification for regarding a computer file as something physical is discussed in

 sten:period\_of\_life, in which a sten:persistent\_object operates during a sten:period; sten:activity, which is one or more instances of sten:period\_of\_life, and which is

NOTE 5: There are particular relationships between an sten:activity and its parts, such as sten:has\_input, sten:has\_output, sten:creates, sten:destroys, and sten:performed\_by.

4. There are many types of sten:persistent\_object. These include:

2007-03-20 the pump with serial number "03-5678" is installed.

5. There are many types of sten:ephemeral\_object. These include:

defined in order to identify a purpose, cause or result.

Top classes in the S-TEN, SUMO and ISO 15926-2 ontologies are shown in Figure 3.

Relationships which are valid for the different types of physical thing are as follows:

**relationship rdfs:Property rdfs:domain rdfs:range** 

at time sten:at\_time sten:classical\_object\_at\_instant sten:instant during sten:during physical\_thing sten:part\_of\_time

whole-part sten:part\_of sten:physical\_thing sten:physical\_thing

assembly sten:component\_of sten:classical\_object sten:classical\_object

The difference between the relationships sten:part\_of, sten:temporal\_part\_of and

sten:has\_part sten:physical\_thing sten:physical\_thing

sten:temporal\_part\_of sten:physical\_thing sten:classical\_object sten:has\_temporal\_part sten:classical\_object sten:physical\_thing

sten:has\_component sten:classical\_object sten:classical\_object

1234" is an asset and is defined *more or less* by the matter it consists of.

interest for operations;

system.

an administrative choice.

Section Information.

*Principal relationships* 

sten:component\_of is illustrated by Figure 4.

temporal whole-part

interest for materials management;


The information about any activity, can include:


An input or output can be an information object.

**information** 

It is necessary to record information about both information objects and their information contents.

**Physical quantities, property and scale** 

For physical things the definition of the ontology will be described in detail. The ontologies for the other categories can be found at http://www.s-ten.eu/deliverables/D2.1/.

#### **Physical things**

#### *Types of physical thing*

The purpose of the structure at the top of the ontology is to specify which relationships are valid for which types of physical thing. These relationships are inherited down the ontology. The principal relationships are discussed in the section Principal relationships. The top of the ontology for S-TEN, is a variant of that in SUMO and ISO 15926-2 as follows:

	- sten:classical\_object which consists of very many interacting particles for a period;
	- sten:classical\_object\_at\_instant which is a sten:classical\_object at an instant; or
	- non-classical thing which is everything else, including small numbers of particles isolated from the rest of the universe, and packets of energy.

For a sten:classical\_object, time, position and causality are all valid concepts. Nonclassical things are not within the scope of S-TEN.

	- sten:persistent\_object which is created by a sten:activity and which continues to exist until destroyed by another sten:activity; and
	- sten:ephemeral\_object which is a part of a sten:persistent\_object which existed before or continued to exist after.

The different between sten:persistent\_object and sten:ephemeral\_object is contentious amongst upper ontologists. An sten:ephemeral\_object is something which has been defined by a person for a purpose. The beginning and end of an sten:ephemeral\_object is not necessarily obvious to another person. Unfortunately, the same can be sometimes be said for a sten:persistent\_object.

other physical objects which participate in the activity, particularly the inputs and

It is necessary to record information about both information objects and their

For physical things the definition of the ontology will be described in detail. The ontologies

The purpose of the structure at the top of the ontology is to specify which relationships are valid for which types of physical thing. These relationships are inherited down the ontology. The principal relationships are discussed in the section Principal relationships. The top of

 sten:classical\_object which consists of very many interacting particles for a period; sten:classical\_object\_at\_instant which is a sten:classical\_object at an instant; or non-classical thing which is everything else, including small numbers of particles

For a sten:classical\_object, time, position and causality are all valid concepts. Non-

sten:persistent\_object which is created by a sten:activity and which continues to

sten:ephemeral\_object which is a part of a sten:persistent\_object which existed

The different between sten:persistent\_object and sten:ephemeral\_object is contentious amongst upper ontologists. An sten:ephemeral\_object is something which has been defined by a person for a purpose. The beginning and end of an sten:ephemeral\_object is not necessarily obvious to another person. Unfortunately, the same can be sometimes

for the other categories can be found at http://www.s-ten.eu/deliverables/D2.1/.

the ontology for S-TEN, is a variant of that in SUMO and ISO 15926-2 as follows:

isolated from the rest of the universe, and packets of energy.

classical things are not within the scope of S-TEN.

before or continued to exist after.

be said for a sten:persistent\_object.

exist until destroyed by another sten:activity; and

a measurement activity performed by a measurement device;

The information about any activity, can include:

An input or output can be an information object.

the performer, which can be a person or device; and

what it is;

outputs.

information contents.

**Physical quantities, property and scale** 

**information** 

**Physical things** 

*Types of physical thing* 

2. A sten:physical\_thing is one of:

3. A sten:classical\_object can be:

a control, maintenance or measurement activity performed by a person

	- sten:facility, which is an object defined by the role it performs, and which is of interest for operations;
	- sten:physical\_asset, which is an object defined by its material, and which is of interest for materials management;
	- computer file, which is an information bearing object defined by a computer system.

EXAMPLE: Consider a pump in a process plant. The switch in the control room which operates this pump is labeled "P\_101". On 2007-03-19, the pump with serial number "98- 1234" is installed as "P\_101". A maintenance activity is carried out overnight, so that on 2007-03-20 the pump with serial number "03-5678" is installed.

The object "P\_101" is a facility, and is defined by the role it performs. The object "98- 1234" is an asset and is defined *more or less* by the matter it consists of.

The qualification *more or less* is important, because a major refurbishment may change many of the parts of asset "98-1234". In this case, whether or not a new asset is created is an administrative choice.

The justification for regarding a computer file as something physical is discussed in Section Information.

	- sten:period\_of\_life, in which a sten:persistent\_object operates during a sten:period;
	- sten:activity, which is one or more instances of sten:period\_of\_life, and which is defined in order to identify a purpose, cause or result.

NOTE 5: There are particular relationships between an sten:activity and its parts, such as sten:has\_input, sten:has\_output, sten:creates, sten:destroys, and sten:performed\_by.

Top classes in the S-TEN, SUMO and ISO 15926-2 ontologies are shown in Figure 3.

#### *Principal relationships*

Relationships which are valid for the different types of physical thing are as follows:


The difference between the relationships sten:part\_of, sten:temporal\_part\_of and sten:component\_of is illustrated by Figure 4.

Intelligent Self-Describing Power Grids 173

An sten:activity (= sumo:Process) is defined in order to identify a purpose, cause or result. S-TEN defines the sten:has\_role\_player relationship between an sten:activity and a sten:persistent\_object. This is not a relationship with a temporal part because an sten:activity is specifically defined to express a relationship between instances of sten:persistent\_object. EXAMPLE 1: Consider the manufacturing sten:activity "F. Bloggs production on 2007-04- 23". This sten:activity creates the sten:physical\_asset with serial number "07/123456". The sten:activity does not create a temporal part of "07/123456", but instead the

sten:persistent\_object which continues to exist after the sten:activity has finished.

EXAMPLE 2 A meeting activity has the roles:

sten:performed\_by identical to sumo:agent. sten:has\_input identical to sumo:resource. sten:has\_output identical to sumo:result.

or sten:has\_part. Examples of this are:

input or output.

role in the sten:activity.

temporal part of the part and of the whole.

*Relationships with parts of life* 

mistaken for objects. This gives the following generic roles:

teller (who counts votes); and

**role definition**

There are many possible roles in an activity, which are specific to the type of activity.

For S-TEN, the roles "input" and "output" have been introduced. The part of speech has also been changed so that the relationships are not identified by nouns and cannot be easily

sten:destroys a sten:has\_input, where the input does not exist after the activity. sten:creates a sten:has\_output, where the output did not exist before the activity.

An sten:activity can have a relationship with an object which is neither sten:has\_role\_player

sten:has\_input\_record and sten:has\_output\_record where a sten:information\_carrier is

sten:investigates where information is sought about an object which may not play a

The relationship sten:component\_of is a sten:simultaneous\_mapping which applies to each

EXAMPLE 1: The statement "*The gearbox with serial number 98/1234 has a sten:component\_of relationship with the car with chassis number 99/4567."* is untrue because the gearbox existed before the car, and may continue to exists after the car if the car is disassembled. If the car

A temporal part can be related to an sten:activity by a sten:has\_part relationship.

sten:causes where one sten:activity causes another to happen;

has the same gearbox throughout its life, then the correct statement is:

*Activity and role* 

 chair person; minute taker;

attender.

Fig**.** 3**.** Top classes in the S-TEN ontology**.** 

Fig. 4. Temporal part and component.

In this example:


## *Activity and role*

172 Semantics in Action – Applications and Scenarios

Fig**.** 3**.** Top classes in the S-TEN ontology**.** 

Fig. 4. Temporal part and component.

relationship with "object X";

 "object a" has a sten:temporal\_part\_of relationship with "object X"; "object b" has a sten:component\_of relationship with "object X";

"object c" has a sten:part\_of (but not sten:temporal\_part\_of or sten:component\_of)

In this example:

An sten:activity (= sumo:Process) is defined in order to identify a purpose, cause or result.

S-TEN defines the sten:has\_role\_player relationship between an sten:activity and a sten:persistent\_object. This is not a relationship with a temporal part because an sten:activity is specifically defined to express a relationship between instances of sten:persistent\_object.

EXAMPLE 1: Consider the manufacturing sten:activity "F. Bloggs production on 2007-04- 23". This sten:activity creates the sten:physical\_asset with serial number "07/123456". The sten:activity does not create a temporal part of "07/123456", but instead the sten:persistent\_object which continues to exist after the sten:activity has finished.

There are many possible roles in an activity, which are specific to the type of activity.

EXAMPLE 2 A meeting activity has the roles:


For S-TEN, the roles "input" and "output" have been introduced. The part of speech has also been changed so that the relationships are not identified by nouns and cannot be easily mistaken for objects. This gives the following generic roles:


A temporal part can be related to an sten:activity by a sten:has\_part relationship.

An sten:activity can have a relationship with an object which is neither sten:has\_role\_player or sten:has\_part. Examples of this are:


#### *Relationships with parts of life*

The relationship sten:component\_of is a sten:simultaneous\_mapping which applies to each temporal part of the part and of the whole.

EXAMPLE 1: The statement "*The gearbox with serial number 98/1234 has a sten:component\_of relationship with the car with chassis number 99/4567."* is untrue because the gearbox existed before the car, and may continue to exists after the car if the car is disassembled. If the car has the same gearbox throughout its life, then the correct statement is:

Intelligent Self-Describing Power Grids 175

A single sten:persistent\_object can have more than one creation and destruction activity

its spatial and temporal boundaries are a choice to support the recording of

A boundary of an sten:ephemeral\_object in time or space is not necessarily obvious to an

it is created by commissioning activities and is destroyed by decomissioning activities.

A sten:facility is operated. The creation and destruction activities are recognised as such by the operator. A single sten:facility can have more than one creation activity because it can

A sten:facility is distinguished from a sten:physical\_asset by the nature of the creation and destruction activities. A sten:facility is created when physical assets are combined into a

The quantity of matter that is a facility can change during the life of the facility, by the removal of some matter and the addition of other matter. It often changes completely, so

EXAMPLE: The feedwater pump for boiler "B\_101" is a facility with tag "P\_101". Operating the boiler involves operating pump "P\_101". The quantity of matter that is pump "P\_101" changes a little whenever pump "P\_101" is maintained, perhaps by replacing a seal or a

If the pump with serial number 98-1234 is installed as "P\_101" until 2006-10-16, and is then replace by the pump with serial number 06-4567, then the quantity of matter that is pump

that all the matter previously present is removed and replaced by different matter.

because can cease to exist for periods of time. <owl:Class rdf:about="&sten;persistent\_object">

</owl:Class>

observer.

</owl:Class>

**facility** 

bearing.

**ephemeral\_object** 

information.

it is an sten:classical\_object;

<rdfs:subClassOf rdf:resource="&sten;classical\_object"/>

An object is an sten:ephemeral\_object if and only if:

<owl:Class rdf:about="&sten;ephemeral\_object"> <rdfs:subClassOf rdf:resource="&sten;classical\_object"/>

An object is a sten:facility if and only if:

it is a sten:persistent\_object;

cease to exist for periods of time.

"P\_101" changes completely on 2006-10-16.

whole that can be operated.

 <owl:sameAs rdf:resource="&iso15926-2;whole\_life\_individual"/> <rdfs:subClassOf rdf:resource="&iso15926-2;physical\_object"/>

<meta:defined\_by rdf:resource="&D2.1;/physical\_things.htm#persistent\_object"/>

<meta:defined\_by rdf:resource="&D2.1;/physical\_things.htm#ephemeral\_object"/>

"A temporal part of the gearbox with serial number 98/1234 has a *sten:component\_of* relationship with the car with chassis number 99/4567." There is no need to be specific about the start and end times of the temporal part of :98/1234, because this can be deduced from the start and end times of :99/4567.

*Ontology for physical thing - taxonomy* 

The following section is an excerpt of the ontology for physical thing and additionally lists some explanations and examples. The ontology is defined in OWL and RDF.

### **physical\_thing**

An object is an sten:physical\_thing if and only if it exists in space and time.

```
<owl:Class rdf:about="&sten;physical_thing"> 
 <owl:sameAs rdf:resource="&sumo;Physical"/> 
 <owl:sameAs rdf:resource="&iso15926-2;possible_individual"/> 
 <meta:defined_by rdf:resource="&D2.1;/physical_things.htm#physical_thing"/> 
</owl:Class>
```
### **classical\_object**

An object is an sten:classical\_object if and only if:


A sten:classical\_object can have more than one creation and destruction activity because can cease to exist for periods of time.

```
<owl:Class rdf:about="&sten;classical_object"> 
 <rdfs:subClassOf rdf:resource="&sten;physical_thing"/> 
 <meta:defined_by rdf:resource="&D2.1;/physical_things.htm#classical_object"/> 
</owl:Class>
```
### **classical\_object\_at\_instant**

An object is an sten:classical\_object\_at\_instant if and only if:


```
<owl:Class rdf:about="&sten;classical_object_at_instant"> 
 <rdfs:subClassOf rdf:resource="&sten;physical_thing"/> 
 <rdfs:subClassOf rdf:resource="&sumo;Object"/> 
 <owl:sameAs rdf:resource="&iso15926-2;event"/> 
 <meta:defined_by rdf:resource="&D2.1;/physical_things.htm#classical_object_at_instant"/> 
</owl:Class>
```
### **persistent\_object**

An object is an sten:persistent\_object if and only if:


A single sten:persistent\_object can have more than one creation and destruction activity because can cease to exist for periods of time.

```
<owl:Class rdf:about="&sten;persistent_object"> 
 <rdfs:subClassOf rdf:resource="&sten;classical_object"/> 
 <owl:sameAs rdf:resource="&iso15926-2;whole_life_individual"/> 
 <rdfs:subClassOf rdf:resource="&iso15926-2;physical_object"/> 
 <meta:defined_by rdf:resource="&D2.1;/physical_things.htm#persistent_object"/> 
</owl:Class>
```
## **ephemeral\_object**

174 Semantics in Action – Applications and Scenarios

"A temporal part of the gearbox with serial number 98/1234 has a *sten:component\_of* relationship with the car with chassis number 99/4567." There is no need to be specific about the start and end times of the temporal part of :98/1234, because this can be deduced

The following section is an excerpt of the ontology for physical thing and additionally lists

some explanations and examples. The ontology is defined in OWL and RDF.

An object is an sten:physical\_thing if and only if it exists in space and time.

<meta:defined\_by rdf:resource="&D2.1;/physical\_things.htm#physical\_thing"/>

<meta:defined\_by rdf:resource="&D2.1;/physical\_things.htm#classical\_object"/>

it is a large number of interacting particles which exists for finite periods of time.

<meta:defined\_by rdf:resource="&D2.1;/physical\_things.htm#classical\_object\_at\_instant"/>

the periods of time during which it exists results from creation and destruction

A sten:classical\_object can have more than one creation and destruction activity because can

from the start and end times of :99/4567.

<owl:Class rdf:about="&sten;physical\_thing"> <owl:sameAs rdf:resource="&sumo;Physical"/>

it is an sten:physical\_thing;

cease to exist for periods of time.

**classical\_object\_at\_instant** 

it is an sten:physical\_thing;

it is an sten:classical\_object;

<owl:Class rdf:about="&sten;classical\_object">

it is a sten:classical\_object at an instant.

<owl:Class rdf:about="&sten;classical\_object\_at\_instant"> <rdfs:subClassOf rdf:resource="&sten;physical\_thing"/> <rdfs:subClassOf rdf:resource="&sumo;Object"/> <owl:sameAs rdf:resource="&iso15926-2;event"/>

An object is an sten:persistent\_object if and only if:

<rdfs:subClassOf rdf:resource="&sten;physical\_thing"/>

An object is an sten:classical\_object\_at\_instant if and only if:

<owl:sameAs rdf:resource="&iso15926-2;possible\_individual"/>

An object is an sten:classical\_object if and only if:

*Ontology for physical thing - taxonomy* 

**physical\_thing** 

</owl:Class> **classical\_object** 

</owl:Class>

</owl:Class>

**persistent\_object** 

activities.

An object is an sten:ephemeral\_object if and only if:


A boundary of an sten:ephemeral\_object in time or space is not necessarily obvious to an observer.

```
<owl:Class rdf:about="&sten;ephemeral_object"> 
 <rdfs:subClassOf rdf:resource="&sten;classical_object"/> 
 <meta:defined_by rdf:resource="&D2.1;/physical_things.htm#ephemeral_object"/> 
</owl:Class>
```
#### **facility**

An object is a sten:facility if and only if:


A sten:facility is operated. The creation and destruction activities are recognised as such by the operator. A single sten:facility can have more than one creation activity because it can cease to exist for periods of time.

A sten:facility is distinguished from a sten:physical\_asset by the nature of the creation and destruction activities. A sten:facility is created when physical assets are combined into a whole that can be operated.

The quantity of matter that is a facility can change during the life of the facility, by the removal of some matter and the addition of other matter. It often changes completely, so that all the matter previously present is removed and replaced by different matter.

EXAMPLE: The feedwater pump for boiler "B\_101" is a facility with tag "P\_101". Operating the boiler involves operating pump "P\_101". The quantity of matter that is pump "P\_101" changes a little whenever pump "P\_101" is maintained, perhaps by replacing a seal or a bearing.

If the pump with serial number 98-1234 is installed as "P\_101" until 2006-10-16, and is then replace by the pump with serial number 06-4567, then the quantity of matter that is pump "P\_101" changes completely on 2006-10-16.

Intelligent Self-Describing Power Grids 177

S-TEN data access is based on semantically annotated Web Services or on agent based systems using as a system component semantically annotated Web Services, too. This approach was chosen, as standard Web Services take into account only the functional or syntactical aspects of a service. This means semantic information is either not available or offered only in an informal and human-interpretable way. It's therefore not possible to derive the intended semantics of input, output, and function or the meaning of parameters of a Web service from its WSDL description only. In that way, two Web services may have the same syntactical definition and, yet, implement different functionality. In this situation, Semantic Web services constitute a promising path to automate the tasks of Web service discovery, composition and invocation and to improve the integration of applications within and across enterprise boundaries. At present, there are three main approaches to enable Semantic Web services: SAWSDL [5], OWL-S [9] and WSMO [10], ranging from pragmatic to more comprehending extensions of standard Web services. Within S-TEN, SAWSDL, as the most pragmatic approach was selected as the only one suitable for implementation at this time, because SAWSDL was supported by the many of the widely used tools of WSDL while the other two had a lack of tools. The basic idea behind SAWSDL is to enhance the WSDL specification with the aim to go beyond a mere syntactical description of a Web service by adding semantics to the functions and the data delivered by a service. As it can be seen in the code excerpt below, featuring a BrowseRequest service, SAWSDL model

references provide the link to the semantics defined in the S-TEN top-level ontology.

<owl:Class rdf:about="&sten;activity">

Fig. 5. Browse Request Service.

</owl:Class>

 <owl:sameAs rdf:resource="&sumo;Process"/> <owl:sameAs rdf:resource="&iso15926-2;activity"/>

**3.3 Semantically annotated web services** 

<rdfs:subClassOf rdf:resource="&sten;ephemeral\_object"/>

<meta:defined\_by rdf:resource="&D2.1;/physical\_things.htm#activity"/>

<owl:Class rdf:about="&sten;facility"> <rdfs:subClassOf rdf:resource="&sten;persistent\_object"/> <meta:defined\_by rdf:resource="&D2.1;/physical\_things.htm#facility"/> </owl:Class>

### **physical\_asset**

An object is a sten:physical\_asset if and only if:


A sten:physical\_asset is owned and maintained. The creation and destruction activities are recognised by the owner or maintainer.

The quantity of matter that is a physical\_asset can change during the life of the physical\_asset, by the removal of some matter and the addition of other matter. There is only rarely a complete change, in which that all the matter previously present is removed and replaced by different matter.

EXAMPLE: The pump with serial number "98-1234" is a physical\_asset. This pump is recorded in the inventory of equipment owned by J. Bloggs and Co..

If the pump with serial number 98-1234 is maintained by replacing the bearings and seals, then the quantity of matter changes, but the physical\_asset continues to exist. Replacing the body of the pump would probably be regarded as a manufacturing a new physical\_asset, rather than a maintenance activity carried out on a existing physical\_asset.

```
<owl:Class rdf:about="&sten;physical_asset"> 
 <rdfs:subClassOf rdf:resource="&sten;persistent_object"/> 
 <meta:defined_by rdf:resource="&D2.1;/physical_things.htm#physical_asset"/> 
</owl:Class>
```
### **period\_of\_life**

An object is a sten:period\_of\_life if and only if:


```
<owl:Class rdf:about="&sten;period_of_life"> 
 <rdfs:subClassOf rdf:resource="&sten;ephemeral_object"/> 
 <rdfs:subClassOf rdf:resource="&iso15926-2;physical_object"/> 
 <meta:defined_by rdf:resource="&D2.1;/physical_things.htm#period_of_life"/> 
</owl:Class>
```
#### **activity**

An object is an sten:activity if and only if:


The ISO 15926 term "activity" is prefered to the SUMO term "process", because it is more general and does not imply that there is an intention or that there is a well defined outcome.

it is created by an assembly or fabrication activity and is ended by a disassembly or

A sten:physical\_asset is owned and maintained. The creation and destruction activities are

The quantity of matter that is a physical\_asset can change during the life of the physical\_asset, by the removal of some matter and the addition of other matter. There is only rarely a complete change, in which that all the matter previously present is removed

EXAMPLE: The pump with serial number "98-1234" is a physical\_asset. This pump is

If the pump with serial number 98-1234 is maintained by replacing the bearings and seals, then the quantity of matter changes, but the physical\_asset continues to exist. Replacing the body of the pump would probably be regarded as a manufacturing a new physical\_asset,

the sten:period during which it exists is a choice to support the recording of

The ISO 15926 term "activity" is prefered to the SUMO term "process", because it is more general and does not imply that there is an intention or that there is a well defined outcome.

<owl:Class rdf:about="&sten;facility">

it is a sten:persistent\_object;

and replaced by different matter.

recognised by the owner or maintainer.

<owl:Class rdf:about="&sten;physical\_asset">

<rdfs:subClassOf rdf:resource="&sten;persistent\_object"/>

 <rdfs:subClassOf rdf:resource="&sten;ephemeral\_object"/> <rdfs:subClassOf rdf:resource="&iso15926-2;physical\_object"/>

An object is a sten:period\_of\_life if and only if:

it is an sten:ephemeral\_object;

<owl:Class rdf:about="&sten;period\_of\_life">

An object is an sten:activity if and only if:

it is an sten:ephemeral\_object;

information.

recycling activity.

</owl:Class> **physical\_asset** 

</owl:Class> **period\_of\_life** 

</owl:Class> **activity** 

<rdfs:subClassOf rdf:resource="&sten;persistent\_object"/>

An object is a sten:physical\_asset if and only if:

<meta:defined\_by rdf:resource="&D2.1;/physical\_things.htm#facility"/>

recorded in the inventory of equipment owned by J. Bloggs and Co..

rather than a maintenance activity carried out on a existing physical\_asset.

<meta:defined\_by rdf:resource="&D2.1;/physical\_things.htm#physical\_asset"/>

<meta:defined\_by rdf:resource="&D2.1;/physical\_things.htm#period\_of\_life"/>

it is defined in order to identify a purpose, cause or result.

<owl:Class rdf:about="&sten;activity"> <rdfs:subClassOf rdf:resource="&sten;ephemeral\_object"/> <owl:sameAs rdf:resource="&sumo;Process"/> <owl:sameAs rdf:resource="&iso15926-2;activity"/> <meta:defined\_by rdf:resource="&D2.1;/physical\_things.htm#activity"/> </owl:Class>

#### **3.3 Semantically annotated web services**

S-TEN data access is based on semantically annotated Web Services or on agent based systems using as a system component semantically annotated Web Services, too. This approach was chosen, as standard Web Services take into account only the functional or syntactical aspects of a service. This means semantic information is either not available or offered only in an informal and human-interpretable way. It's therefore not possible to derive the intended semantics of input, output, and function or the meaning of parameters of a Web service from its WSDL description only. In that way, two Web services may have the same syntactical definition and, yet, implement different functionality. In this situation, Semantic Web services constitute a promising path to automate the tasks of Web service discovery, composition and invocation and to improve the integration of applications within and across enterprise boundaries. At present, there are three main approaches to enable Semantic Web services: SAWSDL [5], OWL-S [9] and WSMO [10], ranging from pragmatic to more comprehending extensions of standard Web services. Within S-TEN, SAWSDL, as the most pragmatic approach was selected as the only one suitable for implementation at this time, because SAWSDL was supported by the many of the widely used tools of WSDL while the other two had a lack of tools. The basic idea behind SAWSDL is to enhance the WSDL specification with the aim to go beyond a mere syntactical description of a Web service by adding semantics to the functions and the data delivered by a service. As it can be seen in the code excerpt below, featuring a BrowseRequest service, SAWSDL model references provide the link to the semantics defined in the S-TEN top-level ontology.

Intelligent Self-Describing Power Grids 179

Fig. 6. Excerpt of ontology for Combined Heat and Power plants.

## **4. S-TEN applications**

Prototype applications have been developed within the S-TEN project validating the S-TEN technology and showing the advantages of the use of web-services, agents and ontologies. Among these prototype applications the following two applications demonstrate the monitoring and control functionality of future power systems:


For these prototypes application-specific ontologies have been defined referencing concepts of the S-TEN top ontology and taking into account existing standards such as IEC 61850 and ISO 15926. The development of these applications has been supported by a user-group set up within the S-TEN project. This user-group, consisting of supply companies, manufacturers and research organizations, assisted in defining the requirements of the various applications thereby ensuring its practical relevance. The following section describes the two applications mentioned above in more detail.

## **4.1 Control of distributed resources in electrical power networks**

The integration of distributed energy resources will influence the structure of future power systems considerably. Power generation will not take place only in a few central power stations but also in a constantly increasing number of distributed energy generators such as wind power plants, photovoltaic plants and combined heat and power plants located in the medium and low voltage system. The prototype application "Control of Distributed Resources in Electrical Power Networks" will enable the monitoring of distributed generators with the aim to improve the management of the distribution network in terms of remote control capabilities and automated network control. The monitoring includes both the observation of measuring values and events (alarms and warnings) and provides event handling. The application has a public presence on the web where its client software can be downloaded at www.s-ten.eu.

### **Ontology**

Based on the S-TEN top-level ontology an application specific ontology has been defined featuring combined heat and power plants (CHP), photovoltaic (PV), wind power devices and additional network devices such as circuit-breakers, inverters, batteries, etc.. An excerpt of the CHP ontology is given in Figure 6.

The Web Services the network resources offer are semantically annotated that is they link to the application ontology and the S-TEN top-level ontology. The XML document (Fig. 7.) shows an example of a browse request and the browse response written according to the S-TEN ontology.

Also, rules definition uses concepts from the S-TEN top-level ontology and the application specific ontology.

### **Architecture and functionality**

A hierarchical architecture has been chosen for the demonstrator (Figure 6). For each DER an S-TEN server has been established. The hierarchical architecture allows adding S-TEN

Prototype applications have been developed within the S-TEN project validating the S-TEN technology and showing the advantages of the use of web-services, agents and ontologies. Among these prototype applications the following two applications demonstrate the

For these prototypes application-specific ontologies have been defined referencing concepts of the S-TEN top ontology and taking into account existing standards such as IEC 61850 and ISO 15926. The development of these applications has been supported by a user-group set up within the S-TEN project. This user-group, consisting of supply companies, manufacturers and research organizations, assisted in defining the requirements of the various applications thereby ensuring its practical relevance. The following section describes

The integration of distributed energy resources will influence the structure of future power systems considerably. Power generation will not take place only in a few central power stations but also in a constantly increasing number of distributed energy generators such as wind power plants, photovoltaic plants and combined heat and power plants located in the medium and low voltage system. The prototype application "Control of Distributed Resources in Electrical Power Networks" will enable the monitoring of distributed generators with the aim to improve the management of the distribution network in terms of remote control capabilities and automated network control. The monitoring includes both the observation of measuring values and events (alarms and warnings) and provides event handling. The application has a public presence on the web where its client software can be

Based on the S-TEN top-level ontology an application specific ontology has been defined featuring combined heat and power plants (CHP), photovoltaic (PV), wind power devices and additional network devices such as circuit-breakers, inverters, batteries, etc.. An excerpt

The Web Services the network resources offer are semantically annotated that is they link to the application ontology and the S-TEN top-level ontology. The XML document (Fig. 7.) shows an example of a browse request and the browse response written according to the S-

Also, rules definition uses concepts from the S-TEN top-level ontology and the application

A hierarchical architecture has been chosen for the demonstrator (Figure 6). For each DER an S-TEN server has been established. The hierarchical architecture allows adding S-TEN

monitoring and control functionality of future power systems:

the two applications mentioned above in more detail.

Control of Distributed Resources in Electrical Power Networks.

**4.1 Control of distributed resources in electrical power networks** 

**4. S-TEN applications** 

Secondary control of microgrids

downloaded at www.s-ten.eu.

of the CHP ontology is given in Figure 6.

**Ontology** 

TEN ontology.

specific ontology.

**Architecture and functionality** 


Fig. 6. Excerpt of ontology for Combined Heat and Power plants.

Intelligent Self-Describing Power Grids 181

**Control System**

**Services Comm Layer Services Layer Services Comm Layer**

**Device Comm Layer Services Layer Services Comm Layer**

**Device Device Device**

S-TEN Registry

The S-TEN server registers its monitored resource in the S-TEN Registry as soon as it is in operation. The registration file contains both the URL of the resource and the services it offers. Each resource sends its data via a communication interface to the S-TEN server where

The registry can be queried for its registered DERs with the option to ask for all DERs, a special DER type or a DER belonging to a certain system area. As result of this query a list of all registered DERs matching the searching criteria is sent back. The user can now select a DER and access it via its URL known from the self-description file held in the S-TEN

Wind power plant A CHP unit A PV plant A

S-TEN Client

S-TEN Server for CHP unit A

**Device Comm Layer Services Layer Services Comm Layer**

**Operator**

**S-Ten System**

S-TEN Server for PV plant A

**S-Ten System**

**S-Ten System**

**Device Comm Layer Services Layer Services Comm Layer**

> S-TEN Server for wind power plant A

Fig. 10. Design of the prototype application.

the data is stored in a database.

registry.

**S-Ten System**

Fig. 9. Hierarchical S-TEN system.

<?xml version="1.0" encoding="UTF-8"?> <BrowseRequest> <SessionID>012154782</SessionID> <ObservedBy>http://www.fgh.de/sten/DER.xml#MDevice\_CHP\_1</ObservedBy> <Observes>http://www.fgh.de/sten/DER.xml#CHP\_1</Observes> <PhysicalProperty>http://www.fgh.de/sten/CHP.xml#ActivePower</PhysicalProperty> <After>1201508970325</ After> <Before>1201512570325</Before> </BrowseRequest>

Fig. 7. Browse request example.

```
<?xml version="1.0" encoding="UTF-8"?> 
<BrowseResponse>
 <ObservedBy>http://www.fgh.de/sten/DER.xml#MDevice_CHP_1</ObservedBy>
 <Observes>http://www.fgh.de/sten/DER.xml#CHP_1</Observes>
 <PhysicalProperty>http://www.labein.es/sten/CHP.xml#Active_Power</PhysicalProperty>
 <Scale>http://www.fgh.de/sten/SIUnits.xml#Watts</Scale>
 <DataTimeValue>
 <TimeSpec>1201508970325</TimeSpec>
 <DecimalValue>25400</DecimalValue>
 </DataTimeValue>
 <DataTimeValue>
 <TimeSpec>1201508932630325</TimeSpec>
 <DecimalValue>30000</DecimalValue>
 </DataTimeValue>
 <DataTimeValue>
 <TimeSpec>1201508932690325</TimeSpec>
 <DecimalValue>31200</DecimalValue>
 </DataTimeValue>
 <BrowseResponseMessage>
 < SuccessIndicator>Browse succeeded</SuccessIndicator>
 </BrowseResponseMessage>
</BrowseResponse>
```
Fig. 8. Browse response example.

systems (servers) that aggregate other S-TEN systems, where the high level S-TEN system provides low level systems' aggregated information and functionalities.

Currently, an operator can access "level 1" S-TEN systems in order to browse data and events and monitor data of a DER. This implies that he only sees the contribution of one DER upstream the feeder per started client application. A future control system embedded in an upper level could aggregate data of all available S-TEN systems within the control area and be in charge of sending the corresponding operation commands to "level 1" S-TEN systems in order to balance the system.

Three servers have been implemented in the application: one for a wind turbine, a second for a PV plant and a third for a CHP unit (Figure 10).

<ObservedBy>http://www.fgh.de/sten/DER.xml#MDevice\_CHP\_1</ObservedBy>

<ObservedBy>http://www.fgh.de/sten/DER.xml#MDevice\_CHP\_1</ObservedBy>

<PhysicalProperty>http://www.labein.es/sten/CHP.xml#Active\_Power</PhysicalProperty>

systems (servers) that aggregate other S-TEN systems, where the high level S-TEN system

Currently, an operator can access "level 1" S-TEN systems in order to browse data and events and monitor data of a DER. This implies that he only sees the contribution of one DER upstream the feeder per started client application. A future control system embedded in an upper level could aggregate data of all available S-TEN systems within the control area and be in charge of sending the corresponding operation commands to "level 1" S-TEN

Three servers have been implemented in the application: one for a wind turbine, a second

provides low level systems' aggregated information and functionalities.

<PhysicalProperty>http://www.fgh.de/sten/CHP.xml#ActivePower</PhysicalProperty>

<Observes>http://www.fgh.de/sten/DER.xml#CHP\_1</Observes>

<Observes>http://www.fgh.de/sten/DER.xml#CHP\_1</Observes>

<Scale>http://www.fgh.de/sten/SIUnits.xml#Watts</Scale>

< SuccessIndicator>Browse succeeded</SuccessIndicator>

<?xml version="1.0" encoding="UTF-8"?>

<SessionID>012154782</SessionID>

 <After>1201508970325</ After> <Before>1201512570325</Before>

Fig. 7. Browse request example.

<?xml version="1.0" encoding="UTF-8"?>

 <TimeSpec>1201508970325</TimeSpec> <DecimalValue>25400</DecimalValue>

 <TimeSpec>1201508932630325</TimeSpec> <DecimalValue>30000</DecimalValue>

 <TimeSpec>1201508932690325</TimeSpec> <DecimalValue>31200</DecimalValue>

<BrowseRequest>

</BrowseRequest>

<BrowseResponse>

<DataTimeValue>

 </DataTimeValue> <DataTimeValue>

 </DataTimeValue> <DataTimeValue>

 </DataTimeValue> <BrowseResponseMessage>

</BrowseResponse>

</BrowseResponseMessage>

Fig. 8. Browse response example.

systems in order to balance the system.

for a PV plant and a third for a CHP unit (Figure 10).

Fig. 9. Hierarchical S-TEN system.

Fig. 10. Design of the prototype application.

The S-TEN server registers its monitored resource in the S-TEN Registry as soon as it is in operation. The registration file contains both the URL of the resource and the services it offers. Each resource sends its data via a communication interface to the S-TEN server where the data is stored in a database.

The registry can be queried for its registered DERs with the option to ask for all DERs, a special DER type or a DER belonging to a certain system area. As result of this query a list of all registered DERs matching the searching criteria is sent back. The user can now select a DER and access it via its URL known from the self-description file held in the S-TEN registry.

Intelligent Self-Describing Power Grids 183

In this application a secondary power-frequency control problem is addressed, such that any variation in the power exchange between the Microgrid and the distribution network is detected and the plan to recover from that situation is generated. Once the plan has been generated, the assigned resources are monitored and any deviation from its scheduled

In order to monitor an industrial process it is necessary to have an operating plan for the physical resources participating in the process, and data from sensors within the system. These are used together to check that everything is taking place according to the plan. Deviations from the plan are detected, and these may require a quick reaction to ensure the safe or economic continuation of the process. The S-TEN project has defined an ontology

This ontology is being used to monitor changes of the output active power of generators within a MicroGrid. The monitoring process has two stages, firstly the measurement of values and their recording using RDF, and secondly the monitoring of these values to make higher level statements about the state of the process which are used by the supervisory

A simple example of monitoring is the detection of changes. The monitoring activity MonitoringActivity\_2\_OfGenerator\_01 is tasked with looking for changes in the output active power of Diesel\_01. The following statements are input to the activity and specify

a sten:TimeVariationOfPhysicalProperty;

A change event is reported (sent as a message or added to the journal formula) if the generator output gets bigger than the 36 kW threshold. Hence the monitoring criterion is

If the measurement crosses this level between 13:10and 13:20 on 2008-07-22, then the

 a sten::MonitoringActivity; sten:monitors :activePowerOfDiesel\_01;

rdfs:subPropertyOf sten:outputActivePower;

 a sten:ChangeCriterion; sten:setlevel [scale:kilowatt 36].

sten:hasMonitoringCriterion :ChangeCriterion1.

behaviour raised and corrected.

 an observation and measurement activity; a time history of a physical property; definition of features of a time history.

:MonitoringActivity\_2\_OfGenerator\_01

The time history activePowerOfDiesel\_01 is defined as:

monitoring activity reports the event as follows:

rdfs:domain :Diesel\_01.

control to modify the operating plan.

:activePowerOfDiesel\_01

**Use of the ontology** 

which covers:

what it does.

defined as follows:

:ChangeCriterion1

Besides, the web services of the selected DER can be viewed and with the help of the semantic web service validator it can be checked if the offered services match the ones the client understands. If the client and server services match, the user can login the DER of interest.

After successful connection with the DER that is after authorisation and authentication the DER tree is displayed revealing the components of the DER, its measured values along with the measurement devices that measure the values and the warnings and alarms available for the measured value. Within the client application data of the DER can then be monitored and browsed and event subscriptions can be performed leading to a notification when the event occurs. When a web service such as "Browse" is selected, a SOAP query is launched in which the data of interest, e.g. the actual power of wind turbine C belonging to wind power plant A, is held. The SOAP query is sent to the S-TEN server of the requested DER which retrieves the data of interest and sends it back to the client. If a control operation is performed (e.g. drive up a CHP), the way of interaction is the same as described for the monitoring data service only that the client gets a status information of the executed service. The responses and requests can be monitored on request.

Event handling is implemented modeling notifications that are sent when warnings and alarms coming from legacy systems occur. E.g. if a wind turbine is equipped with IEC 61850 IEDs (Intelligent Electronic Device) providing alarms, these events will be made available in the prototype application.

Based on semantically interpretable information rules can be defined enabling automatically derivations and subsequent actions, e.g. switching on or off a disconnector within a ring where a short-circuit occured. Besides, observations and data derived from network resources can be linked together and activated with formal rules in order to provide best practise support for the user.

## **4.2 Secondary control of microgrids**

A microgrid can be defined as an integrated Power Delivery system consisting of interconnected loads and distributed energy resources which as an integrated system can operate in parallel with the grid or in an intentional island mode.

It is important for microgrids that the integration of their resources into the LV (Low Voltage) grids, and their relation with the MV (Medium Voltage) network upstream, will contribute to optimize the general operation of the system. For that purpose a hierarchical system control architecture has been designed, that comprises three critical control levels:


In this application a secondary power-frequency control problem is addressed, such that any variation in the power exchange between the Microgrid and the distribution network is detected and the plan to recover from that situation is generated. Once the plan has been generated, the assigned resources are monitored and any deviation from its scheduled behaviour raised and corrected.

#### **Use of the ontology**

182 Semantics in Action – Applications and Scenarios

Besides, the web services of the selected DER can be viewed and with the help of the semantic web service validator it can be checked if the offered services match the ones the client understands. If the client and server services match, the user can login the DER of

After successful connection with the DER that is after authorisation and authentication the DER tree is displayed revealing the components of the DER, its measured values along with the measurement devices that measure the values and the warnings and alarms available for the measured value. Within the client application data of the DER can then be monitored and browsed and event subscriptions can be performed leading to a notification when the event occurs. When a web service such as "Browse" is selected, a SOAP query is launched in which the data of interest, e.g. the actual power of wind turbine C belonging to wind power plant A, is held. The SOAP query is sent to the S-TEN server of the requested DER which retrieves the data of interest and sends it back to the client. If a control operation is performed (e.g. drive up a CHP), the way of interaction is the same as described for the monitoring data service only that the client gets a status information of the executed service.

Event handling is implemented modeling notifications that are sent when warnings and alarms coming from legacy systems occur. E.g. if a wind turbine is equipped with IEC 61850 IEDs (Intelligent Electronic Device) providing alarms, these events will be made available in

Based on semantically interpretable information rules can be defined enabling automatically derivations and subsequent actions, e.g. switching on or off a disconnector within a ring where a short-circuit occured. Besides, observations and data derived from network resources can be linked together and activated with formal rules in order to provide best

A microgrid can be defined as an integrated Power Delivery system consisting of interconnected loads and distributed energy resources which as an integrated system can

It is important for microgrids that the integration of their resources into the LV (Low Voltage) grids, and their relation with the MV (Medium Voltage) network upstream, will contribute to optimize the general operation of the system. For that purpose a hierarchical system control architecture has been designed, that comprises three critical control levels:

 Local Microsource Controller (MC) and Load Controllers (LC): It responds in milliseconds and uses local information and the requests from the microgrid system

 Microgrid system Central Controller: It is the interface with the distribution network and responds to the requests of the Distribution System Operator and optimizes the

 Distribution Management system (DMS): The Distribution Management System is the replica of the Energy Management System (EMS), but for managing distribution

The responses and requests can be monitored on request.

operate in parallel with the grid or in an intentional island mode.

the prototype application.

practise support for the user.

**4.2 Secondary control of microgrids** 

Central Controller (MGCC).

operation of the equipment that it controls.

networks instead of transmission networks.

interest.

In order to monitor an industrial process it is necessary to have an operating plan for the physical resources participating in the process, and data from sensors within the system. These are used together to check that everything is taking place according to the plan. Deviations from the plan are detected, and these may require a quick reaction to ensure the safe or economic continuation of the process. The S-TEN project has defined an ontology which covers:


This ontology is being used to monitor changes of the output active power of generators within a MicroGrid. The monitoring process has two stages, firstly the measurement of values and their recording using RDF, and secondly the monitoring of these values to make higher level statements about the state of the process which are used by the supervisory control to modify the operating plan.

A simple example of monitoring is the detection of changes. The monitoring activity MonitoringActivity\_2\_OfGenerator\_01 is tasked with looking for changes in the output active power of Diesel\_01. The following statements are input to the activity and specify what it does.


The time history activePowerOfDiesel\_01 is defined as:


A change event is reported (sent as a message or added to the journal formula) if the generator output gets bigger than the 36 kW threshold. Hence the monitoring criterion is defined as follows:


If the measurement crosses this level between 13:10and 13:20 on 2008-07-22, then the monitoring activity reports the event as follows:

Intelligent Self-Describing Power Grids 185

DieselEgine\_01 has successfully fulfilled his planned schedule.

Another rule is used to detect that the DieselEgine\_01 can not arrive at its scheduled value and that another generator has to be re-scheduled to cover the difference. In this case, as in the previous rule, design knowledge such as the "ramping" capacity1 of the generator is

a :Plan\_Generator\_01\_1074528964.

Set DieselEgine\_01 as unavailable for further scheduling and

The integration of innovative technologies used by the Semantics Web Community has been proved to be effective for the control of industrial applications and in particular for the control of a MicroGrid. Ontologies derived from standards have been demonstrated to pave the way towards building complex applications by integrating powerful representation mechanisms together with complex decision support systems. The prototype applications have demonstrated that building decision support tools using rule based techniques on top of standard ontologies is a promising way to facilitate the coding of industrial applications

Currently the integration of electrical vehicles into the electrical power system is a challenging task that is worked on worldwide. Controllable electrical vehicles could contribute to a reliable and secure power system operation by charging their batteries when the energy demand is low and feeding back energy when the energy demand is high. Applying Web technologies and agent technologies in combination with ontologies and rules to Vehicle To Grid (V2G) applications is a promising approach for realizing the smart

The ramping capacity represents the amount of Power per Time unit that a generator can raise its

schedule anotherGenerator to generate 36 minus P2 kW.

used to detect the situation. The rule used by the application is as follows:

activePowerOfDieselEgine01\_1074528964 is P1 **and** Current output of DieselEgine\_01 is P2 **and** Ramping capacity of DieselEgine\_01 is R **and**

Current output of DieselEgine\_01 is P2 **and** Ramping capacity of DieselEgine\_01 is R **and**

activePowerOfDieselEgine01\_1074528964\_T

Current Time is T **and**

T < 1074528964 + (36 – P1)/R

This can be reported by the statement:

Current time is T **and**

T - 1074528964 > (36 kW –P1)/R

and to customize them to the user needs.

P2 < 36 kW **and**

P2 >= 36 kW **and**

**Then** 

**IF** 

**Then** 

**5. Outlook** 

grids of the future.

 1

power.


#### **Description of the control scenario**

The technology has been applied for the secondary regulation control of the MicroGrid shown in figure 1. A typical scenario is as follows:


The "Scheduling Monitoring activity" assigned to this task is concerned with the time history of active power of DieselEgine\_01 beginning at the current time and ending when the new required output level has been reached. The application reports when the requirement has been fulfilled, i.e. when a member of the required class has been found, or when it is clear that the requirement cannot be fulfilled.

The specification of the monitoring activity is as follows:


The knowledge stored in the ontology greatly facilitates the development of decision support systems and in this case only one simple rule has to be programmed to detect that DieselEgine\_01 has achieved its objective. At time T, the time history from the start of the plan to T is assessed as follows:

```
If 
activePowerOfDieselEgine01_1074528964 is P1 and
```

```
Current output of DieselEgine_01 is P2 and
Ramping capacity of DieselEgine_01 is R and
Current Time is T and
P2 >= 36 kW and
T < 1074528964 + (36 – P1)/R 
Then 
DieselEgine_01 has successfully fulfilled his planned schedule.
```
This can be reported by the statement:

```
activePowerOfDieselEgine01_1074528964_T 
 a :Plan_Generator_01_1074528964.
```
Another rule is used to detect that the DieselEgine\_01 can not arrive at its scheduled value and that another generator has to be re-scheduled to cover the difference. In this case, as in the previous rule, design knowledge such as the "ramping" capacity1 of the generator is used to detect the situation. The rule used by the application is as follows:

```
IF
```
184 Semantics in Action – Applications and Scenarios

The technology has been applied for the secondary regulation control of the MicroGrid

1. A sudden unbalance between generation and load is dealt within the short term by the modification of the exchange of energy between the MicroGrid and the Distribution

2. The controller of the MicroGrid then reacts to the situation and dispatches (instructs) DieselEgine\_01 to increase power to 36 kW in order to return the system to the normal

3. As soon as the generator is scheduled a "Scheduling Monitoring activity" is started. The main objective of this application is to monitor the ability of DieselEgine\_01 to fulfil its assignment. It may not be able to do so, because of a break down of the generator, a

4. If the scheduling application detects that the plan cannot be fulfilled it reports the

8. ends when a final output of 36 kW has been reached or when the expected time to

The "Scheduling Monitoring activity" assigned to this task is concerned with the time history of active power of DieselEgine\_01 beginning at the current time and ending when the new required output level has been reached. The application reports when the requirement has been fulfilled, i.e. when a member of the required class has been found, or

The knowledge stored in the ontology greatly facilitates the development of decision support systems and in this case only one simple rule has to be programmed to detect that DieselEgine\_01 has achieved its objective. At time T, the time history from the start of the

loss of efficiency caused by the need to support CHP, or some other reason.

5. The plan Plan\_Generator\_01\_1074528964 is defined as the class of time history that:

[sten:partOfTimeVariation : activePowerOfDiesel\_01;

[sten:during [t:hasBeginning [t:inXSDDateTime 2008-07-

[t:hasEnd [t:inXSDDateTime 2008-07-

rdfs:domain

22T13:10]];

22T13:20]]]]

Network.

problem.

 a :ChangeCriterion1. **Description of the control scenario** 

shown in figure 1. A typical scenario is as follows:

balance between generation and load.

6. is the outputActivePower of DieselEgine\_01;

achieve it has been exceeded.

plan to T is assessed as follows:

**If** 

7. begins at the current time – millisecond count 1074528964;

when it is clear that the requirement cannot be fulfilled. The specification of the monitoring activity is as follows:

:MonitoringActivity\_Generator\_01\_1074528964

activePowerOfDieselEgine01\_1074528964 is P1 **and**

 a sten:MonitoringActivity; sten:monitors :activePowerOfDieselEgine\_01; sten:hasMonitoringCriterion :Plan\_Generator\_01\_1074528964.

```
activePowerOfDieselEgine01_1074528964 is P1 and
Current output of DieselEgine_01 is P2 and
Ramping capacity of DieselEgine_01 is R and
Current time is T and
P2 < 36 kW and
T - 1074528964 > (36 kW –P1)/R 
Then 
Set DieselEgine_01 as unavailable for further scheduling and 
schedule anotherGenerator to generate 36 minus P2 kW.
```
## **5. Outlook**

The integration of innovative technologies used by the Semantics Web Community has been proved to be effective for the control of industrial applications and in particular for the control of a MicroGrid. Ontologies derived from standards have been demonstrated to pave the way towards building complex applications by integrating powerful representation mechanisms together with complex decision support systems. The prototype applications have demonstrated that building decision support tools using rule based techniques on top of standard ontologies is a promising way to facilitate the coding of industrial applications and to customize them to the user needs.

Currently the integration of electrical vehicles into the electrical power system is a challenging task that is worked on worldwide. Controllable electrical vehicles could contribute to a reliable and secure power system operation by charging their batteries when the energy demand is low and feeding back energy when the energy demand is high. Applying Web technologies and agent technologies in combination with ontologies and rules to Vehicle To Grid (V2G) applications is a promising approach for realizing the smart grids of the future.

 1 The ramping capacity represents the amount of Power per Time unit that a generator can raise its power.

**Section 3** 

**Visualization** 

## **6. Acknowledgment**

The research described in the paper has been carried out within the S-TEN project ('Intelligent Self-describing Technical and Environmental Networks', www.s-ten.eu, FP6- IST-2005-027683) co-funded by the European Community in the Information Society Technologies program.

## **7. References**


## **Section 3**

186 Semantics in Action – Applications and Scenarios

The research described in the paper has been carried out within the S-TEN project ('Intelligent Self-describing Technical and Environmental Networks', www.s-ten.eu, FP6- IST-2005-027683) co-funded by the European Community in the Information Society

[1] S-TEN Project: Intelligent Self-describing Technical and Environmental Networks,

[2] Berners-Lee, T. Handler, J., Lassila, O. (2001). The Semantic Web, In: *Scientific American Magazine,* Available from: http://www.dblab.ntua.gr/~bikakis/SW.pdf

[4] W3C: Web Services Description Language (WSDL) 1.1, Available from:

[5] W3C: Semantic Annotations for WSDL, Available from:

[6] W3C: OWL Web Ontology Language - W3C Recommendation 10 February 2004,

[7] W3C: RDF Vocabulary Description Language 1.0: RDF Schema W3C Recommendation 10 February 2004, Available from: http://www.w3.org/TR/rdf-schema/ [8] S-TEN consortium( 2007): Ontology for self-describing networks (D2.1), Available from:

[9} W3C: OWL-S: Semantic Markup for Web Services - W3C Member Submission 22 November 2004, Available from: http://www.w3.org/Submission/OWL-S/ [10] W3C: Web Service Modeling Ontology (WSMO) - W3C Member Submission 3 June

2005, Available from: http://www.w3.org/Submission/WSMO/

**6. Acknowledgment** 

Technologies program.

Available from: www.s-ten.eu

http://axis.apache.org/axis2/java/core/

http://www.w3.org/TR/wsdl

[3] The Apache Software Foundation: AXIS 2, Available from:

http://www.w3.org/2002/ws/sawsdl/

http://www.s-ten.eu/deliverables/D2.1/.

Available from: http://www.w3.org/TR/owl-features/

**7. References** 

**Visualization** 

**8** 

*1Kyoto University 2Ehime University*

*Japan*

*3Nagoya Institute of Technology*

**Facet Decomposition and Discouse Analysis:** 

Public debate has become an important process in the planning of public projects where stakeholders including citizens, experts, enterprises, and administrators meet face-to-face to discuss the desirability of a public project in its early planning stages. The roles of public debate are: to clarify the pros and cons of a project, that is, evaluate available information and proposed approaches; to promote mutual understanding of the stakeholders' different

The stakeholders or debate participants have varying values, concerns, and expectations on the benefits and costs of a project and as such, their interests do not necessarily coincide with or complement each other. To promote their interests, debate participants strategically reiterate and legitimize their opinions. The debate facilitator must identify appropriate methods for recognizing and managing uncooperative communication in order for a debate to arrive at a fair consensus. To facilitate effective group communication such as debates, it is important to develop scientific management tools for managing debate content and

Advances in information technology has led to the development of various debate support systems based on corpus linguistics (Shiramatsu *et al*., 2007; Jeong *et al*. 2007) designed to assist facilitators and participants in achieving clear communication and arriving at a consensus by analyzing information from actual dialogue data. Unfortunately, these systems are yet to be fully applied to studies in public discourse. There have been few studies that examined the effectiveness of debate support systems in alleviating stakeholder interest conflict. In addition, systematic and objective means for summarizing and analyzing debate content have not been sufficiently developed. In order to effectively and efficiently manage public debate, techniques that enable deeper understanding of participant opinions

This study proposes a method for visualizing conflict structures in public debates by analyzing debate minutes using a corpus-based discourse analysis. In this method, the opinion of participants recorded in the debate minutes (hereafter called "primary corpus")

perspectives; and to find resolutions that are mutually beneficial for all stakeholders.

**1. Introduction** 

conflict situations.

and management of conflict are essential.

**Visualization of Conflict Structure** 

Hayeong Jeong1, Kiyoshi Kobayashi1, Tsuyoshi Hatori2 and Shiramatsu Shun3

## **Facet Decomposition and Discouse Analysis: Visualization of Conflict Structure**

Hayeong Jeong1, Kiyoshi Kobayashi1,

Tsuyoshi Hatori2 and Shiramatsu Shun3 *1Kyoto University 2Ehime University 3Nagoya Institute of Technology Japan*

## **1. Introduction**

Public debate has become an important process in the planning of public projects where stakeholders including citizens, experts, enterprises, and administrators meet face-to-face to discuss the desirability of a public project in its early planning stages. The roles of public debate are: to clarify the pros and cons of a project, that is, evaluate available information and proposed approaches; to promote mutual understanding of the stakeholders' different perspectives; and to find resolutions that are mutually beneficial for all stakeholders.

The stakeholders or debate participants have varying values, concerns, and expectations on the benefits and costs of a project and as such, their interests do not necessarily coincide with or complement each other. To promote their interests, debate participants strategically reiterate and legitimize their opinions. The debate facilitator must identify appropriate methods for recognizing and managing uncooperative communication in order for a debate to arrive at a fair consensus. To facilitate effective group communication such as debates, it is important to develop scientific management tools for managing debate content and conflict situations.

Advances in information technology has led to the development of various debate support systems based on corpus linguistics (Shiramatsu *et al*., 2007; Jeong *et al*. 2007) designed to assist facilitators and participants in achieving clear communication and arriving at a consensus by analyzing information from actual dialogue data. Unfortunately, these systems are yet to be fully applied to studies in public discourse. There have been few studies that examined the effectiveness of debate support systems in alleviating stakeholder interest conflict. In addition, systematic and objective means for summarizing and analyzing debate content have not been sufficiently developed. In order to effectively and efficiently manage public debate, techniques that enable deeper understanding of participant opinions and management of conflict are essential.

This study proposes a method for visualizing conflict structures in public debates by analyzing debate minutes using a corpus-based discourse analysis. In this method, the opinion of participants recorded in the debate minutes (hereafter called "primary corpus")

Facet Decomposition and Discouse Analysis: Visualization of Conflict Structure 191

difficult to implement than machine learning approaches. Recognizing Textual Entailment (RTE) Challenges (Dagan, 2006; PASCAL, 2005) is a method that focuses on the semantic relationships between texts to determine whether the meaning of one text is derived from another text. Although the RTE datasets included semantic relations such as agreement and conflict, Marnefe (2008) found that opinions in real world debates were much more difficult to classify and as such, more sophisticated classification methods are required for RTE to work in the real world. Cross-document Structure Theory (CST) is another approach for identifying semantic relationships between sentences (Radev, 2000). The CSTBank Corpus built by Radev (2000) is annotated with 18 kinds of semantic relationships including agreement and conflict. As with the RTE, the CST approach does not effectively classify

The rules-based approaches discussed above clarify linguistic characteristics using detailed categorizations and rules to ensure accuracy in analysis. However, they are limited in that they are difficult to implement and their classifications are too simplified to accurately classify more complicated opinions in real-world debates. This study proposes a simpler, more effective approach based on machine learning techniques for visualizing conflict structures in terms of rapid prototyping. Machine learning approaches are more effective and perform better than rules-based approaches. We then apply this approach to a series of real public debates where participants with different backgrounds argue their opinions.

Debate participants' different cognitive frames and subjective interests lead to differing opinions. In order to effectively summarize the debate contents, we need to maintain the relevance between textual and contextual information using qualitative and descriptive analyses (Hashiuchi, 2003). In this study, we used the method of content analysis proposed by Stone *et al*. (1966). Content analysis refers to ". . . any research technique for making inferences by systematically and objectively identifying specified characteristics within text" (Stone *et al*., 1966). The purpose of content analysis is to examine the corpus data of dialogue (e.g., debate minutes), taking into consideration the relevance between the words and the context embedded in the dialogue. Krippendorff (1980) identified the three most important features of content analysis: it is an *unobtrusive* technique; it can assess *unstructured* material; and is *context-sensitive*. These features allow researchers to examine relatively unstructured data for meanings, symbolism, expressed information, and the role this information plays in the lives of the data sources, while at the same time acknowledging the textuality of the data, that is, recognizing that the data are read and understood by others and interpreting

Discourse analysis is a type of content analysis that clarifies debate content and structure by targeting language use in the real world (Schiffrin *et al.,* 2003). Discourse analysis reveals the structural and functional mechanism of language in relation to the utterances in their context. Discourse has traditionally been defined as "anything beyond the sentence." Schiffrin (1994) referred to "utterances" rather then "sentences" because utterances are *contextualized* sentences, that is, they are context- and text-bound. In discourse analysis, context refers to the information that surrounds a sentence and determines its meaning, and includes a broad range of information such as the speaker's knowledge, beliefs, facial

this data based their own contexts (Koreniusa *et al*., 2007).

expressions, gestures, and social circumstance, and culture.

opinions in debates.

**2.2 Content Analysis** 

are decomposed into four facets based on the following defined conceptual categories debate purpose, means, evidences, and attitudes toward the subject project. Using this facets-tagged corpus (hereafter called "secondary corpus"), this study quantitatively measures the content similarity among the recorded opinions and the level of interest dissonance among the participants. The rest of this paper is organized as follows. Section 2 describes the basic idea of the study. Section 3 explains the definition and decomposition of the facets using a support vector machine (SVM). Section 4 presents the discourse analysis using the secondary corpus and visualizes the conflict structure and debate pattern changes. The applicability of the proposed methodology is then discussed by applying it to a real case of public debate. Section 5 presents the conclusions and limitations of the study.

## **2. The basic idea**

#### **2.1 Related studies**

Previous studies on debate support systems have focused on summarizing debate contents by extracting important topics and exploring semantic meanings using statistical indicators of term frequency and co-occurency (Horita and Kanno, 2001; Hiromi *et al*, 2006). While these techniques clarify the main topic of debate and participants' interests from a debate minute using natural language processing and text mining, they do not identify conflict structures and underlying dynamic interaction in a multiple debate process.

Nasukawa and Yi (2003) proposed a sentiment analysis approach for extracting polar opposite sentiments of subjects from a document by identifying the semantic relationship between the subject term and semantic expressions such as "good" and "bad," instead of classifying the entire document as positive or negative. By defining the terms of sentiment expression, it is possible to calculate an individual's attitude toward the subject term and determine whether participants in a discussion have different attitudes toward to subject term While this method certainly has its merits, it is unable to determine the reason for the differences in attitude and the underlying relationships in the debate. In order to enhance mutual understanding and consensus building in debates, it is necessary to clarify the conflict structure and its change throughout multiple debates, evaluate the significance of debate context on participants' sentiments, and develop facilitation techniques for managing conflict structures.

Opinion mining is a relatively new research field that includes review analysis, opinion extraction, and answering opinion questions. Opinion mining utilizes natural language processing techniques and facilitates deep semantic classification of opinions. Sophisticated techniques for deep semantic classification have not yet been developed for and applied to public debates as they require substantial linguistic knowledge and the ability to process additional non-literal information such as heuristic rules. The rules-based approaches to opinion mining also have some limitations. Somasundaran and Wiebe (2010) examined opinions in ideological online debates by developing an argument lexicon using the Multi-Perspective Question Answering (MPQA) annotations. Meanwhile, the Statement Map (Murakami *et al*., 2010) and WISDOM (Akamine *et al.*, 2009) methods focus on analyzing information credibility through opinion mining techniques. These methods classify semantic relationships between subjective statements such as agreement and conflict relationships using rules-based approaches that work well with experimental data but generally are more difficult to implement than machine learning approaches. Recognizing Textual Entailment (RTE) Challenges (Dagan, 2006; PASCAL, 2005) is a method that focuses on the semantic relationships between texts to determine whether the meaning of one text is derived from another text. Although the RTE datasets included semantic relations such as agreement and conflict, Marnefe (2008) found that opinions in real world debates were much more difficult to classify and as such, more sophisticated classification methods are required for RTE to work in the real world. Cross-document Structure Theory (CST) is another approach for identifying semantic relationships between sentences (Radev, 2000). The CSTBank Corpus built by Radev (2000) is annotated with 18 kinds of semantic relationships including agreement and conflict. As with the RTE, the CST approach does not effectively classify opinions in debates.

The rules-based approaches discussed above clarify linguistic characteristics using detailed categorizations and rules to ensure accuracy in analysis. However, they are limited in that they are difficult to implement and their classifications are too simplified to accurately classify more complicated opinions in real-world debates. This study proposes a simpler, more effective approach based on machine learning techniques for visualizing conflict structures in terms of rapid prototyping. Machine learning approaches are more effective and perform better than rules-based approaches. We then apply this approach to a series of real public debates where participants with different backgrounds argue their opinions.

## **2.2 Content Analysis**

190 Semantics in Action – Applications and Scenarios

are decomposed into four facets based on the following defined conceptual categories debate purpose, means, evidences, and attitudes toward the subject project. Using this facets-tagged corpus (hereafter called "secondary corpus"), this study quantitatively measures the content similarity among the recorded opinions and the level of interest dissonance among the participants. The rest of this paper is organized as follows. Section 2 describes the basic idea of the study. Section 3 explains the definition and decomposition of the facets using a support vector machine (SVM). Section 4 presents the discourse analysis using the secondary corpus and visualizes the conflict structure and debate pattern changes. The applicability of the proposed methodology is then discussed by applying it to a real case

Previous studies on debate support systems have focused on summarizing debate contents by extracting important topics and exploring semantic meanings using statistical indicators of term frequency and co-occurency (Horita and Kanno, 2001; Hiromi *et al*, 2006). While these techniques clarify the main topic of debate and participants' interests from a debate minute using natural language processing and text mining, they do not identify conflict

Nasukawa and Yi (2003) proposed a sentiment analysis approach for extracting polar opposite sentiments of subjects from a document by identifying the semantic relationship between the subject term and semantic expressions such as "good" and "bad," instead of classifying the entire document as positive or negative. By defining the terms of sentiment expression, it is possible to calculate an individual's attitude toward the subject term and determine whether participants in a discussion have different attitudes toward to subject term While this method certainly has its merits, it is unable to determine the reason for the differences in attitude and the underlying relationships in the debate. In order to enhance mutual understanding and consensus building in debates, it is necessary to clarify the conflict structure and its change throughout multiple debates, evaluate the significance of debate context on participants' sentiments, and develop facilitation techniques for managing

Opinion mining is a relatively new research field that includes review analysis, opinion extraction, and answering opinion questions. Opinion mining utilizes natural language processing techniques and facilitates deep semantic classification of opinions. Sophisticated techniques for deep semantic classification have not yet been developed for and applied to public debates as they require substantial linguistic knowledge and the ability to process additional non-literal information such as heuristic rules. The rules-based approaches to opinion mining also have some limitations. Somasundaran and Wiebe (2010) examined opinions in ideological online debates by developing an argument lexicon using the Multi-Perspective Question Answering (MPQA) annotations. Meanwhile, the Statement Map (Murakami *et al*., 2010) and WISDOM (Akamine *et al.*, 2009) methods focus on analyzing information credibility through opinion mining techniques. These methods classify semantic relationships between subjective statements such as agreement and conflict relationships using rules-based approaches that work well with experimental data but generally are more

of public debate. Section 5 presents the conclusions and limitations of the study.

structures and underlying dynamic interaction in a multiple debate process.

**2. The basic idea 2.1 Related studies** 

conflict structures.

Debate participants' different cognitive frames and subjective interests lead to differing opinions. In order to effectively summarize the debate contents, we need to maintain the relevance between textual and contextual information using qualitative and descriptive analyses (Hashiuchi, 2003). In this study, we used the method of content analysis proposed by Stone *et al*. (1966). Content analysis refers to ". . . any research technique for making inferences by systematically and objectively identifying specified characteristics within text" (Stone *et al*., 1966). The purpose of content analysis is to examine the corpus data of dialogue (e.g., debate minutes), taking into consideration the relevance between the words and the context embedded in the dialogue. Krippendorff (1980) identified the three most important features of content analysis: it is an *unobtrusive* technique; it can assess *unstructured* material; and is *context-sensitive*. These features allow researchers to examine relatively unstructured data for meanings, symbolism, expressed information, and the role this information plays in the lives of the data sources, while at the same time acknowledging the textuality of the data, that is, recognizing that the data are read and understood by others and interpreting this data based their own contexts (Koreniusa *et al*., 2007).

Discourse analysis is a type of content analysis that clarifies debate content and structure by targeting language use in the real world (Schiffrin *et al.,* 2003). Discourse analysis reveals the structural and functional mechanism of language in relation to the utterances in their context. Discourse has traditionally been defined as "anything beyond the sentence." Schiffrin (1994) referred to "utterances" rather then "sentences" because utterances are *contextualized* sentences, that is, they are context- and text-bound. In discourse analysis, context refers to the information that surrounds a sentence and determines its meaning, and includes a broad range of information such as the speaker's knowledge, beliefs, facial expressions, gestures, and social circumstance, and culture.

Facet Decomposition and Discouse Analysis: Visualization of Conflict Structure 193

Debate participants are interested only in certain aspects of an issue, and as such, are motivated to steer the discussion toward their desired goals. Debate participants such as experts and engineers usually evaluate the benefits of the projects based on scientific and professional evidences highlighting rigor, while general citizens evaluate them based on their common sense and their interests, checking for adequacy. If debate participants consider only one aspect of the issue, that is, either rigor or adequacy only, the debate would not reach a consensus. The most important role of public debate involving the third-party committee is to achieve a common perspective on the project under discussion by recognizing each stakeholder's interest and point of view. To evaluate the progress of the debate toward mutual understanding and consensus, it is necessary to accumulate the debate minutes and summarize the discussions. This study proposes a method of discourse analysis that manages public debate by summarizing the debate contents and tracking the

In this study, we define utterance as the language used to establish one's position on an issue by addressing the position's pros and cons. During a debate, participants legitimize their positions with technical and scientific evidences or their common sense and selfinterests. They strengthen their positions by expressing positive or negative attitudes toward another participant's utterances depending on whether those utterances coincide or contrast their positions. Based on these assumptions, we identified the four facets to be used in this study: facet A refers the positions of participants or their pros and cons on the issue; facet B refers their efforts to strengthen their positions using evidence; facet C refers the characteristics of evidences, that is, whether they pertain to either adequacy or rigor; and facet D refers to the participants' interpretation of each other's utterances, that is, their expression of positive or negative attitudes toward each other's utterances. We coded the utterances using a combination of four facets (i.e., stractable). In this study, the utterances in the debate minutes comprise the primary corpus and the utterances encoded with facet stractables comprise the secondary corpus. Utterances that do not explicitly pertain to a position are considered neutral and as such, classified under a third category for each facet.

In order to create the secondary corpus, we encoded the utterances with the facet stractables. Owing to the enormous amount of data in the primary corpus, decomposing the utterances based on facets is not easy. As such, we created the secondary corpus using a statistical facet learning model that reduces the efforts required of researcher while at the same time maintaining the logical consistency of the primary corpus. This statistical facet learning model is based on a pattern recognition technique that classifies the huge amount of primary corpus into the secondary corpus through the SVM, a type of statistical learning machine. Pattern recognition is generally defined as a method for extracting data features such as those in letters, video, and audio and determining the data categories based on the standard pattern of the features. This study decomposed the facets using the SVM model originally developed by Vapnik (see Kudo, 2002). The SVM is a binary sorter where there is only one border identifying a sample's location. Because there are three categories for each

progress of the debate process toward its goal.

Section 3.2 presents the facet stractables in more detail.

**3.1 Facet decomposition and the Support Vector Machine (SVM)** 

**3. Creating the facet-tagged corpus** 

Hatori *et al*. (2006) suggested a discourse analysis based on facet theory. In this method, utterances in a debate are decomposed into three conceptual units (hereafter called "facets")—"what" should be solved, "how" it should be solved, and "why" it should be solved, in order to effectively examine interest conflict and cognitive dissonance among debate participants. This facet decomposition task depends on how researchers analyze and interpret the utterances in a debate. However, to be able to analyze multiple debates with huge amounts of language data, improved reproducibility and computational approaches are required (Francis, 1982).

This study develops a computational method of facet decomposition and discourse analysis based on corpus linguistics. Corpus is generally defined as a collection of text or written and spoken material that highlights how a language is used. Corpus linguistics is a linguistic method that explains the meanings of words and phrases using a corpus. It is a reconstructive method for analysis of language data using a computer (Sinclair, 1991; Stubbs, 2002; Wang, 2005; McCarthy, 1998). In this study we use SVM learning to decompose debate utterances into facets. Then, using this facets-tagged corpus, we propose an approach for calculating the differences among the utterances of the debate participants and visualize their dynamic change.

### **2.3 Management of public debate**

A public debate is essentially a third-party committee where stakeholders, including experts, public enterprises, and citizens, share their perspectives on public projects in their early planning stage. The purpose of this third-party committee is to promote mutual understanding among the debate participants and ensure transparency of the debate system by opening the discussion process to the various stakeholders and making available information on related projects. The third-party committee is not involved in the decisionmaking, but rather in the collection of information related to public project and the building of public trust in the decision makers (Hatori *et al*., 2008).

The third-party committee is directly entrusted by the government and the citizens and as such, must suggest solutions that are socially desirable and beneficial to the government and its citizens. As representatives of the government and its citizens, committee members must assess solutions based on their social perspective (i.e., check for adequacy) and technical and professional perspectives (i.e., check for rigor). In short, the third-party committee is expected to assess project-related information for adequacy and rigor to facilitate decision making.

Previous studies have used questionnaires and interviews to understand the perspectives of stakeholders on public projects. According to Giddens (1990), these methods are based on the segmented, asynchronous communication between the stakeholders and thus suffer from the problem of faceless commitments. These methods are limited in that questionnaire may contain the subjective perspectives of the researchers and as such both the researcher and participants readily trust the questions and answers on the questionnaire, respectively. In addition, the questionnaire may contain the assumptions and perspectives of the researchers on the projects which the respondents or citizens do not necessarily recognize. In contrast, a debate within a third-party committee realizes the richness of social communication facilitated by the direct meeting among stakeholders. Such debates ensure the effective collection of information and transparency in the decision-making process.

Debate participants are interested only in certain aspects of an issue, and as such, are motivated to steer the discussion toward their desired goals. Debate participants such as experts and engineers usually evaluate the benefits of the projects based on scientific and professional evidences highlighting rigor, while general citizens evaluate them based on their common sense and their interests, checking for adequacy. If debate participants consider only one aspect of the issue, that is, either rigor or adequacy only, the debate would not reach a consensus. The most important role of public debate involving the third-party committee is to achieve a common perspective on the project under discussion by recognizing each stakeholder's interest and point of view. To evaluate the progress of the debate toward mutual understanding and consensus, it is necessary to accumulate the debate minutes and summarize the discussions. This study proposes a method of discourse analysis that manages public debate by summarizing the debate contents and tracking the progress of the debate process toward its goal.

## **3. Creating the facet-tagged corpus**

192 Semantics in Action – Applications and Scenarios

Hatori *et al*. (2006) suggested a discourse analysis based on facet theory. In this method, utterances in a debate are decomposed into three conceptual units (hereafter called "facets")—"what" should be solved, "how" it should be solved, and "why" it should be solved, in order to effectively examine interest conflict and cognitive dissonance among debate participants. This facet decomposition task depends on how researchers analyze and interpret the utterances in a debate. However, to be able to analyze multiple debates with huge amounts of language data, improved reproducibility and computational approaches

This study develops a computational method of facet decomposition and discourse analysis based on corpus linguistics. Corpus is generally defined as a collection of text or written and spoken material that highlights how a language is used. Corpus linguistics is a linguistic method that explains the meanings of words and phrases using a corpus. It is a reconstructive method for analysis of language data using a computer (Sinclair, 1991; Stubbs, 2002; Wang, 2005; McCarthy, 1998). In this study we use SVM learning to decompose debate utterances into facets. Then, using this facets-tagged corpus, we propose an approach for calculating the differences among the utterances of the debate participants

A public debate is essentially a third-party committee where stakeholders, including experts, public enterprises, and citizens, share their perspectives on public projects in their early planning stage. The purpose of this third-party committee is to promote mutual understanding among the debate participants and ensure transparency of the debate system by opening the discussion process to the various stakeholders and making available information on related projects. The third-party committee is not involved in the decisionmaking, but rather in the collection of information related to public project and the building

The third-party committee is directly entrusted by the government and the citizens and as such, must suggest solutions that are socially desirable and beneficial to the government and its citizens. As representatives of the government and its citizens, committee members must assess solutions based on their social perspective (i.e., check for adequacy) and technical and professional perspectives (i.e., check for rigor). In short, the third-party committee is expected to assess project-related information for adequacy and rigor to facilitate decision

Previous studies have used questionnaires and interviews to understand the perspectives of stakeholders on public projects. According to Giddens (1990), these methods are based on the segmented, asynchronous communication between the stakeholders and thus suffer from the problem of faceless commitments. These methods are limited in that questionnaire may contain the subjective perspectives of the researchers and as such both the researcher and participants readily trust the questions and answers on the questionnaire, respectively. In addition, the questionnaire may contain the assumptions and perspectives of the researchers on the projects which the respondents or citizens do not necessarily recognize. In contrast, a debate within a third-party committee realizes the richness of social communication facilitated by the direct meeting among stakeholders. Such debates ensure the effective collection of information and transparency in the decision-making process.

are required (Francis, 1982).

and visualize their dynamic change.

**2.3 Management of public debate** 

making.

of public trust in the decision makers (Hatori *et al*., 2008).

#### **3.1 Facet decomposition and the Support Vector Machine (SVM)**

In this study, we define utterance as the language used to establish one's position on an issue by addressing the position's pros and cons. During a debate, participants legitimize their positions with technical and scientific evidences or their common sense and selfinterests. They strengthen their positions by expressing positive or negative attitudes toward another participant's utterances depending on whether those utterances coincide or contrast their positions. Based on these assumptions, we identified the four facets to be used in this study: facet A refers the positions of participants or their pros and cons on the issue; facet B refers their efforts to strengthen their positions using evidence; facet C refers the characteristics of evidences, that is, whether they pertain to either adequacy or rigor; and facet D refers to the participants' interpretation of each other's utterances, that is, their expression of positive or negative attitudes toward each other's utterances. We coded the utterances using a combination of four facets (i.e., stractable). In this study, the utterances in the debate minutes comprise the primary corpus and the utterances encoded with facet stractables comprise the secondary corpus. Utterances that do not explicitly pertain to a position are considered neutral and as such, classified under a third category for each facet. Section 3.2 presents the facet stractables in more detail.

In order to create the secondary corpus, we encoded the utterances with the facet stractables. Owing to the enormous amount of data in the primary corpus, decomposing the utterances based on facets is not easy. As such, we created the secondary corpus using a statistical facet learning model that reduces the efforts required of researcher while at the same time maintaining the logical consistency of the primary corpus. This statistical facet learning model is based on a pattern recognition technique that classifies the huge amount of primary corpus into the secondary corpus through the SVM, a type of statistical learning machine. Pattern recognition is generally defined as a method for extracting data features such as those in letters, video, and audio and determining the data categories based on the standard pattern of the features. This study decomposed the facets using the SVM model originally developed by Vapnik (see Kudo, 2002). The SVM is a binary sorter where there is only one border identifying a sample's location. Because there are three categories for each

Facet Decomposition and Discouse Analysis: Visualization of Conflict Structure 195

groups according to their roles: facilitator; expert; citizen with a "con" opinion; citizen with a "pro" opinion; and administrator. The minutes from the 14 debates consisted of 8,831

Four researchers from construction consulting companies with experience in mobilizing public involvement made a training set of facet decomposition. To maintain the objectivity of the training set, two researchers conducted facet decomposition on the same utterances. We completed the training set the corresponding facet decomposition. Table 1 shows the

An utterance is coded as facet A(1) if it describes the pros of the project. Statements about the advantages of the project are examples of A(1). Meanwhile, disapproving opinions are classified as facet A(-1) and include statements that mention the disadvantages of the project and suggest new alternatives. Utterances that include evidences are coded as facet B(1) and those without evidences as B(-1). Utterances with rigorous or adequate evidences are coded as C(1) or C(-1), respectively. Facet C(1) includes statements that cite evidences from studies or experiments while facet C(-1) includes statements with evidences based on individual experiences or feelings. Finally, facet D(1) represents positive attitudes toward the project, signaling affirmation or belief in the project. Facet D(-1) represents negative attitudes toward the project. Conflicts can be identified by comparing the difference between the responses in facet A and those in facet D. Facet B determines the plausibility of arguments while facet C sheds light on the participants' reasoning systems. If an utterance cannot be easily coded using the above facets, it is classified under the neutral

utterances. A unit of utterance is a single sentence ended by punctuation mark.

framework of the four facets with the facet elements and the typical language use.

Below are two examples of how utterances are encoded using the facets framework:

speaker's reason for his negative view and thus is denoted as B(1), C(1), and D(-1).

utterence excepting facet A. The training utterance set determines the statistical facet

given the limited public finances. → A (1) B (1) C (-1) D (-1)

only a few of them succeeded. → A (-1) B (1) C (1) D (-1)

Example 1: The dam construction is the only way to ensure the safety of the town's residents

Example 2: There have been many regional development projects that constructed dams but

The first utterance example is decomposed into facet A(1)B(1)C(-1)D(-1). The expression "is the only way" indicates that the speaker agrees to the dam construction, hence, A (1), but does not have a positive attitude toward it, hence, D(-1). The expression "given the limited public finances" is an evidence and is therefore classified as B(1); however, this evidence is also based on the speaker's anxiety and concern, and as such, is classified as C(-1). Meanwhile, the second utterance example expresses pessimism regarding the dam construction, hence, A(-1). The phrase "only a few of them have succeeded" expresses the

As mentioned earlier, researchers created and completed the training utterance set of facet decomposition. Of the total 8,831 utterances, 34% were decomposed as facet A, 65% as facet B, 59% as facet C, and 52% as facet D. Over half of the utterances are selected as training

**3.3 Outline of the facet decomposition** 

category, *X*(0) , {,,,} *X ABC D* .

 facet, that is, *X ABCD* {,,,} , this study uses two bounding hyperplanes ( ),( 1,2) *<sup>X</sup> h i yUh* . These bounding hyperplanes are defined by a high-dimensional discrimination function that determines the amount of terms included in the primary corpus and maximizes the distance between the border and each utterance. Although the discrimination function is usually represented by a nonlinear kernel function, we used a d-dimensional polynomial function that flexibly approximates the function. The SVM model used in this study is able to process large amounts of data and produces good estimation results. In this model, the utterances in the primary corpus, ( 1, , ) *Ui m <sup>i</sup>* , are each classified into one of the three categories of facets, ( ) { ( 1), (0), (1)} *<sup>X</sup> FU X X X <sup>i</sup>* . ( ) *<sup>X</sup> F Ui* determines the corresponding relationship and is defined as:

$$F^X: \mathcal{U}\_i \mapsto \{X(-1), X(0), X(1)\}\tag{1}$$

$$\{X \in \{A, B, C, D\}\}$$

The two types of kernel functions, 1 ( ) *<sup>X</sup> <sup>i</sup> y U* , <sup>2</sup> ( ) *<sup>X</sup> <sup>i</sup> <sup>y</sup> <sup>U</sup>* for the facets *X ABCD* {,,,} , with *<sup>m</sup>* units of utterances ( 1, , ) *Ui m <sup>i</sup>* which are located in the *n* dimensional space ( *n* is the total number of terms in the primary corpus) are defined as follows:

$$\begin{aligned} \text{i.i.} \qquad &F^X(\mathcal{U}\_i) = X(1) \text{ if } \mathcal{Y}\_1 \, ^X(\mathcal{U}\_i) > 0 \, ; \\ \text{ii.} \qquad &F^X(\mathcal{U}\_i) = X(0) \text{ if } \, \mathcal{Y}\_1 \, ^X(\mathcal{U}\_i) \le 0 \text{ and } \, \mathcal{Y}\_2 \, ^X(\mathcal{U}\_j) > 0 \, ; \text{ and } \\ \text{iii.} \qquad &F^X(\mathcal{U}\_i) = X(-1) \text{ if } \, \mathcal{Y}\_2 \, ^X(\mathcal{U}\_i) \le 0 \, . \end{aligned}$$

In addition, we define the facet vector **G***i* for an utterance*Ui* using 0–1 variables ( {0,1}) *<sup>y</sup> i* in order to represent the four facets { , , , } *X ABCD* corresponding to each of the categories *yX X X* { ( 1), (0), (1)} :

$$\begin{aligned} \mathbf{G}\_{i} &= \{ \delta\_{i}^{A(-1)}, \delta\_{i}^{A(0)}, \delta\_{i}^{A(1)}, \delta\_{i}^{B(-1)}, \delta\_{i}^{B(0)}, \delta\_{i}^{B(1)},\\ \delta\_{i}^{\mathbf{C}(-1)}, \delta\_{i}^{\mathbf{C}(0)}, \delta\_{i}^{\mathbf{C}(1)}, \delta\_{i}^{\mathbf{D}(-1)}, \delta\_{i}^{\mathbf{D}(0)}, \delta\_{i}^{\mathbf{D}(1)} \} \end{aligned} \tag{2}$$

The set of facet vector **G***<sup>i</sup>* for the all utterances ( 1, , ) *Ui m <sup>i</sup>* is the secondary corpus.

#### **3.2 Case study**

This paper's case study examines the debates of the Yodo-River committee which was established in 2001 in order to obtain advice for the planning and policy handling of a river improvement project related to the building of a dam and gather the opinions of the representatives of affected citizens and public organizations. The Yodo-river committee meetings consists of a general meeting, four regional meetings, five theme meetings, five working group meetings, and three meetings of the sub-working group. A total of 400 meetings have been held since 2001. The debate minutes are available from the committee website and are downloadable as PDF files. The minutes record the names of speakers and their utterances chronologically.

From these meetings, we selected 14 debates between citizens and experts or between administrators (i.e., river managers) and experts. Participants were classified into five groups according to their roles: facilitator; expert; citizen with a "con" opinion; citizen with a "pro" opinion; and administrator. The minutes from the 14 debates consisted of 8,831 utterances. A unit of utterance is a single sentence ended by punctuation mark.

#### **3.3 Outline of the facet decomposition**

194 Semantics in Action – Applications and Scenarios

*h i yUh* .

facet, that is, *X ABCD* {,,,} , this study uses two bounding hyperplanes ( ),( 1,2) *<sup>X</sup>*

relationship and is defined as:

i. ( ) (1) *<sup>X</sup> FU X <sup>i</sup>* if 1 ( )0 *<sup>X</sup>*

ii. ( ) (0) *<sup>X</sup> FU X <sup>i</sup>* if 1 ( )0 *<sup>X</sup>*

iii. ( ) ( 1) *<sup>X</sup> FU X <sup>i</sup>* if 2 ( )0 *<sup>X</sup>*

categories *yX X X* { ( 1), (0), (1)} :

their utterances chronologically.

( {0,1}) *<sup>y</sup> i* 

**3.2 Case study** 

The two types of kernel functions, 1 ( ) *<sup>X</sup>*

These bounding hyperplanes are defined by a high-dimensional discrimination function that determines the amount of terms included in the primary corpus and maximizes the distance between the border and each utterance. Although the discrimination function is usually represented by a nonlinear kernel function, we used a d-dimensional polynomial function that flexibly approximates the function. The SVM model used in this study is able to process large amounts of data and produces good estimation results. In this model, the utterances in the primary corpus, ( 1, , ) *Ui m <sup>i</sup>* , are each classified into one of the three categories of facets, ( ) { ( 1), (0), (1)} *<sup>X</sup> FU X X X <sup>i</sup>* . ( ) *<sup>X</sup> F Ui* determines the corresponding

( { , , , }) *X ABC D*

units of utterances ( 1, , ) *Ui m <sup>i</sup>* which are located in the *n* dimensional space ( *n* is the

*<sup>j</sup> y U* ; and

In addition, we define the facet vector **G***i* for an utterance*Ui* using 0–1 variables

in order to represent the four facets { , , , } *X ABCD* corresponding to each of the

( 1) (0) (1) ( 1) (0) (1) ( 1) (0) (1) ( 1) (0) (1) { , , , ,,,

*A AAB BB i i i i i ii C CCD DD i iii ii*

This paper's case study examines the debates of the Yodo-River committee which was established in 2001 in order to obtain advice for the planning and policy handling of a river improvement project related to the building of a dam and gather the opinions of the representatives of affected citizens and public organizations. The Yodo-river committee meetings consists of a general meeting, four regional meetings, five theme meetings, five working group meetings, and three meetings of the sub-working group. A total of 400 meetings have been held since 2001. The debate minutes are available from the committee website and are downloadable as PDF files. The minutes record the names of speakers and

From these meetings, we selected 14 debates between citizens and experts or between administrators (i.e., river managers) and experts. Participants were classified into five

 

 

The set of facet vector **G***<sup>i</sup>* for the all utterances ( 1, , ) *Ui m <sup>i</sup>* is the secondary corpus.

,,, ,,}

 

**<sup>G</sup>** (2)

 

*<sup>i</sup> y U* , <sup>2</sup> ( ) *<sup>X</sup>*

total number of terms in the primary corpus) are defined as follows:

*<sup>i</sup> y U* and <sup>2</sup> ( )0 *<sup>X</sup>*

*<sup>i</sup> y U* ;

*<sup>i</sup> y U* .

 

: { ( 1), (0), (1)} *<sup>X</sup> FU X X X <sup>i</sup>* (1)

*<sup>i</sup> <sup>y</sup> <sup>U</sup>* for the facets *X ABCD* {,,,} , with *<sup>m</sup>*

Four researchers from construction consulting companies with experience in mobilizing public involvement made a training set of facet decomposition. To maintain the objectivity of the training set, two researchers conducted facet decomposition on the same utterances. We completed the training set the corresponding facet decomposition. Table 1 shows the framework of the four facets with the facet elements and the typical language use.

An utterance is coded as facet A(1) if it describes the pros of the project. Statements about the advantages of the project are examples of A(1). Meanwhile, disapproving opinions are classified as facet A(-1) and include statements that mention the disadvantages of the project and suggest new alternatives. Utterances that include evidences are coded as facet B(1) and those without evidences as B(-1). Utterances with rigorous or adequate evidences are coded as C(1) or C(-1), respectively. Facet C(1) includes statements that cite evidences from studies or experiments while facet C(-1) includes statements with evidences based on individual experiences or feelings. Finally, facet D(1) represents positive attitudes toward the project, signaling affirmation or belief in the project. Facet D(-1) represents negative attitudes toward the project. Conflicts can be identified by comparing the difference between the responses in facet A and those in facet D. Facet B determines the plausibility of arguments while facet C sheds light on the participants' reasoning systems. If an utterance cannot be easily coded using the above facets, it is classified under the neutral category, *X*(0) , {,,,} *X ABC D* .

Below are two examples of how utterances are encoded using the facets framework:

Example 1: The dam construction is the only way to ensure the safety of the town's residents given the limited public finances. → A (1) B (1) C (-1) D (-1)

Example 2: There have been many regional development projects that constructed dams but only a few of them succeeded. → A (-1) B (1) C (1) D (-1)

The first utterance example is decomposed into facet A(1)B(1)C(-1)D(-1). The expression "is the only way" indicates that the speaker agrees to the dam construction, hence, A (1), but does not have a positive attitude toward it, hence, D(-1). The expression "given the limited public finances" is an evidence and is therefore classified as B(1); however, this evidence is also based on the speaker's anxiety and concern, and as such, is classified as C(-1). Meanwhile, the second utterance example expresses pessimism regarding the dam construction, hence, A(-1). The phrase "only a few of them have succeeded" expresses the speaker's reason for his negative view and thus is denoted as B(1), C(1), and D(-1).

As mentioned earlier, researchers created and completed the training utterance set of facet decomposition. Of the total 8,831 utterances, 34% were decomposed as facet A, 65% as facet B, 59% as facet C, and 52% as facet D. Over half of the utterances are selected as training utterence excepting facet A. The training utterance set determines the statistical facet

Facet Decomposition and Discouse Analysis: Visualization of Conflict Structure 197

 

<sup>2</sup>*P R <sup>F</sup> P R* 

Table 2 presents the results of the facet decomposition. The total number of facet decomposition performed by the researchers exceeded that of the utterances as each utterance can be classified into multiple facet elements. 35% of the utterances expressed the pros of the dam project; 33% mentioned the cons percent of the project; 31% of the utterances had evidences while 24% did not; 21% had rigorous evidence while 41% had adequate evidence; 12% expressed a positive attitude toward the project and 55% expressed a negative attitude. Over 50% of the utterances were negative responses to the project that had rigorous evidence. This result does not mean, however, that over half of the debate participants had a negative opinion of the project because the utterances were only the opinions of those who spoke. Meanwhile, in the facet decomposition resulting from the SVM, 4% expressed the pros of the project and 1% expressed the cons; 26% had evidences and 20% did not; 10% presented rigorous evidence and 24% presented adequate evidence; 5% expresses positive attitudes toward the project and 34% negative attitudes. The accuracy of the SVM decomposition was calculated using Equation 3b which resulted in an Fmeasure between 0.48 and 0.66. In general, the accuracy level should be between 0.6 and 0.7. There are at least two possible reasons for this low level of accuracy. First, many of the utterances were decomposed into the third category, neutral (X(0)). Second, the accuracy of the decomposition for facet A was particularly low because most participants imply rather than express the pros and cons of the project and as such, their utterances cannot be easily decomposed into the appropriate facets. The final facet classification lies on the researcher. To ensure the objectivity of the researchers' decomposition, a feedback system is used to review and revise the facet decomposition based on the discourse patterns. Subsection 4.2

The less than ideal accuracy level of the SVM decomposition is not uncommon, as only few classification techniques utilizing natural language processing for discourse data have achieved sufficient accuracy. As such, the results of the SVM decomposition in this study are still useful for determining facet decomposition. The final facet decomposition was improved by adjusting the two classifications by the SVM which supported and facilitated

The SVM decomposition provides useful information that helps researchers in their final analysis and significantly reduces their efforts for confirming the secondary corpus. In

refers to the frequency that both the researchers and the SVM

. The accuracy of the SVM model was determined by

(3a)

(3b)

as 1 *<sup>y</sup> i* 

. Finally,

where *P* ,

**3.4 Results** 

 

and *R*

presents the analysis of the discourse patterns.

the researchers' tasks.

*i* 

 . *A*

decomposed the utterance as 0 *<sup>y</sup>*

the F-value calculated as follows:


Table 1. Framework of Facet Decomposition.

decomposition using the SVM. We then investigate the accuracy of the SVM decomposition by calculating the matching rate between the results done by the researchers and that by the SVM.

The accuracy of the SVM decomposition was estimated by using kernel-fold cross validation wherein we divide the 8,831 utterances into K subgroups and use K-1 subgroups as the training set (i.e., the utterances facet-encoded by researcher) while the remaining subgroup is decomposed into facets using the SVM. In this study, 10 *K* . By changing the subgroup of the training set to the other subgroup, the statistical facet decomposition for all the utterances using the SVM can be implemented a total of K times. The dummy variable *<sup>y</sup> i* indicates the categorized elements *yX X X* { ( 1), (0), (1)} of each facet { , , , } *X ABC D* of utterances ( 1, , ) *Ui m <sup>i</sup>* . refers to the matching frequency of the results of the facet decomposition from both the researchers and the SVM where 1 *<sup>y</sup> i* . refers to the frequency that training set was 1 *<sup>y</sup> i* but was categorized by the SVM as 0 *<sup>y</sup> i* . refers to the frequency that training set was 0 *<sup>y</sup> i* but was identified by the SVM as 1 *<sup>y</sup> i* . Finally, refers to the frequency that both the researchers and the SVM decomposed the utterance as 0 *<sup>y</sup> i* . The accuracy of the SVM model was determined by the F-value calculated as follows:

$$A = \frac{\alpha + \lambda}{\alpha + \beta + \gamma + \lambda} \tag{3a}$$

$$F = \frac{\mathbf{2} \, P \times R}{P + R} \tag{3b}$$

where *P* , and *R* .

#### **3.4 Results**

196 Semantics in Action – Applications and Scenarios

**Elements**  *X*(1) *X*( 1) *X*(0)

> including greetings, questions, and other utterances during the progress of the debate. These utterances are categorized as X(0) and not X(1) or X(-1).

Pros Cons Neutral utterances

Without Evidence

Adequacy of evidence focusing on psychological facts (e.g., anxiety, trust, impression), sensitizing concepts (values, norms, belief), community spirit, ethical facts, fairness, rightfulness, etc.

Negative attitudes such as avoidance, constraint, doubt, distrust, disregard, ignorance, disruption, etc.

decomposition using the SVM. We then investigate the accuracy of the SVM decomposition by calculating the matching rate between the results done by the researchers and that by the

The accuracy of the SVM decomposition was estimated by using kernel-fold cross validation wherein we divide the 8,831 utterances into K subgroups and use K-1 subgroups as the training set (i.e., the utterances facet-encoded by researcher) while the remaining subgroup is decomposed into facets using the SVM. In this study, 10 *K* . By changing the subgroup of the training set to the other subgroup, the statistical facet decomposition for all the utterances using the SVM can be implemented a total of K times. The dummy

results of the facet decomposition from both the researchers and the SVM where 1 *<sup>y</sup>*

indicates the categorized elements *yX X X* { ( 1), (0), (1)} of each

*i* 

*i* 

refers to the matching frequency of the

but was identified by the SVM

but was categorized by the SVM as

*i* . **Definition** 

Refers to whether participants provided evidences to strengthen their position and help

With evidence, including scientific, mathematical, and psychological evidence

Rigor of evidence focusing on statistical facets, scientific facts, existing facts, and engagement

Positive attitudes such as agreement,

esteem, encouragement, compromise, etc.

Table 1. Framework of Facet Decomposition.

facet { , , , } *X ABC D* of utterances ( 1, , ) *Ui m <sup>i</sup>* .

refers to the frequency that training set was 1 *<sup>y</sup>*

refers to the frequency that training set was 0 *<sup>y</sup>*

understand their perspectives

Refers to the characteristics of the evidences (i.e., whether they are rigorous or adequate)

Refers to the participants' bargaining attitudes to the project

others

Refers to the position of the participants (i.e., whether they have taken a "pro" or "con" position on the project.

*Facet A* 

*Facet B* 

*Facet C* 

*Facet D* 

SVM.

variable *<sup>y</sup> i* 

0 *<sup>y</sup> i* .

Table 2 presents the results of the facet decomposition. The total number of facet decomposition performed by the researchers exceeded that of the utterances as each utterance can be classified into multiple facet elements. 35% of the utterances expressed the pros of the dam project; 33% mentioned the cons percent of the project; 31% of the utterances had evidences while 24% did not; 21% had rigorous evidence while 41% had adequate evidence; 12% expressed a positive attitude toward the project and 55% expressed a negative attitude. Over 50% of the utterances were negative responses to the project that had rigorous evidence. This result does not mean, however, that over half of the debate participants had a negative opinion of the project because the utterances were only the opinions of those who spoke. Meanwhile, in the facet decomposition resulting from the SVM, 4% expressed the pros of the project and 1% expressed the cons; 26% had evidences and 20% did not; 10% presented rigorous evidence and 24% presented adequate evidence; 5% expresses positive attitudes toward the project and 34% negative attitudes. The accuracy of the SVM decomposition was calculated using Equation 3b which resulted in an Fmeasure between 0.48 and 0.66. In general, the accuracy level should be between 0.6 and 0.7. There are at least two possible reasons for this low level of accuracy. First, many of the utterances were decomposed into the third category, neutral (X(0)). Second, the accuracy of the decomposition for facet A was particularly low because most participants imply rather than express the pros and cons of the project and as such, their utterances cannot be easily decomposed into the appropriate facets. The final facet classification lies on the researcher. To ensure the objectivity of the researchers' decomposition, a feedback system is used to review and revise the facet decomposition based on the discourse patterns. Subsection 4.2 presents the analysis of the discourse patterns.

The less than ideal accuracy level of the SVM decomposition is not uncommon, as only few classification techniques utilizing natural language processing for discourse data have achieved sufficient accuracy. As such, the results of the SVM decomposition in this study are still useful for determining facet decomposition. The final facet decomposition was improved by adjusting the two classifications by the SVM which supported and facilitated the researchers' tasks.

The SVM decomposition provides useful information that helps researchers in their final analysis and significantly reduces their efforts for confirming the secondary corpus. In

Facet Decomposition and Discouse Analysis: Visualization of Conflict Structure 199

*p p pz A D*

( , ) cos( ) | || |

**F F**

*W W*

 

The similarity between the facet vectors of two participants is calculated based on the cosine

(1)

*D*

*y A D D*

*y A y A*

*i j pi pj*

( 1) (1) (1)

( 1) ( 1)

1 1 1 ( 1) (1)

*p p p A D*

*W W*

*W W*

( 1) (1) *z z*

*pi pj*

**F F**

**F F**

*pi pj*

<sup>1</sup> ( , ) cos { ( , )} *<sup>i</sup> <sup>j</sup> i i <sup>j</sup> dsim p p sim p p* (7)

2

2

*z*

(5)

(6)

(8)

(10)

(9)

2 2

() ()

1 1 1

 

*dis p p dis p p*

*dis p p dis p p*

MDS reproduces the distance ( , ) *<sup>i</sup> <sup>j</sup> dis p p* on a two-dimensional space by minimizing the sum of the squares of the distance between ( , ) *<sup>i</sup> <sup>j</sup> dis p p* and the dissimilarities ( , ) *<sup>i</sup> <sup>j</sup> dsim p p* . Using stress, which represents the degree of incompatibility between the dissimilarity and the

(,) (,)

(,) (,)

*z z z*

{ ( , ) ( , )}

*dis p p dsim p p*

*ij ij*

2

{ (, ) }

*dis p p dis*

*i j*

2

*C*

*z*

{ ( , )}

*dis p p*

*i j*

1

1

*z z*

1

*i ji z z*

 

1

1

*i ji*

1

*z z*

1

 

*i ji*

*W W*

We can determine the dissimilarity of the facet vectors through the inverse of the cosine

*j i*

*p p y y*

*j i*

*p p y y*

Here, *<sup>k</sup> p* indicates a set of facet vector. Facet matrix S is defined as:

**S**

*sim p p*

All distances ( , ) *<sup>i</sup> <sup>j</sup> dis p p* are arranged in a correlation matrix *D*:

**D**

distance, an accurate coordinate value is produced:

*Stress*

*dis* denotes the average of the distance between two participants:

*dis*

angle distance:

function:

**F**

**F**

addition, the SVM helps monitor and improve the performance of rule-based approaches for opinion mining (e.g., Dagan, 2006; Radev, 2000). The SVM method is facilitates the comparison of the results of two approaches using disagreements to track changes in the discourse rules. Future studies should test the applicability of the SVM method to rulesbased approaches.


Note: The symbols ○, ●, and × indicate categories X(1), X(-1), and neutral, respectively.

Table 2. Facet Decomposition Results.

## **4. Discourse analysis of the public debate**

### **4.1 Similarity of utterance meaning**

Using the secondary corpus (i.e., the facet-encoded utterance developed in the previous section), we conducted the discourse analysis in order to examine the interest conflict among the participants and visualize the discourse patterns during the debate process. Using non-metric multi-dimensional scaling (MDS), we distributed the participants on a two-dimensional space based on the similarity of their utterance meanings (Kruskal, 1964a and 1964b; Qian, 2004). Among the Z participants <sup>1</sup> ( ,..., ) *i z p p p* , the dissimilarity of interest (,) *<sup>i</sup> <sup>j</sup> dsim p p* between two participants *<sup>i</sup> p* and *<sup>j</sup> p* is represented as the distance ( , ) *<sup>i</sup> <sup>j</sup> dis p p* in a two-dimensional space. If the interest dissimilarity between the participants is (,) *<sup>i</sup> <sup>j</sup> dsim p p* < (, ) *i k dsim p p* , then the distance is ( , ) *<sup>i</sup> <sup>j</sup> dis p p* < (, ) *i k dis p p* .

For a participant *<sup>k</sup> <sup>p</sup>* ' the relative frequency of dummy variables ( ( 1), (0), (1), , ( 1), (0), (1)) *<sup>y</sup> j yA A A D D D* is defined, making up the facet vector from the secondary corpus as illustrated below:

$$\mathcal{W}\_{\mathcal{Y}}^{pk} = \frac{\sum\_{j \in \Omega pk} \mathcal{S}\_j^{\mathcal{Y}}}{\sum\_{j \in \Omega pk} \sum\_{y = A(-1)}^{D(1)} \mathcal{S}\_j^{\mathcal{Y}}} \tag{4}$$

addition, the SVM helps monitor and improve the performance of rule-based approaches for opinion mining (e.g., Dagan, 2006; Radev, 2000). The SVM method is facilitates the comparison of the results of two approaches using disagreements to track changes in the discourse rules. Future studies should test the applicability of the SVM method to rules-

Cons/Pros With/Without

**Facet A Facet B Facet C Facet D** 
