**4. Overview of the general WiDOP model**

The problem of automatic object reconstruction remains a difficult task to realize in spite of many years of research. Major problems result from geometry and appearance of objects and their complexity, and impact on the collected data. For example, variations in a viewpoint may destroy the adjacency relations inside the data, especially when the object surface shows considerable geometrical variations. This dissimilarity affects geometrical or topological relations inside the data and even gets worse, when partial occlusions result in a disappearance of object parts. Efficient strategies therefore have to be very flexible and in principle need to model almost all factors having impact of the representation of an object in a data set. That leads to the finding, that at first a semantic model of a scene and the objects existing therein is required. Such a semantic description should be as close to the reality as possible and as necessary to take most relevant factors into account, which may have impact on later analysis steps. At least this comprises the objects to be extracted with their most characteristic features (geometry, shape, texture, orientation,...) and topological relations among each other. The decision upon features to be modelled should be affected by other important factors in an analysis step like characteristics of the data, the algorithms and their important features. Such a model might be supported by a DL-OWL ontology structure formed out of RDFS nodes and properties where the nodes represent classes or objects as their instances and the links show relationships of various characteristics. Such a network then contains the knowledge of that type of scene, which has to be processed. This knowledge base will act as basis for further detection and annotation activities and has to work in cooperation with numerical algorithms.

Up to this point, the new conception is still in concordance to other knowledge related set ups, although the degree of modelling goes farther because all relevant scene knowledge

From Unstructured 3D Point Clouds to Structured Knowledge - A Semantics Approach 227

processing. Knowledge will be structured and formalized based on IFC schema, XML files, the domain concept which is the Deutsche Bahn scene in this case and 3D processing domain experts, etc., using classes, instances, relations and rules. An object in the ontology can be modelled as presented; a room has elements composed of walls, a ceiling and a floor. The sited elements are basic objects. They are defined by their geometry (plane, boundary, etc.), features (roughness, appearance, etc.), and also the qualified relations between them (adjacent, perpendicular, etc.). The object "room" gets its geometry from its elements, and further characteristics may be added such as functions in order to estimate the existent sub elements. For instance, a "classroom" will contain "tables", "chairs", "a blackboard", etc. The detection of the object "room" will be based on an algorithmic strategy which will look for the different objects contained in the point cloud. This means, using different detection algorithms for each element, based on the above mentioned characteristics, will allow us to classify most of the point region in the different element categories. It corresponds to the spatial structure of any facility, and it is an instance of semantic knowledge defined in the ontology. This instance defines the rough geometry and the semantics of the building elements without any real measurement. This model contains also knowledge extracted from the technical literature of the domain and knowledge from experts of the domain too. In addition, the ontology is as well enriched with knowledge about 3D processing algorithms and populated with the results of experiences undertaken on 3D point clouds, which define the empirical knowledge extracted from point clouds regarding a specific

Numerical processing includes a number of algorithms or their combination to process the spatial data. Strategies include geometric element detection (straight line, plane, surface, etc.), projection - based region estimation, histogram matrices, etc. All of these strategies are either under the guidance of knowledge, or use the modelled prior knowledge to estimate the object intelligently and optimally. Alongside with 3D point clouds, various types of input, data sets can be used such as images, range images, point clouds with intensity or color values, point clouds with individual images oriented to them or even stereo images without a point cloud. All sources are exploited for application to particular strategies. Knowledge not only describes the information of the objects, but also gives a framework for the control of the selected strategies. The success rate of detection algorithms using RANSAC (Tarsha-Kurdi, et al., 2007), Iterative Closest Point (Milella, et al., 2006) and Least Squares Fitting (Cantrell, 2008) should significantly increase by making use of the knowledge background. However, we are planning not only to process point data sets but also based on a surface and volume representation like mesh, voxels and bounding Boxes. These methods and others will be selected in a flexible way, depending on the semantic

In order to manage the interaction between the knowledge part and the 3D processing one, a new layer labelled WiDOP processing materialized within rules is created. This layer ensures the control and the management of the knowledge transaction and the decision taken based on SWRL languages and its extensions through several steps explained within

domain of application.

context.

**4.3 The WiDOP processing** 

**4.2 The 3D processing algorithms** 

will be integrated. But another aspect will be considered also allowing to significantly improving processing strength. That is to integrate knowledge even on the algorithmic side. This means to make use of the flexibility of knowledge processing for decisions and control purposes inside the algorithmic processing chain. Even a propagation of findings from processing results into new knowledge for subsequent steps should be possible, what would give a completely new degree of dynamics and stability into the evaluation process.

It will finally leads to the conceptual view shown in Fig 5 where the general architecture for the suggested solutions is presented. It's composed of three parts: the knowledge model, the 3D processing algorithms execution and the interaction management and control part labelled WiDOP processing materialized within swrl rules and Built-Ins extensions, ensuring the interaction between the above sited parts. In contrast to existing approaches, we aim at the utilization of previous knowledge on objects. This knowledge can be contained in databases, construction plans, as-built plans or Geographic Information Systems (GIS). The suggested solution named as knowledge based detection of objects in point clouds (WiDOP) has its roots in the knowledge base which then guides individual algorithmic steps. Results from algorithms are also analyzed by the knowledge base and the reasoning engine, then deciding upon subsequent algorithmic steps is taken also from the knowledge base. Accordingly, detected objects and their features are populated to the knowledge base, which will permanently evolve until the work is done.

Fig. 5. WiDOP: Overview system.

#### **4.1 The knowledge model**

The needed knowledge for such purpose will be modelled within a top level ontology describing the general concept behind the knowledge domain. The suggested approach is intended to use semantics based on OWL technology for knowledge modelling and processing. Knowledge will be structured and formalized based on IFC schema, XML files, the domain concept which is the Deutsche Bahn scene in this case and 3D processing domain experts, etc., using classes, instances, relations and rules. An object in the ontology can be modelled as presented; a room has elements composed of walls, a ceiling and a floor. The sited elements are basic objects. They are defined by their geometry (plane, boundary, etc.), features (roughness, appearance, etc.), and also the qualified relations between them (adjacent, perpendicular, etc.). The object "room" gets its geometry from its elements, and further characteristics may be added such as functions in order to estimate the existent sub elements. For instance, a "classroom" will contain "tables", "chairs", "a blackboard", etc. The detection of the object "room" will be based on an algorithmic strategy which will look for the different objects contained in the point cloud. This means, using different detection algorithms for each element, based on the above mentioned characteristics, will allow us to classify most of the point region in the different element categories. It corresponds to the spatial structure of any facility, and it is an instance of semantic knowledge defined in the ontology. This instance defines the rough geometry and the semantics of the building elements without any real measurement. This model contains also knowledge extracted from the technical literature of the domain and knowledge from experts of the domain too. In addition, the ontology is as well enriched with knowledge about 3D processing algorithms and populated with the results of experiences undertaken on 3D point clouds, which define the empirical knowledge extracted from point clouds regarding a specific domain of application.

#### **4.2 The 3D processing algorithms**

226 Semantics – Advances in Theories and Mathematical Models

will be integrated. But another aspect will be considered also allowing to significantly improving processing strength. That is to integrate knowledge even on the algorithmic side. This means to make use of the flexibility of knowledge processing for decisions and control purposes inside the algorithmic processing chain. Even a propagation of findings from processing results into new knowledge for subsequent steps should be possible, what would

It will finally leads to the conceptual view shown in Fig 5 where the general architecture for the suggested solutions is presented. It's composed of three parts: the knowledge model, the 3D processing algorithms execution and the interaction management and control part labelled WiDOP processing materialized within swrl rules and Built-Ins extensions, ensuring the interaction between the above sited parts. In contrast to existing approaches, we aim at the utilization of previous knowledge on objects. This knowledge can be contained in databases, construction plans, as-built plans or Geographic Information Systems (GIS). The suggested solution named as knowledge based detection of objects in point clouds (WiDOP) has its roots in the knowledge base which then guides individual algorithmic steps. Results from algorithms are also analyzed by the knowledge base and the reasoning engine, then deciding upon subsequent algorithmic steps is taken also from the knowledge base. Accordingly, detected objects and their features are populated to the

The needed knowledge for such purpose will be modelled within a top level ontology describing the general concept behind the knowledge domain. The suggested approach is intended to use semantics based on OWL technology for knowledge modelling and

**WiDOP Ontology Model** 

**knowledge IFC files** 

**Object detection**

**General model processing**

> **Elements detection**

**Ontologies** 

**3D processing algorithms** 

**Expert** 

**Reasoning module**

give a completely new degree of dynamics and stability into the evaluation process.

knowledge base, which will permanently evolve until the work is done.

**Knowledge**

**3D processing**

Fig. 5. WiDOP: Overview system.

**Point cloud**

**SWRL Rules**

**4.1 The knowledge model** 

Numerical processing includes a number of algorithms or their combination to process the spatial data. Strategies include geometric element detection (straight line, plane, surface, etc.), projection - based region estimation, histogram matrices, etc. All of these strategies are either under the guidance of knowledge, or use the modelled prior knowledge to estimate the object intelligently and optimally. Alongside with 3D point clouds, various types of input, data sets can be used such as images, range images, point clouds with intensity or color values, point clouds with individual images oriented to them or even stereo images without a point cloud. All sources are exploited for application to particular strategies. Knowledge not only describes the information of the objects, but also gives a framework for the control of the selected strategies. The success rate of detection algorithms using RANSAC (Tarsha-Kurdi, et al., 2007), Iterative Closest Point (Milella, et al., 2006) and Least Squares Fitting (Cantrell, 2008) should significantly increase by making use of the knowledge background. However, we are planning not only to process point data sets but also based on a surface and volume representation like mesh, voxels and bounding Boxes. These methods and others will be selected in a flexible way, depending on the semantic context.

#### **4.3 The WiDOP processing**

In order to manage the interaction between the knowledge part and the 3D processing one, a new layer labelled WiDOP processing materialized within rules is created. This layer ensures the control and the management of the knowledge transaction and the decision taken based on SWRL languages and its extensions through several steps explained within

From Unstructured 3D Point Clouds to Structured Knowledge - A Semantics Approach 229

As intermediate steps, the different geometries within a specific 3D point clouds are detected and stored within the ontology structure. Once done, the existent topological relations between the detected geometries are qualified and then populated in the knowledge base. Finally, detected geometries are annotated semantically, based on existing knowledge's related to the geometric characteristics and topological relations, where the input ontology contains knowledge about the Deutsche Bahn railway objects and

This section discusses the different aspects related to the domain concept top level ontology structure installed behind the WiDOP Deutsche Bahn prototype (Ben Hmida, et al., 2010). It´s composed mainly by the classes and their relationships. Hence, we try to discuss theses

The domain ontology presents the core of our research and provides a knowledge base to the created application. The global schema of the modelled ontology structure offers a suitable framework to characterize the different Deutsche Bahn elements from the 3D

To guide the processing algorithm sequence creation based on the target object

To ensure the semantic annotation of the different detected objects inside the target

In fact, the ontology is managed through different components of description logics where the class axioms contain their own prefixes used to define their names. One of the big advantages of using prefix is that the same class could be used by applying different prefix for the class. Other advantages include the simplification in defining the resource and to solve the ambiguity for different context. The hierarchical structure of the top level class axioms of the ontology is given in Fig 7, where we find five main classes within other data

The class axiom DC:DomainConcept which represents the different object found in the target scene can be considered the main class in this ontology as it is the class where the target objects are modelled, this class is further specialized into classes representing the different detected object. However, the importance of other classes cannot be ignored. They are used to either describe the object geometry through the Geom:Geometry class axiom by defining its geometric component or the bounding box of the object that indicate its coordinates or to either describe its characteristics through the Charac:Characteristics class axiom. Additionally, the suitable algorithms are automatically selected based on its compatibility within the object geometry and characteristics via the Alg:Algorithm class. Add to that, other classes, equally significant, play their roles in the backend. The connection between the basic mentioned classes is carried out through object and data

processing point of view. The created ontology is used basically for two purposes:

and objects properties able to characterize the scene in question.

knowledge about 3D processing algorithms.

**5. Description of the WiDOP knowledge base** 

component in term of axiom representing them.

characteristics.

scene.

**5.1 Class axioms** 

properties axioms.

the next section. The semantic within the ontologies expressed through OWL-DL language can be used for further inferences. For instance, the following rule asserts that a Bounding Box with lines higher then 5 m are masts where Masts, Bounding Boxes and lines are all individual-valued properties. The DL syntax related to such an expression is Mast ⊑ (�������� ��� � � �������� ���� � � �as�����t� �� ��) while the swrl conversion of such an expression is BoundingBox(?x) hasLine(?x,?y) hasHeight (?y,?h) swrlb:GreaterThan (?h, 5) Mast(?x).

The set of built-ins for SWRL is motivated by a modular approach that will allow further extensions in future releases within taxonomy. SWRL's built-ins approach is also based on the reuse of existing built-ins in XQuery and XPath, which are themselves based on XML Schema by using the Datatypes. This system of built-ins should as well help in the interoperation of SWRL rules with other Web formalisms by providing an extensible, modular built-ins infrastructure for Semantic Web Languages, Web Services, and Web applications. Many built-ins are defined. These built-ins are keys for any external integration where we take advantages of this extensional mechanism to integrate new Builtins for 3D processing and topological processing.
