**2.2. EOD pilot description**

The Earth Observation Data Processing and Distribution Pilot (EOD) consists of the implementation of the Elecnor Deimos' geo-data processing, storage and distribution platform of Deimos-2 satellite using cloud technologies. The main functionalities of the system are the following:


Optimization of an Earth Observation Data Processing and Distribution System http://dx.doi.org/10.5772/intechopen.71423 179

**Figure 1.** Earth Observation Data Processing and Distribution pilot (EOD)'s architecture.

#### *2.2.1. EOD architecture*

processing and distribution to end users with variable demands. Kecskemeti et al. [12] introduced the ENTICE approach to solve these problems. The ENTICE environment consists of a ubiquitous repository-based technology, which provides optimised virtual machine (VM) image creation, assembly, migration and storage for federated clouds. The webpage of ENTICE can be

ENTICE facilitates the implementation of cloud applications by simplifying the creation of lightweight virtual machine images (VMIs) by means of functional descriptors. These functional descriptors define at high and functional levels the VMIs and contribute to define the system Service Level Agreement (SLA) to facilitate the optimization of the VMIs in terms of performance, costs, size and quality of service (QoS) needed. Then, the VMIs are automatically decomposed and distributed to meet the application runtime requirements. In addition, ENTICE facilitates elastic autoscaling. The benefits of using ENTICE are the following:

In the EOD pilot, ENTICE is used as middleware between the federated infrastructure

The Earth Observation Data Processing and Distribution Pilot (EOD) consists of the implementation of the Elecnor Deimos' geo-data processing, storage and distribution platform of Deimos-2 satellite using cloud technologies. The main functionalities of the system are the following:

• **Acquisition of raw data:** When the imagery data are ingested from the satellite into the ground station, the system is notified and the ingestion component automatically ingests

• **Processing of data:** Once the data are ingested, it is processed in the product processors.

• **Archiving and cataloguing geo-images:** The different products obtained from the processing of raw data are archived and catalogued in order to provide these images or high added

• **Offering user services:** This is the front-end of the system. It allows end users to select the

found in [13].

• Reduction of up to 80% storage.

178 Multi-purposeful Application of Geospatial Data

• 95% elastic Quality of Service.

• VMIs optimization up to 60%.

• Reduction on the costs of deployment.

• Elimination of cloud infrastructure vendor lock-in.

the raw data into the cloud for its processing.

product that they want to visualize or to download.

There are several processing levels to provide different products.

described in Section 3.1 and the gs4EO application software.

• VMIs creation 25% faster.

• VMIs delivery 30% faster.

• Scalability and elasticity.

**2.2. EOD pilot description**

value services to end users.

The main objectives of the EOD pilot is to process real data of Deimos-2 satellite in a realistic scenario of normal operation and the validation of the processing chain module as part of the cloud infrastructure. Ramos and Becedas [14] proposed an original architecture of the gs4EO suit to be implemented in cloud. Based on that work, the architecture for the EOD pilot has been redesigned and implemented, see **Figure 1**.

The architecture is composed of the following components:

	- To identify which outputs shall be generated by the processors.
	- To generate the Job Orders. They contain all the necessary information that the processors need. Furthermore, these eXtensive Markup Language (XML) files include the interfaces

and addresses of the folders in which the input information to the processors is located and the folders in which the outputs of the processors have to be sent. They also include the format in which the processors generate their output.

• **archive4EO:** In this module, the processed images are stored and catalogued for their dis-

Optimization of an Earth Observation Data Processing and Distribution System

http://dx.doi.org/10.5772/intechopen.71423

181

• **Shared storage:** It is a storage module shared by all the modules of the architecture in which all the inputs and outputs of the different modules of the architecture are stored.

The testing infrastructure used in the experiment is formed by hardware deployed in three different locations and managed in a federated manner: DMU infrastructure (in Deimos UK in United Kingdom), DMS infrastructure (in Deimos Space in Spain) and DME infrastructure (in Deimos Engenharia in Portugal). The hardware resources deployed in every location are described in **Table 1**. The ENTICE middleware was installed in the DMU infrastructure, which is acting as master. It also contains an object store with interface to Amazon Simple Storage Service (Amazon S3) for cloud bursting. DMS and DME infrastructures are slaves of DMU infrastructure and contain object stores also with interfaces to Amazon S3. A block diagram describing the interrelations of the testing infrastructure is depicted in **Figure 3**. The virtualization of the infrastructure was done with OpenNebula. Kernel-based Virtual Machine (KVM) was used as hypervisor. The creation of the virtual machines was done with Packer, whereas the automatic deployment of the virtual machines was done with Ansible. **Figure 4** shows a diagram describing the logic process of automatic generation of the virtual machines that constitute the EOD software. The image building process takes advantage of

**Location Name Model CPU RAM (GB) HD (GB) OS**

3.4 GHz

3.4 GHz

1.86 GHz

2.37 GHz

Node1 Dell Intel 2 Core 3 GHz 6 230 CentOS

Dual Core 3800+

8 160 CentOS

16 250 CentOS

4 250 CentOS

16 2048 CentOS

4 256 CentOS

7.2.1511

7.2.1511

7.2.1511

7.2.1511

7.2.1511

7.2.1511

DMU Node-1 Dell Optiplex790 Intel Core i7–2600

DMS Node-2 Dell Intel 8 Core

**Table 1.** Hardware resources in the testing infrastructure.

DME Node1 HP AMD Athlon 64 X2

Node-2 Dell Optiplex790 Intel Core i7–2600

OpenNebula-fe Dell Optiplex745 Intel Core 2 6300

tribution. It offers a Catalogue Service for the Web (CSW) interface.

**3. Experiment setup**

**3.1. Testing infrastructure**

• **user4EO:** It is a web service in which the end users can access to the products.

	- Calibration: (L0 and L0R processing levels) to convert the pixel elements from instrument digital counts into radiance units.
	- Geometric correction: (L1A processing level) to eliminate distortions due to misalignments of the sensors in the focal plane geometry.
	- Geolocation: (L1BR processing level) to compute the geodetic coordinates of the input pixels.
	- Orthorectification: (L1C processing level) to produce orthophotos with vertical projection, free of distortions.

