**3.1 Components of VR/AR**

An augmented reality system, from a hardware perspective, consists of a sensor(s), processor(s), and display(s). AR systems superimpose computer graphics imagery on the real world, and, for this blending, the observer's positions must be known (with extreme precision), stored, and related with the positions and geometry of the items that are to be projected on the real environment. From a general perspective, it is quite easy to draw the three-dimensional images that you want to superimpose on what the user observes in the real world. The task to be solved is to fully define a correct perspective while defining as accurately as possible the position of the observer's eye. To offer a truly useful experience, portable and easy-to-use optical systems must be considered.

The AR processor coordinates and analyzes sensor inputs, stores, and retrieves data, carries out the tasks of the application program, and generates the appropriate signals to display. Computing systems for augmented reality can range in complexity from simple handheld devices such as smartphones and tablets to laptops, desktop computers, and workstation class machines all the way through powerful distributed systems. The scene must be updated smoothly and at a rate that the participant in the experience perceives as a constant stream of information. AR applications must sustain a frame rate of at least 15—preferably more— frames per second for the participant to perceive the display as continuous. Displays that are simulating the feel of a solid object must be updated about 1000 times per second or else the object will feel "mushy." To superimpose information (graphics or data) stored in a computer (control center) on the actual environment, a device called a beam splitter is typically used. This divider does not divide but combines the images of the real environment with those placed in the monitor environment. As a result, the viewer is presented with a double exposure photograph. Because the optics are typically fixed, in AR systems there is only one depth at which both the computer-generated imagery and the real-world imagery are in focus. If realworld and virtual-world scenes are both in focus, it will be easier to perceive them simultaneously.

By the other hand, the components necessary for building and experiencing VR are divided into two main components, the hardware and the software mechanisms. The hardware components are composed of computer workstation, sensory displays, process acceleration cards, tracking system and input devices. The workstation is an object (computer or microcomputer) that is intended for technical or scientific work. These objects are typically used by one person at a time, but are connected to a network to facilitate multi-user operations. Sensory screens are the artifacts that are responsible for displaying virtual environments. These displays can be as basic as common computer display units to head-mounted displays (HMDs) with headphone mounts for a 3D viewing and audio experience.

When over-the-head (helmet) displays are used, the presentation of the images is given right in front of the viewer's eyes. With the helmet's sensors (the orientation ones) the artificial segments are controlled that make the user experience a complete virtual environment. In most cases, a set of optical lens and mirrors are used to enlarge the view to fill the field of view and to direct the scene to the eyes [58]. It is fundamental to have process acceleration cards to update the display with new sensory information and the tracking system (mechanical, electromagnetic, ultrasonic, and infrared trackers) that follows the position and orientation of a user in the virtual environment.

There are also VR software key components: 3D modeling software, 2D graphics software, digital sound editing software and VR simulation software. 3D modeling software is used in constructing the geometry of the objects in a virtual world and

#### **Figure 2.**

*Reality roadmap according to Intel®, which prefers merged reality to mixed reality; the term "mixed reality" has been thrown about a lot as of late but pinning down a precise definition has proven elusive.*

specifies the visual properties of these objects, and 2D graphics software permits to manipulate texture to be applied to the objects which enhance their visual details. With the digital sound editing software is mixed and edited sounds that objects make within the virtual environment and with the VR simulation software the components are put together. It is used to program how these objects behave and set the rules that the virtual world follows.

In this research is presented a mechanism of Mixed Reality (MR), which combines Virtual Reality and Augmented Reality, called also 'Hybrid Reality' or 'Extended Reality'. MR [59] is understood as a proposal that can be placed anywhere between digital environments that belong to complete virtuality up to the absolute perception of the real environment. MR works with concepts and objectives of AR and VR (**Figure 2**). The use of helmets or glasses allows the user to enter a digital world with all the information that could be useful. The continuity between reality and virtuality is the basis for the interaction between objects in the physical world with items in the virtual world, which will finally be the future of these technologies.

### **4. Application for earthquakes**

Modeling objects and the building structure in the virtual world is the first step for environment construction. In this research the employed techniques entail software for modeling/constructing bi- and three- dimensional objects and backgrounds. Once the 3D modeling of the objective scene is built, the model brings in the VR/AR mechanisms (in this research is used Unity3D-Unreal), in this way the static model is converted into a synergistic environment. Based on the experiences modeling urban infrastructure, these alternatives are appropriate for metropolis projects. Then the virtual contents (movements, gestures, and commands) were established. Physics engines were sometimes used to assist the simulation of emergencies, i.e., for a building shaking according with the accelerations registered during an earthquake, in each virtual room the movements of the room must be experienced, but in addition to the objects that could also move and even fall.

*Intelligent VR-AR for Natural Disasters Management DOI: http://dx.doi.org/10.5772/intechopen.99337*

An avatar is used to manage the commands that will make the experience in the virtual environment one with dedicated goals. The avatar is the representation of the person, and it reflects the response behavior (of him and/or the participants) to the stimuli of the virtual world. For an avatar to move and act, a device such as a mouse, keyboard, pad, and telephone (in the 2D case) and gamepad and joystick (for the 3D offer) is used. The AR phase contains, in addition to the possibilities of movements and executing commands, data and information from the real world that could be useful for the designed task. Cameras and sensors are widely used and combined with the image recognition function of an AR engine. HMDs (glasses) provides the ergonomic solution for a virtual environment task using direct vision. Users can experience high-level immersion and better react as if they were in the real world.

The September 19th, 2017, a M 7.1 earthquake hit Mexico City causing 370 people were killed, over 6,000 were injured, 230 totally collapsed buildings and more than 7 000 small houses were damaged. The situation required the intervention of hundreds of geotechnics and structural specialists to qualify conditions and to permit people to return to their homes (**Figure 3**). Exactly 32 years before (Sept 19th, 1985, more than 10,000 deaths, 30,000 destroyed buildings, and 68,000 injured people) a M8.1 event had showed the fact that the city is built on the unstable and sinking ground of a dried-out lakebed and this promoted a better geo-zonification of the risk.

In 1985 the functioning of the city was failed for months, the reconstruction took years and led many to relocate from the most affected areas to the city's outskirts in search of the safety of the bedrock. The districts affected in 1985 and 2017 events were quite different and, because of this, the type of damage and the technical needs to manage rescue activities were also distinctive. These experiences have undoubtedly shown that for designing an efficient post-earthquake relief plan, it must be recognized that Mexico City is particularly exposed because of its huge population and strategic importance for the country and that this vulnerability has grown the last years due to the expansion of the urban settlements in risky areas (nodules of extreme poverty), the environmental devastation, the deterioration

#### **Figure 3.**

*The Mexican earthquake from the 19 September 2017 led to significant building damage in the capital Mexico City and the states of Morelos and Puebla. The damage data in houses/buildings and soils characteristics highlights the correlation between damage drivers that little has been studied or they have not been fully understood.*

of life levels, the economic activities concentration that require the transport of substances (water, potable and used, gas and hydrocarbons) by underground infrastructure, and the growing complexity of transportation process.

Despite the best intentions and the enormous efforts of the governments that have attended these emergencies, the situations dangerously evolved and the period in which the city was detained, and the population subjected to chaos, spread out for months (or even years in 1985) damaging, primarily, the poorer and fragile (socially) population centers. Hereby, there is an increasing need for a holistic and more efficient natural disaster management.

The application that is explained in the following is VR-AR engine to train and to administrate the three phases of this kind of NDM: preparedness, response, and recovery. This VR instrument has the capability bridge the gap between knowledge and action with the necessary velocity and efficiency when disasters happen. Exploiting the scenarios reconstructed, the first-responders, civilians, and city-planners can be prepared to work under a series of conditions built in virtual/ augmented-life, expanding the learning and response experiences. The heterogeneity of soils properties (and responses), earthquakes damage intensities, and the actions effectiveness (professional teams and their capacities) is displayed in the digital metropolis based on the results from neural topologies that predict the spatial variability of risk components (exposure and vulnerability).

#### **4.1 Pre-emergency preparedness**

For the preparation of engineers and specialists to perform i. state declaration (for soils and structures), ii. routes recognition (safest and more efficient), and iii. Provision of resources, it was necessary to construct specific VR-AR tools. The training under real-life conditions goes in two directions: geo-situations and structures. Simulations are about first-responders' reactions for specific scenarios: 1. extreme vulnerable communities, 2. structural pathologies (residential homes and multi-family buildings) and 3. buried infrastructure and 4. soil masses.

The immersive experience of an earthquake (during the shaking and/or immediately after it has happened) is simulated in locations where fragile infrastructure coupled with danger zones (susceptible soils). In **Figure 4** is shown how is recreated the structural masonry commonly used in poverty nodes in Mexico City. The structuration refers to the practice of using masonry, brick, or stone, as mass self-supporting. It is one of the oldest building methodologies, and by far the most resilient, however when self-construction (the inhabitants develop the structuring without the support of engineers or architects) does not follow basic rules for the placement of vertical and horizontal reinforcement elements (columns and beams) and the mezzanine and ceiling slabs are extremely light (even without steel reinforcement), the response of these units to ground movements is very unfavorable.

The VR-AR tool (principally the VR box) drives the user to detect first the kind of cracks, fissures or other pathologies presents in a house and to relate them with a classifying guide (**Figure 5**). After this, the observer must look for overgrown trees and shrubs, cracked drains, leaking rainwater goods, as some of the things that may lead to structural problems but that are not related to the seismic inputs.

Part of this investigation, but still in a preliminary stage, is the issue of the entry and exit routes of the emergency teams. The evacuation of threatened communities and assistance to those potentially blocked by the effects of an earthquake is a complex and vital issue. Transportation system is conceived as the role that sustain the economic and social well-being of the communities so disaster or extreme hazard such as earthquake has a major impact on the resilience of the communities. In Mexico City suburbs, road infrastructure is linked to many factors such as users,

*Intelligent VR-AR for Natural Disasters Management DOI: http://dx.doi.org/10.5772/intechopen.99337*

#### **Figure 4.**

*A training room: after an earthquake M8, subduction zone, site Lake zone, failure conditions for a concrete frame system (1 floor). Analyzing cracks in walls and columns the user is being trained to correct classify the damage level after an extreme earthquake.*

climate, economic level, material, topography, and periodic maintenance, therefore, part of the real-time analysis for the planning of entry and exit routes must be the superposition of layers on information from massive sources (traffic control systems) and the spatial variation of these susceptibilities. A comprehensive review of social infrastructure (hospitals, schools, recreation centers, markets, department stores, fire and police stations, etc.) must be done to detect temporal routes options as part of the adaptive routing solution.

#### **4.2 Response during the emergency**

The Response during Emergency refers to the actions that people may take, in the case of earthquakes, immediately after the ground movement has ended. The emergency period, when the events are extreme, can be extended depending on the size of the heavily damaged areas or in which the collapses have occurred in public places with large concentrations of people. The AR box was developed to assist technical crews which are organized according to the risk levels at each site. The entrance to housing units is categorized according to their structure and size: types of walls, types of columns, types of slabs, types of foundations and, on the other hand, singlefamily houses, multi-family houses, light and small buildings, large buildings.

In this stage of NDM, the personal is in real danger so it is mandatory that they have a support outside the damaged site: an automated or human control center. The superposition of information about the structures and materials permits rescue teams to take immediately decisions about evacuations or calls for additional help

#### **Figure 5.**

*A training room: after an earthquake M8, a sinkhole has formed, and it is necessary to determine the sources that erode/dissolve as well as call the specific help team. The repair conditions are stored in the event DataMart.*

(**Figure 6**). Also, it is important that the field teams are provided with a dedicated storage for the found conditions. Once the *in-situ* situations are loaded, they are integrated to a set for specific analysis (feed forward neural network) that defines the risk level and spatially categorize the geographical situation. In this way, state maps are built in real time. Additional maps can be displayed using the stored information, for example by factor, by action demanded (posterior attention, evacuation, or human rescue) and by supply (call for requirement of special equipment/ machines or other inspection/rescue teams) (**Figure 7**).

In addition, there is a section of AR toolbox that is exclusively dedicated to the attention of leaks. Water and gas, the latter considered a priority due to the secondary effect of the explosions, are attended by specialized professionals that work in accordance with specific regulated policies (**Figure 8**).

In some zones of the metropolis the vulnerability of the soils to the arrival of seismic waves is very high. The most superficial layers in these areas suffer cracking and subsidence processes among the most alarming scales in the world (**Figure 9**). The periods of drought and torrential rains aggravate the susceptibility to the collapse/cracking. This situation maintains small buildings and buried facilities in a very susceptible state, making them prone to be more affected when an extreme earthquake hit. The coincidence between construction deficiencies (poor technical conceptions) and degradation of materials (highly deformed, cracked and collapsed soils), when the seismic waves arrive, constitutes a challenging scenario.

*Intelligent VR-AR for Natural Disasters Management DOI: http://dx.doi.org/10.5772/intechopen.99337*

#### **Figure 6.**

*Immediately after the earthquake has ended, the trained teams go out into the field and begin to verify the conditions through the lenses that are communicated with the control room, a) since street, an engineer is checking a multi-level building that has lost the walls on one side of its perimeter, b) inside a 1-story house, a user checks the cracking, catalogs it and stores it.*

#### **Figure 7.**

*Example of the integral maps: peak ground accelerations PGA superposed on the number of damaged houses (1 to 3 floors) -red circles- in a small region of the Mayor's Office of Tláhuac, southeast of Mexico City.*

#### **Figure 8.**

*The status review of the pipelines that transport dangerous substances (in this case gas) is essential to qualify the inhabited regions as safe. The application created for one of the largest companies in the country is shown. The user has the information about the arrangements in the field as well as the readings that must be verified on the special monitors. At all times, the user can be assisted by his human colleagues in the control room.*

#### **Figure 9.**

*In the southeastern of Mexico City, where the poorest neighborhoods of the metropolis are concentrated, during the 2017 earthquake manifestations such as cracks, subsidence and collapses (steps) in the most superficial soils were exacerbated. This had the effect of the uninhabitability of thousands of homes. Since then, the city government undertook one of the largest reconstruction campaigns in the modern history of the city.*

#### *Intelligent VR-AR for Natural Disasters Management DOI: http://dx.doi.org/10.5772/intechopen.99337*

The toolbox for geotechnical engineers, intend to train them to detect aspects about differential subsidence, sudden deformations, and sinkholes. In the immediate aftermath of a major earthquake, they must also learn to catalog an additional symptom of deterioration: cracks and fissures in natural masses. This is particularly important as damage to build units can develop days after the seismic event if the openings in the soils and rocks are not properly treated (**Figure 10**). The registered details are stored and sent to the control center where are analyzed with a CART (classification tree) to determine if the site is on the "zero set" (cases where their conditions are on the top of risk levels and demands immediate actions).

The machine learning analysis is based in layers of Geo-descriptors that permits to qualify the susceptibility to sink-fracture in specific Mexico City regions. With the analysis of 6 variables (Geo-position, Soil heterogeneity, Groundwater Level, Urban loads, Type of foundations, Use of nearby streets) a CART (**Figure 11**) determines the relative influence of each one on the cracks' development and relates present conditions to a risk level. The user of the tool can request from the control center the result of the evaluation with CART so that he, when faced with any doubt about the state of the soils, can make a decision and qualify the situation.

#### **Figure 10.**

*The vision of the user in the field is shown, a) in the Tláhuac area, in a serious crack, he observes the information on geotechnical zoning, b) check, in one of the most damaged areas, the repair of the drainage ducts, observes the optimal filling conditions according to the regional sinking map c) at site, where a leak and step deformation are present, infers which is the damaged section and record it in the application to call a specialized team.*

#### **Figure 11.**

*CART for conclude about the susceptibility to crack. The model uses (a) information from maps of geo and anthropic properties and (b) survey of field damage. This tool (c) permits the user to qualify the susceptibility of a site to the development of a crack. (d).*

For example, let us examine a site in the southern poor, and very susceptible to cracking, region in the metropolis. In **Figure 12** the user's vision when entering the site is shown. The characteristics of the breaks that must be recorded are geometry, materials (under consideration of soil evaluation according to the SUCS unified soil classification system), relative movement between the flanks of the crack/ step/ subsidence, among others. When the user asks for the CART response for this site, the geotechnical information (boreholes) that the AR-tool finds near the site is first shown and then the evaluation is presented with a disaggregated description of the factors that lead to that level of susceptibility. In the case exposed, the presence of a non-continuous thin layer of semi-rigid material embedded in the plastic clay, its proximity to the battery of water pumping wells (operating at relatively shallow depths) and the shape and depth of the rigid base, are the conditions that drives to the site to a high susceptibility to crack.

#### **4.3 Recovery post-emergency**

When the conditions of habitability and urban services have begun the path towards normalization, government administrators and practitioners (particularly engineers dedicated to the generation of infrastructure) should consolidate the

*Intelligent VR-AR for Natural Disasters Management DOI: http://dx.doi.org/10.5772/intechopen.99337*

#### **Figure 12.**

*Conditions at the entrance of a trench found under a foundation (3 floor house). The case is studied with the AR tool because of a call after finding a deep crack. The user of AR can measure the geometry and send/store this data. It also has information from nearby geotechnical surveys that help him interpret what observes. The videos of what happens are recorded for later analysis.*

assistance programs (repair of minor damages in houses and buildings as well as attention to water pipelines) and larger-scale plans for the correction of structures that are considered not to follow what is established in the code or that require the application of complex engineering solutions (structural reinforcement, total or partial demolitions, complex restoration, control of settlements (fill), repair of sinkholes, etc.).

In this case the teams (engineers, designers, government, and urban planners) are provided with integral tools to evaluate causes-effects and directly determine how many resources the city and community needs to correct the risk situations and how to improve its resilience for future, which is one of the most effective long-term strategies. These virtual collaborative immersive spaces allow different experts from all over the instances to work together in the same virtual environment, creating enhanced and coordinated solutions. Resilience, poverty, security, among others, are social aspects that are presented in maps in superposition on the kind of damages and necessities of reparation. In this way the decisions about resources can be developed on a solidarity base that prioritizes the needs of the most marginalized communities. An additional aspect, very important for the NDM, is that one of the layers shown is the prediction of accelerations in the different geotechnical zones of the city when certain earthquakes attack the metropolis. With these interpretations and the effects, routes can be drawn for the adaptation of services and infrastructure to anticipate the potential scenarios of shutdown of functions in the event of a mega earthquake.
