**4.1 Status Monitor Agent**

The Status Monitor Agent considers all symptoms and observations from environmental and internal data in order to identify and classify events according to their priority and their nature (critical or incipient). Based on internal events and context information, this agent is able to infer new knowledge about the current condition of the vehicle with regard to the availability for operation of its components (i.e. status). In a similar way, environmental data is also considered for detecting and classifying external events in order to keep the situation awareness of the vehicle updated.

Fig. 7. Schematic representation of the autonomous mission generation, replan and repair processes using partial plan representation of the mission plans.

#### **4.2 Mission plan adaptation agent**

10 Underwater Vehicles

• Event: represents a series of correlated symptoms (low power consumption, position drift), Two subclasses of Events are defined: *CriticalEvent* for high priority events and

• Status: links the latest and most updated event information to the systems being monitored

Please note how some of these concepts are related to concepts of the Core Ontology (e.g. an *observation* comes from a *sensor*). These Core Ontology elements are the enablers for the knowledge exchange between service-oriented agents. This will be shown in the

The Plan Application Ontology is used to express the *SAV* of the mission planner agent. It uses concepts originally defined by the *Planning Domain Definition Language* (PDDL). The PDDL language was originally created by Ghallab et al. (1998) to standardise plan representation. Concepts are extracted from the language vocabulary and the language grammar is used for

For the adaptation mission planning process, the Planning Application Ontology also requires concepts capable of representing the diagnosis of incidents or problems occurring in some parts of the mission plan (van der Krogt, 2005). Some of the most important concepts

• Resource: state of an object (physical or abstract) in the environment (vehicle, position,

Please note how some of these concepts are also related to concepts of the Core Ontology (e.g.

The adaptive mission planning process involves the detection of events, the effects that these events have on the mission plan and the response phase. The detection of events is performed by the status monitoring agent. The mission plan diagnosis and repair is undertaken by the

The Status Monitor Agent considers all symptoms and observations from environmental and internal data in order to identify and classify events according to their priority and their nature (critical or incipient). Based on internal events and context information, this agent is able to infer new knowledge about the current condition of the vehicle with regard to the availability for operation of its components (i.e. status). In a similar way, environmental data is also considered for detecting and classifying external events in order to keep the situation

• Action: Modification of the state of resources (calibrate, classify, explore, etc.),

describing the relationships and constraints between these concepts.

IncipientEvent for the remaining ones.

(e.g. sidescan transducer),

demonstration scenario of Section 5.3.

**3.3.2 Planning Application Ontology**

identified for mission plan adaptability are:

• Execution: When an action is executed successfully, • Failure: An unsuccessful execution of an action,

a list of *capability* concepts is required to perform a mission *action*).

• Gap: A non-executable action,

**4. Adaptive mission planning**

adaptive mission planner agent.

awareness of the vehicle updated.

**4.1 Status Monitor Agent**

sensor, etc.),

This section describes the mission environment model used by the mission plan repair techniques presented in this chapter. We continue using the mission environment model previously described. We generalise this model to any step *q* in the mission execution timeline. An instance of an UUV mission environment at a given step *q* can be simply defined as Π*<sup>q</sup>* = Σ*q*, Ω*<sup>q</sup>* . The mission domain model Σ*<sup>q</sup>* contains the set of propositions defining the available resources in the system *PV* and the set of actions or capabilities *AV*. The mission problem model Ω*<sup>q</sup>* contains the current platform state *xq* and the mission requirements *QO*.

Based on this model, we analyse how to calculate mission plans on the plan space (Sacerdoti, 1975). A plan space is an implicit directed graph whose vertices are partially specified plans and whose edges correspond to refinement operations. In a real environment where optimality can be sacrificed by operability, partial plans are seen as a suitable representation because they are a flexible constrained-based structure capable of being adapted.

A partial plan *ψ* is a tuple containing a set of partially instantiated actions and a set of constraints over these partially grounded actions. Constraints can be of the form of ordering constraints, interval preservation constraints, point truth constraints and binding constraints. Ordering constraints indicate the ordering in which the actions should be executed. Interval preservation constraints link preconditions and effects over actions already ordered. Point truth constraints assure the existence of precondition facts at certain points of the plan. Binding constraints on the variables of actions are used to ground the actions to variables of the domain. Figure 8 shows a partial plan representation of an UUV mission. Partial plans are flexible to modification. They provide an open approach for handling extensions such as temporal and resource constraints. Due to nature of the constraints, it is easy to explain a partial plan to a user. Additionally, it is easily extensible to distributed multi-agent mission planning.

Figure 7 explains the processes of mission plan repair and mission plan replan for mission plan adaptation for UUVs using a partial plan representation of the mission plans. At the initial step, a partial ordered plan *ψ*<sup>0</sup> is generated satisfying the original mission environment Π0. The *ψ*<sup>0</sup> is then grounded into the minimal mission plan *π*<sup>0</sup> including all constraints in *ψ*0. At step *q*, the semantic knowledge-based framework is updated by the diagnosis information Π˙ *<sup>q</sup>* providing a modified awareness of the mission environment Π*q*. From here, two mission adaptation processes are possible: *Mission replan* generates a new partial plan *ψq*, as done at the first stage, based only on the knowledge of Π*q*. On the other hand, *mission plan repair* re-validates the original plan by ensuring minimal perturbation of it. Given the partial plan at the previous step *<sup>ψ</sup>q*−<sup>1</sup> and the diagnosis information <sup>Π</sup>˙ *<sup>q</sup>*, the mission repair problem produces a solution partial plan *ψ<sup>q</sup>* that satisfies the updated mission problem Π*q*, by modifying *ψq*−1. The final step for both approaches is to ground *ψ<sup>q</sup>* to its minimal mission plan

Plan repair uses a strategy to repair with new partial plans the plan gaps identified during the plan diagnosis stage. Our approach uses an iteration of unrefinement and refinement

<sup>211</sup> Embedded Knowledge and Autonomous Planning:

Planning in the plan space is slower than in the state space because the nodes are more complex. Refinement operations are intended to achieve an open goal from the list of mission requirements or to remove a possible inconsistency in the current partial plan. These techniques are based on the least commitment principle, and they avoid adding to the partial plan any constraint that is not strictly needed. A refinement operation consists of one or more of the following steps: adding an action, an ordering constraint, a variable binding constraint

A partial plan is a solution to the planning problem if has no flaw and if the sets of constraints are consistent. Flaws are either subgoals or threats. Subgoals are open preconditions of actions that have not been linked to the effects of previous actions. Threats are actions that could introduce inconsistencies with other actions or constraints. We implemented a recursive non-deterministic approach based on the Partial ordered Planning (PoP) framework (Penberthy & Weld, 1992). This framework is sound, complete, and systematic. Unlike other Plan space planners that handle both types of flaws (goals and threats) similarly, each PoP recursive step first refines a subgoal and then the associated

In our implementation, we introduce a previous step capable of performing an unrefinement of the partial plan when necessary. During the unrefinement strategy we remove refinements from the partial plan that are reported by the plan diagnosis phase to be affecting the consistency of the mission plan with the mission environment, i.e. to remove constraints and

In simple terms, when changes on the *ABox* Planning Application Ontology are sensed (Π˙ *<sup>q</sup>*) that affect the consistency of the current partial plan *ψq*−1, the plan repair process is initiated. The plan repair stage starts an unrefinement process that relaxes the constraints in the partial

new mission environment. However, this relaxation could open some subgoals and introduce threats in the partial plan that need to be addressed. The plan repair stage then executes a refinement process searching for a new mission plan *ψ<sup>q</sup>* that is consistent with the new mission environment Π*<sup>q</sup>* and removing these possible flaws. By doing this, it can be seen that the new mission plan *ψ<sup>q</sup>* is not generated again from Π*<sup>q</sup>* (re-planned) but recycled from *ψq*−<sup>1</sup> (repaired). This allows re-use of the parts of the plan *ψq*−<sup>1</sup> that were still consistent with Π*q*.

The combination of the status monitor agent, the adaptive mission planner, the mission executive and the semantic knowledge-based framework is termed as the Semantic-based Adaptive Mission Planning system (SAMP). The SAMP system implements the four stages

The status monitor agent reports to the knowledge base the different changes occurring in the environment and the modifications of the internal status of the platform. The knowledge base stores the ontology-based knowledge containing the expert orientation provided *a priori* and the observations reported by the status monitor. A mission planner agent generates and

of the OODA-loop. Figure 9 represents the customised version of Figure 3 for SAMP.

*<sup>q</sup>*−<sup>1</sup> is now relaxed to be able to cope with the

strategies on a partial-ordered planning framework to adapt the mission plan.

The Path Towards Permanent Presence of Underwater Networks

or a causal link.

threats (Ghallab et al., 2004).

finally the actions if necessary.

**5. Results**

**5.1 Architecture**

plan *ψq*−<sup>1</sup> that are causing the mission plan to fail. The remaining temporal mission partial plan *ψ*�

Fig. 8. Example of a partial ordered plan representation of an autonomously generated UUV mission. The ordering constraints are represented using the graph depth, interval preservation constraints are represented with black arrows, point truth constraints are represented with PTC-labelled arrows, and binding constraints are shown in the top left box.

*πq*. It can be seen that mission repair better exploits the orientation capabilities for decision making: instead of taking the new mission environment as a given, it uses the diagnosis information about the changes occurred to guide the adaptation process.

We have now identified the benefits of mission plan repair over mission replan. Mission plan repair modifies the partial plan *ψq*, so that it uses a different composition, though it still maintains some of the actions and the constraints between actions from the previous partial plan. However, mission plan adaptation can also be achieved by mission execution repair by looking directly at the mission plan instantiation *πq*. Execution repair modifies the instantiation of the mission plan *<sup>π</sup><sup>q</sup>* such that a ground action *<sup>g</sup>ah <sup>q</sup>* that was previously instantiated by some execution *eq* is newly bound by another action execution instance *e q*.

Executive repair is less expensive and it is expected to be handled directly by the mission executive agent. Plan repair, however, is computationally more expensive and requires action of the mission planner agent.

The objective is to maximise the number of execution repairs over plan repairs and, at the plan repair level, maximise the number of decisions reused from the previous mission instantiation. The information provided by the semantic-base knowledge base during the plan diagnosis phase is critical.

Executive repair fixes plan failures identified in the mission plan during the diagnosis stage. Our approach uses ontology reasoning in combination with an action execution template to adapt the mission plan at the executive level.

Once a mission plan *π<sup>q</sup>* is calculated by the mission planner, its list of ground actions is transferred to the executive layer. In this layer, each ground action *gah <sup>q</sup>* of *π<sup>q</sup>* gets instantiated into an action execution instance *e<sup>t</sup> <sup>q</sup>* using the action template for the action *ah* available in the Core Ontology of the knowledge base. At the end of this phase, each *e<sup>t</sup> <sup>q</sup>* contains the script of commands required to perform its correspondent ground action. Flexibility in the execution of an action instance is critical in real environments. This is provided by a timer, an execution counter, a time-out register and a register of the maximum number of executions in the action execution instance. Additionally, three different outputs control the success, failure or time-out of its execution. These elements handle the uncertainty during the execution phase and enable the executive repair process. This minimise the number of calls to the adaptive mission planner agent and therefore the response time for adaptation.

Plan repair uses a strategy to repair with new partial plans the plan gaps identified during the plan diagnosis stage. Our approach uses an iteration of unrefinement and refinement strategies on a partial-ordered planning framework to adapt the mission plan.

Planning in the plan space is slower than in the state space because the nodes are more complex. Refinement operations are intended to achieve an open goal from the list of mission requirements or to remove a possible inconsistency in the current partial plan. These techniques are based on the least commitment principle, and they avoid adding to the partial plan any constraint that is not strictly needed. A refinement operation consists of one or more of the following steps: adding an action, an ordering constraint, a variable binding constraint or a causal link.

A partial plan is a solution to the planning problem if has no flaw and if the sets of constraints are consistent. Flaws are either subgoals or threats. Subgoals are open preconditions of actions that have not been linked to the effects of previous actions. Threats are actions that could introduce inconsistencies with other actions or constraints. We implemented a recursive non-deterministic approach based on the Partial ordered Planning (PoP) framework (Penberthy & Weld, 1992). This framework is sound, complete, and systematic. Unlike other Plan space planners that handle both types of flaws (goals and threats) similarly, each PoP recursive step first refines a subgoal and then the associated threats (Ghallab et al., 2004).

In our implementation, we introduce a previous step capable of performing an unrefinement of the partial plan when necessary. During the unrefinement strategy we remove refinements from the partial plan that are reported by the plan diagnosis phase to be affecting the consistency of the mission plan with the mission environment, i.e. to remove constraints and finally the actions if necessary.

In simple terms, when changes on the *ABox* Planning Application Ontology are sensed (Π˙ *<sup>q</sup>*) that affect the consistency of the current partial plan *ψq*−1, the plan repair process is initiated. The plan repair stage starts an unrefinement process that relaxes the constraints in the partial plan *ψq*−<sup>1</sup> that are causing the mission plan to fail.

The remaining temporal mission partial plan *ψ*� *<sup>q</sup>*−<sup>1</sup> is now relaxed to be able to cope with the new mission environment. However, this relaxation could open some subgoals and introduce threats in the partial plan that need to be addressed. The plan repair stage then executes a refinement process searching for a new mission plan *ψ<sup>q</sup>* that is consistent with the new mission environment Π*<sup>q</sup>* and removing these possible flaws. By doing this, it can be seen that the new mission plan *ψ<sup>q</sup>* is not generated again from Π*<sup>q</sup>* (re-planned) but recycled from *ψq*−<sup>1</sup> (repaired). This allows re-use of the parts of the plan *ψq*−<sup>1</sup> that were still consistent with Π*q*.
