**3. Ant Colony Optimization and Ant-Routing Algorithms**

Ant algorithms are inspired by the natural foraging behavior of certain species of ants. Based on the famous double bridge experiment, reported on by Goss, Aron, Denebourg and Pasteels in 1989 [28], ant-inspired optimization was then codified into an *Ant Colony Optimization* metaheuristic [29] which was originally implemented in algorithms such as *Ant System* [30] and *Ant Colony System* [31].

In general, algorithms using the *Ant Colony Optimization* metaheuristic work as follows: an optimization problem is transformed into a graph *G* ¼ ð Þ *V*, *E* , ants travel along the graph using pheromones (if present) to choose a path stochastically and after the ants have finished their travel, the pheromone values in the graph are updated according to the "goodness" of the solutions found by the ants. Many algorithm variants, also improve their results with a local search phase that is applied before updating the pheromone values. Besides other combinatorial

optimization problems, these algorithms have been shown to be able to solve the traveling salesman problem. In *Ant System*, the first algorithm to implement the *Ant Colony Optimization* metaheuristic, the ants choose their path according to Eq. (1):

$$p\_{ij}^k = \begin{cases} \frac{\pi\_{ij}^a \eta\_{ij}^\beta}{\sum\_{c\_{il} \in N(s^p)} \pi\_{il}^a \eta\_{il}^\beta} & \text{if} \quad c\_{ij} \in N(s^p), \\\ 0 & \text{otherwise} \end{cases} \tag{1}$$

where an ant *k* in a city *i* chooses the next city *j* with probability *p<sup>k</sup> ij* with *s <sup>p</sup>* the partial solution constructed so far and *N s<sup>p</sup>* <sup>ð</sup> the set of possible edges leading only to cities not visited so far. The parameters *α* and *β* balance the importance of pheromone versus the local heuristic *ηij* ¼ 1*=dij* with *dij* the distance between city *i* and city *j*.

Pheromones are updated using the update rule in Eq. (2):

$$
\tau\_{\vec{\eta}} \leftarrow (\mathbf{1} - \rho) \cdot \tau\_{\vec{\eta}} + \sum\_{k=1}^{m} \Delta \tau\_{\vec{\eta}}^{k}. \tag{2}
$$

with *ρ* the evaporation rate, *m* the number of ants and Δ*τ<sup>k</sup> ij* proportional to the inverse of the lenght of the tour ant *k* took if that link was chosen (0 otherwise).

A variant aimed specifically at (wired) networks is *AntNet* [32]. These algorithms were not developed with Wireless Multi-Hop Networks in mind, though. As described above, Wireless Multi-Hop Networks have their own challenges in addition to the challenges of routing in a fixed network.

In the following, we will describe the seminal ant routing algorithm developed for Wireless Multi-Hop Networks, *AntHocNet* [33], and then give an overview of the typical features of other ant routing algorithms for these types of networks.

## **3.1 AntHocNet**

*AntHocNet* [33], 2005, by Di Caro, Ducatelle, and Gambardella is the seminal ant algorithm developed for mobile ad-hoc networks. It addresses the special challenges that such wireless networks pose: bandwidth is typically less than in fixed networks, and links can change their quality or break. Therefore, *AntHocNet* is realized as a hybrid algorithm that combines features from pro-active and reactive routing protocols. In this way, it does not waste resources to set up paths before any packet is sent, which might not exist anymore by the time they are eventually needed.

Like all wireless routing algorithms, nodes running *AntHocNet* need to determine which other nodes are reachable by wireless transmission (iow. the one-hop neighborhood). *AntHocNet* nodes do this by broadcasting very short "hello" messages at regular intervals. Receiving nodes then set up these neighboring nodes in their respective routing tables, but without any routing information yet. These "hello" messages are also used to detect link failures.

When a new data packet is to be sent from a source node *s* to a destination node *d*, the algorithm enters its reactive path setup phase. There exist two possibilities: either there already exists routing information for a path between *s* and *d* (after the protocol has run for a while and packets have already been sent) or not. Depending on whether routing information already exists or not, *AntHocNet* sends so-called "forward ants" either by broadcasting them (if no routing information for the required route exists yet) or by unicasting them stochastically along one of the

already known routes. For unicasting, the pheromone routing tables at the intermediate nodes are exploited and the next hop *n* towards the destination *d* is chosen stochastically with a probability *Pnd* according to Eq. (3):

$$P\_{nd} = \frac{\left(\boldsymbol{T}\_{nd}^{i}\right)^{\beta}}{\sum\_{j \in \mathcal{N}\_d^i} \left(\boldsymbol{T}\_{jd}^i\right)^{\beta}}, \beta \ge \mathbf{1}.\tag{3}$$

with *<sup>i</sup>* the current node, *<sup>n</sup>* the next hop, <sup>T</sup> the respecitve pheromone value, <sup>N</sup> *<sup>i</sup> d* the set of possible neighbors and a coefficient *β* that controls how explorative the algorithm behavior is.

If there is no pheromone information available yet, the forward ant is broadcasted. To avoid flooding the network with too much traffic, these broadcast ants are restricted in several ways: 1) after a number of hops, they are killed, 2) when a node receives several ants stemming from the same broadcast (that took different paths to reach this node), it will only pass on those ants which came via sufficiently good paths (using the number of hops and travel time as metrics). The threshold for this can be set by a parameter *α*1. In this way, several parallel paths can be explored while the worst paths are quickly excluded and overhead (which is always of special concern in wireless networks) is kept at a reasonable level. A second parameter *α*<sup>2</sup> is used to spread paths more widely among the network: broadcast ants which took a different first hop than previous ants stemming from the same broadcast, this less restrictive parameter *α*<sup>2</sup> is applied instead of *α*1.

Ants memorize the path they travel and when a forward ant has reached the destination node, a so-called backward ant is created which travels back the path P ¼ *s* ! *n*<sup>1</sup> ! *n*<sup>2</sup> ! … ! *d* it came. This backward ant then updates all the pheromone information along the path according to Eq. (4):

$$T\_{nd}^{i} = \gamma T\_{nd}^{i} + (\mathbf{1} - \boldsymbol{\gamma})\tau\_{d}^{i}, \quad \boldsymbol{\gamma} \in [0, 1]. \tag{4}$$

where *τ<sup>i</sup> <sup>d</sup>* is an expression of the "goodness" of the path, based on an estimate of the average time to send a packet over the path P calculated from measurements at each node's MAC layer.

After one or several path(s) has been found and while a data session is running, *AntHocNet* forwards the data packets stochastically along all the available paths using the same Eq. (3) as the forward ants but with a higher value of *β*. This means that data packets have a more exploitative behavior than ant packets which explore more.

During a data session, *AntHocNet* enters its pro-active phase and sends forward ants in addition to the data packets. These again use Eq. (3) but have a small probability of being broadcast instead. The ants that follow the existing path via Eq. (3) update the current quality of the existing path while those ants that are broadcast can potentially find new, better paths which will then be immediately used as potential paths to route data. Due to the way paths are determined and updated during the pro-active phase and due to the stochastic nature of the data routing, data packets are sent in an automatically load-balanced way through the network which expecially helps with wireless transmission as two parallel paths use the same transmission medium and therefore can potentially greatly influence each other.

*AntHocNet* also addresses link failures, which occur much more frequently in the wireless domain. As mentioned before, link failure can be detected via "hello" messages – if there has not been an "hello" message for a certain amount of time

(several times the regular sending interval), a link to a node will be considered broken. A link is also considered broken if a unicast message to a node fails. The algorithm then enters its local path repair mode where it broadcasts so-called "path repair ants" that work just like the forward ants in reactive mode, except that they are more limited in their maximum number of allowed broadcasts. If a path can be repaired within a certain amount of time, the data packets (which will have been buffered in the meantime) will be sent to the destination node. If the path can not be re-established within a reasonable time limit, the data is discarded and link failure notifications are broadcasted to the surrounding nodes.
