Computational Biology and Chemistry

E.


Introduction
In the paradigm run-and-tumble model of chemotaxis, bacterial cells of E. coli or Salmonella drift towards spatial regions optimum for growth and survival (e.g. with high nutrient concentration) by means of a motile system consisting of around six helical flagella, each with a rotary motor at its base, embedded in the cell wall.If the motors turn counter-clockwise (CCW), the flagella come together to form a spinning bundle and the cell swims forward (a run) for ∼1.0 s.If one or more of the motors turn clockwise (CW), the bundle becomes unstable and the cell turns in place at random (a tumble) for ∼0.1 s, with no net displacement (Berg and Brown, 1972).The purpose of a tumble is to reorient the cell to a new (random) direction.In the absence of an attractant gradient, this results in a diffusive random walk with a diffusion constant of ∼4 × 10 −6 cm 2 /s (Berg, 2003).
In the presence of a gradient of attractant (or repellent), the bacteria use temporal comparisons of the attractant concentration over the preceding ∼3-4 s to determine if conditions are improving or deteriorating (Clark and Grant, 2005;Hazelbauer et al., 2008).If the former, tumbles are suppressed, increasing the mean run length, and the cells drift in their current, favourable direction (within limitations due to Brownian forces).Conversely, when conditions deteriorate, the cell increases its chances of moving in a favourable new direction by tumbling.This combination of runand-tumble swimming and bias control leads to a drift velocity (in steep gradient of attractant) of around ∼7 m/s (Berg and Turner, 1990).The response to attractant gradients is consistent with the Weber-Fechner law and gives a constant bias change in exponential gradients (Dahlquist, 2002).
A detail often omitted from most mathematical models of E. coli chemotaxis (e.g.see the review by Tindall et al., 2008) is that during a tumble, reorientation is not perfect or complete, i.e. the new direction is not chosen completely at random; instead, the bacterium is more likely to reorient within the forward hemisphere, so that the average angle between successive runs is not 90 • (corresponding to orthogonality and therefore complete reorientation) but ∼63 • (Berg and Brown, 1972).This was computed by tracking swimming bacteria, recording the directional bearing during each individual straight swim (run) and computing the absolute mean difference between successive run directions.Recently, Locsei (2007) analysed a model of run-and-tumble with directional persistence and found that, in the weak chemotaxis limit, this so-called 'persistence of direction' can increase the drift velocity of a bacterium up an attractant gradient by up to 38% relative to the memoryless case.
However, it was also found in this analysis that the 'persistence parameter' ˛that maximises the drift velocity (˛= 0.78) is larger by a factor of more than 2 compared with the value observed in live E. coli (˛= 0.33).This is intriguing, since in the absence of other influences, one might expect evolution to have optimised the search strategy so that the persistence observed in E. coli is close to the theoretically determined value that maximises the drift velocity (if persistence confers some selective advantage).A possible explanation for the discrepancy may be that increasing drift velocity is not the only determinant of optimal foraging.The in vivo persistence of direction may reflect a compromise between two (potentially competing) aims, (1) to move quickly towards nutrients and (2) to spend most of the time in regions of high nutrient density (Locsei, 2007), once these are located.While drift velocity is important for (1), it says nothing at all about (2).A more balanced measure of the efficiency of the chemotactic strategy is the overall (or mean) quantity of nutrient located over time.In this paper we investigate the effect of directional memory on the optimality of run-and-tumble chemotaxis by using this measure.
A further piece of evidence in favour of directional persistence serving some function is that, in E. coli, the rate of signal termination and adaptation is larger than would be expected through dephosphorylation alone.The chemotactic pathway directly intervenes, via the protein CheZ, to terminate the signal 'early' (Bren et al., 1996).Since longer tumble times lead to more randomization (less persistence), terminating the chemotactic signal early may be a way to enforce a shorter tumble time, thus implementing a particular value of directional persistence.
Thinking of evolution as a process of stochastic optimisation (that is, optimisation that incorporates probabilistic elements, either in the problem data or in the algorithm itself through random parameter values or random choices) over the fitness landscape (Edwards, 1987) leads to the natural question of when this 'algorithm' is able to find the optimal solution to such an optimisation task.In general, due to the constraints on evolution, it is not guaranteed that evolution would in fact visit the global optimum of this landscape (or remain there if this occurred).This would depend on a number of factors such as the ruggedness and structure of the landscape, mutation-selection balance, the mutation or replication rate, the distribution of mutational jump distances on the landscape, competition with other species and stochasticity (drift).We therefore suggested recently (Nicolau et al., in preparation) that an evolutionary strategy might be called evolutionarily optimal if it is optimal up to the limitations imposed by evolution as an optimisation strategy.Our investigation of the role of directional persistence is also motivated by this concept of the evolutionary (as opposed to global) optimality of a biological strategy.
To gain insight into the evolution of directional persistence in E. coli chemotaxis, and to understand its relationship to the optimal chemotactic strategy, we here pursue two complementary analyses.We firstly use direct Monte Carlo simulations of a model of run-andtumble chemotaxis to determine the effect of directional memory on the efficiency of run-and-tumble in environments with different attractant gradients.We then simulate the evolution of directional memory (in isolation from other chemotactic mechanisms) in the same environments and compare the results with those from the direct simulation.

Methods
Our simulations in this paper are based on the model of bacterial chemotaxis proposed by Segall and Block (1986) and Schnitzer (1993).n bacteria are initially placed (and oriented) at random in the 2D simulation space.An attractant is also placed in this space, with a linear gradient so that the attractant concentration at the point (x, y) is given by where k is the gradient, (x 0 , y 0 ) is the origin and c max is the concentration of attractant at the peak of the gradient.We assumed a linear gradient because the slope k is a constant, allowing different steepnesses of gradient to be directly compared.Simulations run with a Gaussian attractant confirm that this does not change the findings reported here.
At each pass of the algorithm, each bacterium is visited in turn, similarly to the algorithm described for molecular motion in Nicolau et al. (2007).In modelling chemotaxis, we follow the approach of Clark and Grant (2005).The cells compare their average receptor occupancy, approximated by the values of C, between 4 and 1 s in the past, c 1-4 , to the average receptor occupancy during the past 1 s, c 0-1 , to produce the biaser b = c 0-1 − c 1-4 .If b > 0, the cell reduces the tumbling rate tumble from the ambient value 0 by an amount dependent on b: tumble = 0 − ␥f(b), where f(b) is a monotonically increasing function of b and is a sensitivity coefficient that is positive for positive chemotaxis and negative for negative chemotaxis.Following Clark and Grant (2005), we assume here a linear relation, i.e. f(b) = b.We show in recent work (Nicolau et al., in preparation) that simulations of the evolution for a digital genotype coding for chemotactic swimming strategy indicate that the characteristic time period of integration (the recent and distant windows) is optimised by evolution to compensate for environmental noise.Here we assume the scale of temporal comparisons experimentally determined for E. coli (1 and 3 s, for the recent and distant windows respectively, see Nicolau et al., in preparation).Without loss of generality, we set , the sensitivity of the response of the bacterium to changes in attractant concentration, to 1.If b < 0, tumble is retained at the ambient value 0 .
If a bacterium, i, is in a "run", its orientation Â i and position p i are updated according to the system of equations: where D rot is the rotational diffusion coefficient (set to 0.15 rad 2 /s, the value in a mammalian gut, see Strong et al., 1998), and v is the mean velocity of the bacterium (25 m/s, Darnton et al., 2007).
After each computation of the new position and orientation, the simulation time t is incremented by an increment ıt (0.1 s in this paper).A random number between 0 and 1, r, is compared to 1/ tumble .If r < 1/ tumble then the cell tumbles and chooses a new direction: where ϑ tumble is the persistence of direction parameter.Note that the difference between (3) and the reorientation in Eq. ( 2) is in the magnitude of ϑ tumble .Persistence is defined differently here than in Locsei (2007) because of the different modelling approaches in the two studies.If ϑ tumble = 0 then the new direction is identical to the previous direction (no reorientation).Large values of ϑ tumble result in the new direction being independent of the old direction (perfect reorientation).In E. coli, the mean angle between successive run directions has been found to be ϕ ≈ 63 • (Locsei, 2007), corresponding to a directional persistence of ≈1.1 rad in our model.

Direct Simulation
We first investigated the effect of ϑ tumble on the mean quantity of attractant located by a population of n bacteria (Fig. 1).All simulation results presented here are obtained with n = 200.The simulations were run for 1000 s of real time (based on the swimming speed and body size of WT E. coli).Both of these values reflect a compromise between numerical accuracy and computational resources.Unless otherwise stated, all data points in the figures are averages taken over the bacterial population.
The results show a transition between two regimes.If the gradient is shallow (so that attractant is relatively evenly spread), above a characteristic threshold value of ϑ tumble (greater than 1.0 rad or ∼55 • ), all strategies are approximately equal.Above this value, the total attractant acquired plateaus at a level dependent mainly on the gradient.This threshold value varies very weakly with increasing gradient.Variations in efficiency at high ϑ tumble are dominated by simulation stochasticity.At high gradients, however, a maximum efficiency is reached for values of ϑ tumble between 0.9 and 1.1 rad.Values larger than this result in reduced efficiency.The effect becomes increasingly important as gradients become steeper.The crossover between the two regimes takes place at gradient values of around 1.5 units of concentration per bacterial body diameter.
These data (Fig. 1) indicate that in environments characterised by shallow gradients, a preference for the forward hemisphere is neither beneficial nor harmful as long as the reorientation during a tumble ensures a minimum average reorientation, ϑ tumble > 1.0 (or ∼55 • between runs).Looking at the effect from the opposite end, perfect reorientation is neither necessary nor harmful at shallow gradients.On the other hand, in steep-gradient environments, Fig. 1.Attractant acquired by a population (as a proportion of the maximum attainable) versus the persistence parameter used by individuals in that population in different gradient environments (denoted "grad").A minimum degree of reorientation (with a persistence parameter greater than a threshold of ∼1.0) is necessary to maximise efficiency; after this point, increasing the degree of reorientation does not improve efficiency and in steeper gradient environments, it reduces it relative to the optimal value.All values are means over the population.complete reorientation reduces the amount of attractant located.Similarly to the low-gradient case, a minimum amount of reorientation must be ensured but in contrast, if the gradient is large, tumbles causing extreme reorientation are harmful.
To further gain insight into this behaviour, we compared the relative performance of the optimal directional persistence strategy (ϑ tumble = 1.1, or ∼60 • between runs), denoted Eff persistence with the perfect reorientation strategy (ϑ tumble large, corresponding to a mean of 90 • between runs), denoted Eff.The results are shown in Fig. 2 and confirm that in shallow gradients, directional memory has no impact on the efficiency of chemotaxis while in high gradients, it outperforms perfect reorientation by around 40-50%.The transition between these two regimes occurs in a small gradient interval between 1.75 and 2 concentration units per bacterial body length.

Evolutionary Model
We now describe a variation of the model that was used to simulate the evolution of directional persistence.Instead of fixing the persistence parameter ϑ tumble in each simulation experiment, each bacterium i is initially assigned an individual value of ϑ tumble (denoted ϑ tumble,i ) of 0, corresponding to a complete inability to tumble.We then allow the values of the persistence parameter to evolve by natural selection over 100 generations.In the first generation, the simulation is run in the same way as described above (t max = 1000 s of real time based on the swimming speed of WT E. coli, n = 200).At the end of the generation, the bacteria that have acquired more attractant than the mean taken over the population are allowed to survive and divide once.The other 50% of the population is removed from the simulation.This method ensures constant population size, for simplicity.
Each daughter cell is identical to the parent except that, to simulate mutation, its persistence is incremented by a random amount where as above, Á = N(0, 1) and m is a positive parameter controlling the rate of mutation.We chose m = 0.4, a value that keeps simulation times manageable but the mean mutation in the persistence parameter to less than 0.25 per generation.Varying this value in the range 0.1-0.6 does not change the outcome of the evolution but affects the speed of the process (the number generations required for a stable strategy to emerge); we therefore chose a relatively large value in order to reduce simulation time.
If, due to the stochastic value of the mutation term in (4), the persistence parameter of an offspring is ever negative, then it is set to 0, since in a real bacterium or moving particle, the persistence can only take a positive value.This cycle is then repeated over 100 generations, keeping track in each generation of the mean amount of attractant acquired and the mean value of the persistence over the population.

Results of In Silico Evolution
Evolutionary sequences were run for different gradient values.Fig. 3 (top) shows the evolution of the mean population directional persistence.The results are broadly consistent with those of the direct simulations.In shallow gradients, because long tumbles are not harmful but short tumbles below the threshold of ϑ tumble > 1.0 (corresponding to a mean angle between runs of ∼55 • ) are detrimental, individuals with large reorientation strategies are not disfavoured while those with tumbles below this threshold are, correspondingly, less able to locate attractant.Selection thus results in an upwards drift of the mean persistence value, which diverges.In steep gradients, however, overly reorienting tumbles are detrimental (see Figs. 1 and 2).There, the tug-of-war between these two effects, namely (1) the need for a minimum randomisation of direction during a tumble and (2) the benefits obtained from limiting the length of tumbles, results in a self-limiting evolutionary trajectory and the mean population persistence plateaus at a value of around ϑ tumble = 1.1 (corresponding to a mean angle between runs of around ∼60 • ) after many generations.Fig. 3 (bottom) shows the mean amount of attractant found in each generation.At shallower gradients, where variations in persistence are not crucial, a plateau is reached very quickly, while in steeper gradients, the emergence of the optimum strategy requires more generations.

Qualitative Features of the Evolutionary Runs
A different way to visualise the evolution data is to plot persistence on one axis and the efficiency (attractant found) on the other, as in Fig. 4.These series (which we call "evolution characteristics" in what follows) contain three important pieces of information.Firstly, the total length of each characteristic in the horizontal direction shows the digital "genotype space" explored by the population.Thus a horizontally short characteristic indicates a self-limiting evolutionary process with a (finite) optimal genotype, as is the case in the large-gradient case here (bottom characteristic).A long characteristic indicates no equilibrium (corresponding to a global or local optimum) was found in the evolutionary run (but that does not mean that one does not exist).
Secondly, the vertical size of each characteristic is an indication of the sensitivity of the fitness (attractant found, in this case) to the genotype (persistence, in this case) over the generations.Consistently with the discussion above, the sensitivity of the efficiency on the reorientation strategy is highest in steep gradient environments (bottom characteristic, shown with vertical braces) and lowest in shallow gradient environments (top curve).
Thirdly, each characteristic shows two regimes along the horizontal axis, (a) an initial rapid increase in the population mean persistence from 0 (points scattered along the x-axis) and (b) a cluster of points, indicating the emergence of a stable or quasistable equilibrium in the population.The elongation of this cluster is an indication of the sensitivity of selection associated with digital genotype changes near the equilibrium.Thus, the equilibrium cluster for the high-gradient case is the most compact, because individuals with ϑ tumble > 1.25 or ϑ tumble < 0.9 are very unlikely to reproduce (x-direction) and because small changes in ϑ tumble near the equilibrium ϑ tumble result in large changes in fitness (ydirection).

The Role of Persistence and Exploration versus Exploitation
We report that in a model of run-and-tumble chemotaxis in the presence of a linear attractant profile, a moderate directional persistence is beneficial (in sharp gradient environments) or at least neutral (in shallow gradient environments) as long as the randomisation during a tumble is above a threshold value of ∼55 • .These results are consistent with the conclusions of Locsei (2007), namely that directional persistence is beneficial, presumably by increasing drift velocity.We find that, in the former regime, the optimal value of ϑ tumble = 1.1 ± 0.15, corresponding to ϕ = 63 • ± 9 • between runs, while in E. coli, ϕ ≈ 63 • between runs.This finding is supported independently both by the in silico evolution data and by a direct brute force search through the range of possible values of ϑ tumble .These results suggest that persistence of direction not only increases chemotactic drift in isolation from any other aims of the chemotactic strategy (such as remaining in a region of high nutrient concentration, once having drifted there), but is in fact both evolutionarily and globally optimal in the context of run-and-tumble and specifically at a parameter set very near to that found in vivo and identified by simulations of the model.Its effect is substantial but moderate: even in the steep-gradient regime (k = 2-2.5),45% more attractant is located with a ϑ tumble = 1.1 strategy than with a ϑ tumble = 5 (complete reorientation) strategy.This is consistent with the analysis of Locsei, who found a difference of 38% in mean drift velocity up a gradient between the two equivalent cases (Locsei, 2007).In the shallow-gradient regime, we report that, in the framework of our model, persistence of direction is neither beneficial nor detrimental, so long as a minimum amount of reorientation takes place during a tumble.
These results are not counter-intuitive.If persistence of direction is a means of increasing chemotactic drift velocity, then one would expect its impact to be greatest in steep-gradient environments, in which this function is more important.More generally, in shallow-gradient environments, the specific chemotactic strategy and chemotactic ability in a broad sense, would not be expected to be as important as in steep-gradient environments.This point can be illustrated by noting that in the limit as the gradient tends to 0, an organism with no chemotactic ability (e.g. an immotile bacterium diffusing at random) would locate as much attractant as one with a highly efficient and sophisticated chemotaxis mechanism.Conversely, in the limit as the gradient becomes large, differences in attractant concentration become large over small distances and hence the ability to move to favourable regions quickly becomes increasingly important.
Why might evolution select for directional memory, i.e. why might directional memory increase chemotactic efficiency?A straightforward answer is that since tumbles that reorient to a level just above the minimum threshold (Fig. 3) result in maximum efficiency, there is no selective advantage from spending large amounts of time in unnecessarily long reorienting tumbles without net displacement in a favourable direction.Therefore minimally long tumbles may simply be a way to partition swimming time more efficiently (since tumbles result in little net displacement and therefore no favourable drift).
It has also recently been shown using a continuous mathematical model of E. coli chemotaxis that persistence of direction increases drift velocity up a gradient (Locsei, 2007).However, we speculate that this is not the only direct (as opposed to indirectly increasing efficiency by reducing tumbling time) reason.Firstly, our simulation times are so large (1000 s) compared with the average drift time (tens of seconds for the gradients used here) that the vast majority of the simulation time is spent near the maximum, drifting up and down a narrow portion of the gradient, where this effect would be small.Secondly, increasing drift velocity would be expected to have a larger effect when the typical gradient distance is large (shallow gradients) and a smaller one when the width is smaller (sharp gradients).In contrast, our results indicate that the directional persistence function is only useful in sharp-gradient environments, where again this effect would be less important.
We propose that another effect is at play in addition (not in opposition) to increased drift velocity.This is based on the observation that a very large value of persistence (reorientation) is associated with a significant probability of choosing a new direction that is close to the old direction due to the periodicity of direction; that is, reorienting for a long time is more likely to result in a new direction close to the previous direction, than reorienting for a moderate time.Thus, a slight preference for the forward hemisphere decreases the chance of choosing such a direction and paradoxically may be better at producing reorientation steps that result in a new direction that is on average perpendicular to the old direction (i.e.complete orthogonal reorientation).This may lead to a more efficient algorithm for partitioning the search space (Hanson et al., 2006).
It is intriguing that the optimal value of ϑ tumble varies very weakly with attractant gradient.In the shallow gradient regime, the ϑ tumble threshold above which tumbles are sufficiently disorientating is also a weak function of the gradient.Over all the cases considered here, we find (both through in silico evolution and direct Monte Carlo simulation) that 0.95 < ϑ tumble < 1.25 is always optimal.This is a rather narrow window, corresponding to only a ∼15 • difference in forward hemisphere preference.One possibility is that the optimal range would be different for different attractant profiles.However, in our simulations, using different gradient forms (e.g.Gaussian, quadratic, etc.) does not considerably affect the results.Using randomly placed multiple, additive, sources of attractant with these forms also does not alter our findings (data not shown).
We speculate that the slimness of this optimal range results from a compromise between two different aims of spatially "blind" foraging strategies (without the ability to compute gradients locally, relying instead on temporal comparisons) and that it is a universal property of such algorithms in two dimensions.The first of these two aims is to explore as much of the available space as possible in the shortest time possible.This is maximised by traveling in a straight line with no reorientations, ϑ tumble = 0 (ballistic motion).The second aim is to explore (or occupy) space locally once a favourable region has been found.This is achieved with frequent reorientations and large values of ϑ tumble and in the limit as this parameter tends to infinity, leads to pure diffusion or sub-diffusion.
These two aims are competing.The region between the two limits corresponds to superdiffusion, a diffusion-like process characterised by the mean squared distance of a particle (organism) from its starting site varying with a fixed power of time called the anomalous exponent (usually denoted ˛), with 1 < ˛< 2 (Nicolau et al., 2007).If small changes in the exploration strategy (captured by the anomalous exponent) lead to large changes in the respective benefits obtained independently from these two functions (exploration and exploitation), then the region of compromise would be expected to be narrow.This in turn would translate, in the case of superdiffusion by directional memory (as opposed to e.g.Levy flights), into a narrow region for ϑ tumble .Measurements of ϑ tumble or comparable parameters for other foraging systems would confirm (or deny) this supposition.For example, Hanson et al. (2006) report that filamentous fungi exploring microconfined geometries launch daughter hypha at ∼70 • to the parent hypha -not 90 • , which would maximise the efficiency of spatial partitioning.
Deviations from diffusive search behaviour to subdiffusion or superdiffusion appear to be ubiquitous at all biological scales (Viswanathan et al., 1999) including cell membranes (Nicolau et al., 2007) and human travel (Brockmann et al., 2006).These strategies have presumably evolved in order to maximise the efficiency of searching in different kinds of environments.Understanding the relationship between the distribution of a resource of interest in the environment and the evolved search strategy (and its super-, respectively, sub-diffusivity) is an important challenge in the study of foraging behaviour and its evolution.
In recent work (Nicolau et al., in preparation) we show how a run-and-tumble-like chemotactic swimming strategy can emerge naturally from simulations of an evolutionary process and how in silico evolution can reproduce the observed properties of E. coli chemotaxis and related bacteria down to fine quantitative details such as the scale of temporal comparisons, running and tumbling times and the persistence-of-direction property.Given these findings and the very high efficiency predicted here for run-and-tumble chemotaxis, we might expect that all bacteria employ this foraging strategy.There are myriad factors not considered in this crude model, including energy consumption, bacterial size, turbulence in the environment and number of attractant or repellent sources.Several different chemotaxis strategies for bacteria have been identified to date, including run-and-stop and run-and-reverse (Porter et al., 2008).Presumably these have arisen because different environments place different pressures on foraging micro-organisms.
A challenge for the future is to shed light on the factors that contribute to the efficiency of a foraging strategy in a particular type of environment.For example, are strategies that use directional memory or superdiffusion efficient in a patchy food environment?If so, how does the optimal anomalous exponent or directional memory parameter vary with the patch density and size distribution?Are the efficiencies of run-and-tumble, run-and-reverse and run-andstop different relative to the characteristic environments in which the respective bacteria live?

Conclusions
We considered the variation in chemotactic efficiency as a function of the persistence of direction (and its evolution) for model bacteria resembling E. coli.We used a simplified version of a detailed earlier model to investigate this in depth by directly representing persistence of direction as a parameter in the model rather than as an effect due to time spent tumbling.A brute force search of the persistence parameter space shows that a minimum value of persistence of direction of ∼60-65 • is required to achieve optimal chemotaxis in shallow gradients but that in steep gradients there is a small penalty associated with overly reorienting tumbles.Evolutionary simulations confirm these findings and show that a digital population evolves a persistence corresponding to that measured in E. coli only in steep gradients but that in shallow gradients large reorientations are favoured.We speculate on the reasons why a preference for the forward hemisphere might arise from an evolutionary process, concluding that three factors may be at play, with the effects of these being roughly additive: (a) minimising the time spent tumbling (which is not being used to explore new areas), (b) increasing drift speed up a gradient and (c) optimally partitioning the search space to find a compromise between exploration and exploitation.We present a new method of graphically visualising the evolution of a biological trait or function, which we call an evolutionary characteristic.

Fig. 2 .
Fig. 2. Efficiency of chemotactic swimming predicted for WT E. coli, i.e. with a mean of around 63 • between successive runs, relative to the complete reorientation case, as a function of varying directional persistence parameter ϑ tumble .All values are means across the population.

Fig. 3 .
Fig. 3. Evolution of populations placed in different attractant gradients.Top: Evolution of persistence in a population with zero initial persistence.Bottom: Attractant acquired during each generation.All values are means across the population taken at the end of individual generations.

Fig. 4 .
Fig.4.Evolutionary characteristic plot: the scatter pattern of a biological function output (attractant acquired, in this case) against an evolved parameter (persistence, in this case), showing the size of the genotypic space explored (horizontal width of each scatterplot), fitness-genotype sensitivity (vertical height of each scatterplot) and the self-limiting nature of a continuous mutation-selection evolutionary process (size and shape of the cluster at the tail of each scatterplot).