**3.2 Bidding strategy**

266 Bio-Inspired Computational Algorithms and Their Applications

experimental economics studies of the CDA and other auction mechanisms. The aim of each zip agent is to maximize the profit generated by trading in the market. A standard genetic algorithm is then applied to optimize the values of the eight parameters governing the behavior of the ZIP traders which previously must be set manually. The result showed that GA-optimized traders performed better than those populated by ZIP traders with manually set parameter values (Cliff, 1998a; Cliff, 1998b). This work is then extended to 60 parameters to be set correctly. The experiment showed promising result when compared to the ZIP traders with eight parameters (Cliff, 2006). Genetic algorithm is also used to optimize the auction market parameters setting. Many tests have been conducted on ZIP to improve the agent traders and the auction market mechanism using genetic algorithm (Cliff, 2002a; Cliff, 2002b). Thus, ZIP was able to demonstrate that genetic algorithm can perform well in

In another investigation, a UDA (utility-based double auction) mechanism is presented (Choi et, al. 2008). In UDA, a flexible synchronous double auction is implemented where the auctioneer maximizes all traders' diverse and complex utility functions through optimization modeling based on genetic algorithm. It is a double auction mechanism based on dynamic utility function integrating the notion of utility function and genetic algorithm. The GA-optimizer is used to maximize total utility function, composed of all participants' dynamic utility functions, and matches the buyers and sellers. Based on the experimental

As mentioned, this work is an extension of Anthony's work (Anthony, 2003) to tackle the problem of bidding in multiple auctions that employ varying auctions protocols. This section details the electronic marketplace simulation, the bidding strategies and the genetic

The market simulation employed three different auction protocols, English, Vickrey and Dutch that run simultaneously in order to simulate the real auction environment. The market simulation is used in this work to evaluate the performance of the evolved bidding

The marketplace simulator shown in Fig. 2 consists of concurrent running auctions that employ different protocols. These protocols are English, Dutch and Vickrey. All of these auctions have a known starting time and only English and Vickrey auctions have a known ending time. The bidding agent is given a deadline (*tmax*) by when it must obtain the desired item and it is told about the consumer's private valuation (*pr*) for this item. The agent must

The marketplace announces the current bid values and the current highest bids for English auctions and the current offers for Dutch auctions at each time step. At the end of a given auction, it determines the winning bid and announces the winner. This set of information is used by the agent when deciding in which auction to participate, at what value to bid and in

strategies. The following section explains how the market simulation works.

evolving the parameters of bidding agents and the strategies.

result, it performance is better than a conventional double auction.

**3. The bidding strategy framework** 

algorithm implemented in the previous work.

**3.1 The electronic market place simulation** 

only buy an instance of the desired item.

which time to bid.

The bidding algorithm for this framework is shown in Fig. 3. Let *Item\_NA* be a boolean flag to indicate whether the target item has already been purchased by the agent. Assume that the value of *pr* is based on the current reliable market prices observed from past auctions and that the marketplace is offering the item which the agent is interested in. While the bidder agent has not obtained the desired item, the bidder agent needs to build an active auctions list in order to keep track of the current active auction. Active auction is defined as auction that is ongoing or just started but has not reach the ending time yet.

```
while (t ≤ tmax ) and (Item_NA = true) 
 Build active auction list; 
 Calculate current maximum bid using the agent's strategy; 
 Select potential auctions to bid in, from active auction list; 
 Select target auction as one that maximizes agent's expected utility; 
 Bid in the target auction using current maximum bid as reservation price at this time; 
Endwhile
```
Fig. 3. The bidding agent's algorithm

```
for all i є A 
 if ((t ≥ σi ) and (t ≤ ηi ) or (Si (t) = ongoing) 
 add i to L(t) 
 endif 
endfor
```
Fig. 4. Building active auction list algorithms

In order to build the active auction list, the bidder agent follows the algorithm as shown in Fig. 4. *Si* (*t*) is a boolean flag representing the status of auction *i* at time *t*, such that *i є A* and *Si* (*t*) *є (ongoing; completed)*. Each auction *i є A*, has a starting time *σi*, and its own ending time *ηi*. The active auction list is built by taking all the auctions that are currently running at time *t*. In English and Vickrey auctions, any auction that has started but has not reached its ending time is considered as active. *Si* (*t*) is used in Dutch auctions since the ending time of this type of auction is not fixed.

Performance of Varying Genetic Algorithm Techniques in Online Auction 269

The remaining auction left tactic, on the other hand, considers the number of remaining auctions that the bidder agent is able to participate in order to obtain the item. This tactic bids closer to *pr* as the number of the remaining auctions decreases when the bidder agent is running out of opportunities to obtain the desired item. The current bid value is calculated

> *f tp ra ra r* =α

1 | | *ra ra ra c t k k*

= +−

difference between the two function is the *c(t)*. *c(t)* is a list of auctions that have been completed between time 0 and t. The *β* value for this tactic is identical to the remaining time

The desire for a bargain tactic is the bidder agent that is interested in getting a bargain for obtaining the desired item. In this scenario, the bidder agent needs to take into account all the ongoing auctions and the time left to obtain the item. The current bid value is calculated

with the current bid value. The Dutch and English are considered solely in this expression as only these two auctions have current bid value. As a consequence, the minimum bid value is calculated based on the current bid value and also the proportion of the time left in the auction. These values are summed and averaged with respect to the number of active

( ) ( ) ( ) <sup>1</sup> ( )

<sup>−</sup> <sup>=</sup> <sup>−</sup>

*<sup>t</sup> t v <sup>t</sup>*

*i Lt i i*

η σ

*fba* =+ − ωα

ω

1 | |

where *vi* is the current highest bid value in an auction *I* at time *t,* and *I є L(t)*;

*L t* ≤ ≤

() ( )

= +−

1 *ba ba ba <sup>t</sup> tk k*

*ra* is quite similar to the terms use in

( ) ( ) <sup>1</sup>

*A* 

> ω

ω

*i i*

1

β

max

*t* 

σ

β

( ) (3)

α

() () () *t t ba r* ( ) *p t* (5)

( )*t* takes into account all the ongoing auctions along

( )*t* is calculated based on the formula

(6)

(7)

(4)

*rt* , whereby the only

based on the following expression:

The polynomial function

tactic between 0.005 *≤ β ≤* 1000.

based on the following expression:

In the expression above, the variable

auctions at that particular time. The expression for

*σi*, and *ηi* is the start and end time of auction *i* 

α

ω

*ba* ( )*t* is defined as:

α

*ra* ( )*t* is a polynomial function of form:

α

α

where α

as below:

The expression for

After the bidder agent builds the active auctions list, the bidder agent will start calculating the current maximum bid based on the agent strategy. The current maximum bid is defined as the amount of the agent willing to bid at the current time that is lesser than or equal to the agent's private valuation. Four bidding constraints are used to determine the current maximum bid namely the remaining time left, the remaining auction left, the desire for bargain and the level of desperateness.

The remaining time tactic considers the amount of bidding time the bidder agent has to obtain the desire item. This tactic determines the bid value based on the bidding time left. Assuming that the bidding time t is between 0 and *tmax* (0 ≤ t ≤ *tmax*), the current bid value is calculated based on the following expression:

$$f\_{rt} = \alpha\_{rt}(t)P\_r \tag{1}$$

where α*rt* ( )*t* is a polynomial function of the form:

$$\alpha\_{rt}\left(t\right) = k\_{rt} + \left(1 - k\_{rt}\right) \left(\frac{t}{t\_{\text{max}}}\right)^{\frac{1}{\beta}} \tag{2}$$

This function is a time dependent polynomial function where the main consideration is the time left from the maximum time allocated. *krt* is a constant that determines the value of the starting bid of the agent in any auction multiplied by the size of the interval. This time dependent functions can be defined as those that start bidding near *pr* rapidly to those only bid near *pr* right at the end along with all the possibilities in between with variation of the valueα*rt* ( )*t* . Different shapes of curve can be obtained by varying the values of *β* by using the equation defined above. There are unlimited numbers of possible tactics for each value of *β.* In this tactic, *β* value is defined between 0.005 *≤ β ≤* 1000. It is possible to have two different behaviors for *β*. When *β <* 1, the tactic will bid with a low value until the deadline is almost reached, whereby this tactic concedes by suggesting the private valuation as the recommended bid value. When *β >* 1, the tactic starts with a bid value close to the private valuation and quickly reaches the private valuation long before the deadline is reached. Fig. 5 shows the different shape of the curves with varying *β* values.

Fig. 5. The curve with varying *β* value. (Anthony, 2003)

The remaining auction left tactic, on the other hand, considers the number of remaining auctions that the bidder agent is able to participate in order to obtain the item. This tactic bids closer to *pr* as the number of the remaining auctions decreases when the bidder agent is running out of opportunities to obtain the desired item. The current bid value is calculated based on the following expression:

$$f\_m = \alpha\_m(t)p\_r \tag{3}$$

where α*ra* ( )*t* is a polynomial function of form:

268 Bio-Inspired Computational Algorithms and Their Applications

After the bidder agent builds the active auctions list, the bidder agent will start calculating the current maximum bid based on the agent strategy. The current maximum bid is defined as the amount of the agent willing to bid at the current time that is lesser than or equal to the agent's private valuation. Four bidding constraints are used to determine the current maximum bid namely the remaining time left, the remaining auction left, the desire for

The remaining time tactic considers the amount of bidding time the bidder agent has to obtain the desire item. This tactic determines the bid value based on the bidding time left. Assuming that the bidding time t is between 0 and *tmax* (0 ≤ t ≤ *tmax*), the current bid value is

> *f tP rt rt r* =α

() ( )

= +−

*rt* ( )*t* . Different shapes of curve can be obtained by varying the values of *β* by using the equation defined above. There are unlimited numbers of possible tactics for each value of *β.* In this tactic, *β* value is defined between 0.005 *≤ β ≤* 1000. It is possible to have two different behaviors for *β*. When *β <* 1, the tactic will bid with a low value until the deadline is almost reached, whereby this tactic concedes by suggesting the private valuation as the recommended bid value. When *β >* 1, the tactic starts with a bid value close to the private valuation and quickly reaches the private valuation long before the deadline is reached. Fig.

This function is a time dependent polynomial function where the main consideration is the time left from the maximum time allocated. *krt* is a constant that determines the value of the starting bid of the agent in any auction multiplied by the size of the interval. This time dependent functions can be defined as those that start bidding near *pr* rapidly to those only bid near *pr* right at the end along with all the possibilities in between with variation of the

1 *rt rt rt <sup>t</sup> tk k*

( ) (1)

(2)

1

β

max

*t* 

bargain and the level of desperateness.

where α

valueα

calculated based on the following expression:

*rt* ( )*t* is a polynomial function of the form:

α

5 shows the different shape of the curves with varying *β* values.

Fig. 5. The curve with varying *β* value. (Anthony, 2003)

$$\alpha\_{n} = k\_{n} + \left(1 - k\_{n}\right) \left(\frac{c\left(t\right)}{\left|\left|A\right|\right)}\right)^{\frac{1}{\beta}}\tag{4}$$

The polynomial function α*ra* is quite similar to the terms use inα*rt* , whereby the only difference between the two function is the *c(t)*. *c(t)* is a list of auctions that have been completed between time 0 and t. The *β* value for this tactic is identical to the remaining time tactic between 0.005 *≤ β ≤* 1000.

The desire for a bargain tactic is the bidder agent that is interested in getting a bargain for obtaining the desired item. In this scenario, the bidder agent needs to take into account all the ongoing auctions and the time left to obtain the item. The current bid value is calculated based on the following expression:

$$f\_{hu} = o(t) + \alpha\_{hu}(t)(p\_r - o(t))\tag{5}$$

In the expression above, the variableω( )*t* takes into account all the ongoing auctions along with the current bid value. The Dutch and English are considered solely in this expression as only these two auctions have current bid value. As a consequence, the minimum bid value is calculated based on the current bid value and also the proportion of the time left in the auction. These values are summed and averaged with respect to the number of active auctions at that particular time. The expression for ω( )*t* is calculated based on the formula as below:

$$\rho o(t) = \frac{1}{|L(t)|} \left( \sum\_{1 \le i \le L(t)} \frac{t - \sigma\_i}{\eta\_i - \sigma\_i} v\_i(t) \right) \tag{6}$$

where *vi* is the current highest bid value in an auction *I* at time *t,* and *I є L(t)*;

*σi*, and *ηi* is the start and end time of auction *i* 

The expression for α*ba* ( )*t* is defined as:

$$\alpha\_{ha}\left(t\right) = k\_{ha} + \left(1 - k\_{ha}\right) \left(\frac{t}{t\_{\text{max}}}\right)^{\frac{1}{\beta}}\tag{7}$$

Performance of Varying Genetic Algorithm Techniques in Online Auction 271

*tmax* Deadline given to the agent to obtain the desired item

*wrt* Relative weight for the remaining time tactic *wra* Relative weight for the remaining auction tactic *wba* Relative weight for the desire for a bargain tactic *wde* Relative weight for the desperateness tactic

Fitness function is an objective function that quantifies the optimality of a solution in a genetic algorithm so that the particular chromosome may be ranked against all the other chromosomes. The main focus of the strategies evaluation in this work is the success rate and average utility of the strategies. Three fitness equations are used to evaluate the performance of the strategies namely the success rate, the agent's utility function and agent's utility with penalty. The success rate is the rate in obtaining the desired item and the

( ) *<sup>r</sup>*

*r p v Uv c p* <sup>−</sup> = +

where *v* represents the winning bid and *c* is an arbitrary constant 0.001 to ensure that the agent receives some value when the winning bid is equivalent to its private valuation. The third fitness equation involves a variation of the agent utility. If the agent fails to get the item, a penalty that ranges from 0.01 to 0.05 is incurred. Basically, Fitness Equation 1 is used if the delivery of the item is of utmost importance to the user. Fitness Equation 2 is used when the agent is looking for a bargain. Fitness Equation 3 is used when both the delivery of the item and looking for a bargain are equally important. The fitness score is then computed

*Elitism* is an operator used to retain some number of the best individuals in each generation to the next generation in order to ensure that the fittest individual is not lost during the evolution process (Obitko, 1998). Elitism is applied in this work to retain ten percent of the

(8)

*i*

*pr* Agent's private valuation

*krt k* for the remaining time tactic *βrt β* for the remaining time tactic *kra k* for the remaining auction tactic *βra β* for the remaining auction tactic *kba k* for the desire for a bargain tactic *βba β* for the desire for a bargain tactic *kde k* for the desperateness tactic *βde β* for the desperateness tactic

*fitness* Fitness score for the individual

Table 1. Bidding strategies representation

second fitness function is the agent's utility

by taking the average utility from a total of 2000 runs.

**3.3.3 Selection operators** 

**3.3.2 Representation** 

The valid range for the constant *kba* is 0.1 ≤ *kba* ≤ 0.3 and the *β* value is 0.005 ≤ *β* ≤ 0.5. The *β* value is lower than 1 as bidder agent that is looking for bargain will never bid with the behavior of *β* >1. The *β* value is, therefore, constantly lower than 1 in order to maintain a low bid until the closes to the end time. Hence, the value of *β* < 0.5 is used.

The level of desperateness tactic is the bidder agent's desperateness to obtain the target item within a given period and thus, the bidder agent who possesses this behavior tend to bid aggressively. This tactic utilizes the same minimum bid value and the polynomial function as the desire for bargain tactic but with a minor variation to the *β* and *kde* value. The valid range for the constant *kde* for this tactic is 0.7 ≤ *kde* ≤ 0.9 while the *β* value is 1.67 ≤ *β* ≤ 1000. The *β* value is higher than 1 in this case as the bidder agent that is looking for bargain will never bid with the behavior of *β* <1. As a result, the *β* value is always higher than 0.7 since the bidder agent will bid close to the private valuation.

There is a weight associated to each of this tactic and this weight is to emphasize which combination of tactics that will be used to bid in the online auction. The final current maximum bid is based on the combination of the four tactics by making use of the weight. Fig. 6 shows various combinations of the bidding constraints based on the different weight associated to the bidding tactics. It can also be seen that different bidding patterns are generated by varying the value of weights of the bidding constraints.

Fig. 6. Various combinations of the bidding constraints

#### **3.3 Genetic algorithm**

#### **3.3.1 Representation**

Floating point encoding is applied in this particular work as floating point encoding has shown to produce faster, more consistent and more accurate results (Janikow & Michalewicz, 1991). The floating encoding is, therefore, represented using an array of structure. The individuals that are represented in a floating point array structure are shown in Table 1.


Table 1. Bidding strategies representation

#### **3.3.2 Representation**

270 Bio-Inspired Computational Algorithms and Their Applications

The valid range for the constant *kba* is 0.1 ≤ *kba* ≤ 0.3 and the *β* value is 0.005 ≤ *β* ≤ 0.5. The *β* value is lower than 1 as bidder agent that is looking for bargain will never bid with the behavior of *β* >1. The *β* value is, therefore, constantly lower than 1 in order to maintain a low

The level of desperateness tactic is the bidder agent's desperateness to obtain the target item within a given period and thus, the bidder agent who possesses this behavior tend to bid aggressively. This tactic utilizes the same minimum bid value and the polynomial function as the desire for bargain tactic but with a minor variation to the *β* and *kde* value. The valid range for the constant *kde* for this tactic is 0.7 ≤ *kde* ≤ 0.9 while the *β* value is 1.67 ≤ *β* ≤ 1000. The *β* value is higher than 1 in this case as the bidder agent that is looking for bargain will never bid with the behavior of *β* <1. As a result, the *β* value is always higher than 0.7 since

There is a weight associated to each of this tactic and this weight is to emphasize which combination of tactics that will be used to bid in the online auction. The final current maximum bid is based on the combination of the four tactics by making use of the weight. Fig. 6 shows various combinations of the bidding constraints based on the different weight associated to the bidding tactics. It can also be seen that different bidding patterns are

Floating point encoding is applied in this particular work as floating point encoding has shown to produce faster, more consistent and more accurate results (Janikow & Michalewicz, 1991). The floating encoding is, therefore, represented using an array of structure. The individuals that are represented in a floating point array structure are shown

bid until the closes to the end time. Hence, the value of *β* < 0.5 is used.

generated by varying the value of weights of the bidding constraints.

the bidder agent will bid close to the private valuation.

Fig. 6. Various combinations of the bidding constraints

**3.3 Genetic algorithm 3.3.1 Representation** 

in Table 1.

Fitness function is an objective function that quantifies the optimality of a solution in a genetic algorithm so that the particular chromosome may be ranked against all the other chromosomes. The main focus of the strategies evaluation in this work is the success rate and average utility of the strategies. Three fitness equations are used to evaluate the performance of the strategies namely the success rate, the agent's utility function and agent's utility with penalty. The success rate is the rate in obtaining the desired item and the second fitness function is the agent's utility

$$\mathcal{U}I\_i\left(\upsilon\right) = \left(\frac{p\_r - \upsilon}{p\_r}\right) + c \tag{8}$$

where *v* represents the winning bid and *c* is an arbitrary constant 0.001 to ensure that the agent receives some value when the winning bid is equivalent to its private valuation. The third fitness equation involves a variation of the agent utility. If the agent fails to get the item, a penalty that ranges from 0.01 to 0.05 is incurred. Basically, Fitness Equation 1 is used if the delivery of the item is of utmost importance to the user. Fitness Equation 2 is used when the agent is looking for a bargain. Fitness Equation 3 is used when both the delivery of the item and looking for a bargain are equally important. The fitness score is then computed by taking the average utility from a total of 2000 runs.

#### **3.3.3 Selection operators**

*Elitism* is an operator used to retain some number of the best individuals in each generation to the next generation in order to ensure that the fittest individual is not lost during the evolution process (Obitko, 1998). Elitism is applied in this work to retain ten percent of the

Performance of Varying Genetic Algorithm Techniques in Online Auction 273

numerical test problems. The evolution of the bidding strategies by Anthony and Jennings (Anthony & Jennings, 2002) employed a fixed crossover and mutation probability based on the literatures. However, these recommended values may not perform at its best in the genetic algorithm as it has been proven that the parameter values are dependent on the nature of problems to be solved (Engelbrecht, 2002). In this experiment, the crossover and mutation rates are fine tuned with different combination of probabilities in order to discover the best combination of genetic operators' probabilities. Thus, the main objective of this experiment is to improve the effectiveness of the bidding strategies by "hand tuning" the values of the crossover rate and mutation rate to allow a new combination of static crossover and mutation rates to be discovered. By improving the algorithm, more effective bidding

The experiment is subdivided to two parts. The first one varies the crossover rate and the second one varies the mutation rate. At the end of this experiment, the combination rate discovered is compared and empirically evaluated with the bidding strategies evolved in

Table 2 and 3 show the evolutionary and parameter setting for the genetic algorithm. The parameters setting in the simulated environment for the empirical evaluations are shown in Table 4. These parameters include the agent's reservation price; the agent's bidding time and the number of active auctions. The agent's reservation price is the maximum amount that the agent is willing to pay for the item while the bidding time is the time allocated for the agent to obtain the user's required item. The active auctions are the list of auctions that is ongoing before time tmax. Fig. 7 shows the pseudocode of the genetic

Representation Real Values Number

Selection Tournament Selection

Mutation Creep Operator

Crossover Probability 0.2, 0.4, 0.6, 0.8 Mutation Probability 0.2, 0.02, 0.002

Termination Criteria After 50 Generation

Table 2. Genetic algorithm evolutionary setting

Table 3. Genetic algorithm parameter setting

Number of Generations 50 Number of Individuals 50 Elitism 10%

Number of Run 10

Crossover Extension Combination Operator

strategies can be found during the exploration of the solution.

Anthony's work (Anthony, 2003).

**4.1 Experimental setup** 

algorithm.

best individuals to the new population and to ensure that a significant number of the fitter individuals will make it to the next generation. Tournament selection is applied in the genetic algorithm for selecting the individuals to the mating pools for the remaining ninety percent of the population (Blickle & Thiele, 2001). Tournament selection technique was chosen because it is known to perform well in allowing a diverse range of fitter individuals to populate the mating pool (Blickle & Thiele, 1995). By implementing the tournament selection, fitter individuals can contribute to the next generation genetic construction and the best individual will not dominate in the reproduction process compared to the proportional selection.

### **3.3.4 Crossover process**

The extension operator floating point crossover operator is used this work (Beasley *et al.* 1993b). This operator works by taking the differences between the two values, adding it to the higher value (giving the maximum range), and subtracting it from the lower value (giving the minimum range). The new values for the genes are then generated between the minimum and the maximum range that were derived using this operator (Anthony & Jennings, 2002).

### **3.3.5 Mutation process**

Since the encoding is a floating point, the mutation operator used in this work must be a non-binary mutation operator. Beasley has suggested a few non-binary mutation operators such as random replacement, creep operator and geometric creep (Beasley *et al.* 1993b) that can be used. The *creep* operator which adds or subtracts a small randomly generated amount from selected gene is used to allow a small constant of 0.05 to be added or subtracted from the selected gene depending on the range limitation of the parameter (Anthony & Jennings, 2002).

### **3.3.6 Stopping criteria**

The genetic algorithm will repeat the process until the termination criteria are met. In this work, the evolution stops after 50 iterations. An extensive experiment was conducted to determine the point at which the population converges. It was decided to choose 50 as the stopping criterion since it is was observed that the population will always converge before or at the end of the 50 iterations.

Anthony's work has some shortcoming where the crossover and mutation rate used in the work is based on literature review recommended values. However, researches have shown that the crossover rate and mutation rate applied in the application are application dependent, thus, simulation need to be conducted in order to find the suitable crossover and mutation rate. Besides that, other variations of genetic algorithm have proven to perform better that traditional genetic algorithm which is worthwhile to be investigated.
