**Mediated Heuristic Approaches and Alternative Social Welfare Definitions for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces**

Ivan Marsa-Maestre1, Miguel A. Lopez-Carmona1, Enrique de la Hoz1 and Mark Klein2 *1Universidad de Alcala 2Massachusetts Institute of Technology 1Spain 2USA* 

#### **1. Introduction**

176 Social Welfare

Round, David K., Ron P. McIver. 2006. "Teaching Third-Degree Price Discrimination."

Schmalensee, Richard. 1981. "Output and Welfare Implications of Monopolistic Third-Degree Price discrimination." *American Economic Review*, 71(1): 242-247. Schwartz, Marius. 1990. "Third-Degree Price Discrimination and Output: Generalizing a

Shih, Jun-ji, Chao-cheng Mai, and Jung-chao Liu. 1988. "A General Analysis of the Output

Stole, Lars. 2007. "Price Discrimination and Imperfect Competition" in M. Armstrong and R.

Varian, Hal R. 1985. "Price Discrimination and Social Welfare." *American Economic Review*,

Varian, Hal R. 1989. "Price Discrimination,"in R. Schmalensee and R. Willig (eds) *Handbook* 

Varian, Hal R. 1992. *Microeconomic Analysis*. 3th edition. New York: W.W. Norton and

Varian, Hal R. 2006. *Intermediate Microeconomics: A Modern Approach*. 7th edition. New York:

Weber, Sylbain, and Cyril Pasche. 2008. "Price Discrimination." *Journal of Industrial* 

Weyl, E. Glen and Michal Fabinger. 2011. "A Restatement of the Theory of Monopoly."

*of Industrial Organization*: Volume I, North-Holland, Amsterdam.

Effect under Third-Degree Price Discrimination." *Economic Journal*, 98(389): 149--58.

Porter, eds., *Handbook of Industrial Organization* Volume 3. New York: North-

Robinson, Joan. 1933. *The Economics of Imperfect Competition*, London: Macmillan.

Welfare Result." *American Economic Review*, 80(5): 1259-1262.

*Journal of Economic Education*, 236-243.

Holland.

75(4): 870-875.

Company.

W.W. Norton and Company.

*Organization Education*, Vol. 2 : Iss. 1, Article 3.

Available at SSRN: http://ssrn.com/abstract=1851326.

Automated negotiation provides an important mechanism to reach agreements among distributed decision makers (Beer et al., 1999; Kraus et al., 1998; Lai et al., 2004; Rosenschein & Zlotkin, 1994). It has been extensively studied from the perspective of e-commerce (Guttman et al., 1998; He et al., 2003; Lopez-Carmona et al., 2006; Sierra, 2004), though it can be seen from a more general perspective as a paradigm to solve coordination and cooperation problems in complex systems (Jennings, 2001; Klein et al., 2002), providing a mechanism for autonomous agents to reach agreements on, e.g., task allocation, resource sharing, or surplus division (Fatima et al., 2004; Kersten & Noronha, 1998; Zhang et al., 2005).

Most research in multiparty automated negotiation has been focused on building efficient mechanisms and protocols to reach agreements among multiple participants, being an objective to optimize some type of social welfare measurement (Hindriks et al., 2009). Examples of such measurements would be the *sum or product of utilities*, the *min* utility, etc... However, social welfare has not been usually placed itself as an integral part of the negotiation process.

There are remarkable works which incorporate a social welfare criterion within the search process (Ehtamo et al., 1999; Heiskanen et al., 2001; Li et al., 2009a). In these works, the authors build mechanisms to obtain fair agreements by using fair direction improvements in the joint exploration of the negotiation space. These proposals present however several limitations. Firstly, they work only when utility functions are derivable and quasi-concave. Secondly, the protocols are usually prone to untruthful revelation to bias the direction generated by the mediator. Finally, we argue that, in some scenarios, the classic notions of social welfare do not adequately represent the social goal of the negotiation, and that the type of consensus by which an agreement meets in some specific manner the concerns of all

to decouple agents' individual objectives (normally, to maximize their profit) from the social

<sup>179</sup> Mediated Heuristic Approaches and Alternative Social Welfare Definitions

In this chapter, we will discuss the application of nonlinear optimization techniques to mediated negotiations involving highly uncorrelated utility spaces, and the effect of the

Multi-attribute negotiation may be seen as an interaction between two or more agents with the goal of reaching an agreement about a range of issues which usually involves solving a conflict of interests between the agents. This kind of interaction has been widely studied in different research areas, such as game theory (Nash, 1950; Rosenschein & Zlotkin, 1994), distributed artificial intelligence (Ehtamo et al., 1999; Faratin et al., 1998; 2002; Gatti & Amigoni, 2005;

Though there has been an increasing interest in complex negotiations in the last years, little efforts have been made to study complexity itself within negotiation (apart from computational complexity, which has been thoroughly studied in many scenarios). Therefore, if we want to be able to assess complexity in negotiations, we need to resort to other knowledge areas. An area where many authors have dealt with complexity characterization and measurement is optimization. In fact, negotiation scenarios and optimization problems are often closely related, since there are many similarities in the ways both problem families are defined and addressed (Sawaragi et al., 1985; Shi et al., 2006; Wierzbicki et al., 1993). Therefore, some of the concepts studied in multiobjective optimization may be used in

In the context of a multi-attribute negotiation, complexity may depend at least on the number of issues, the level of interdependency between the preferences on the issues, the size of the issue domain, and the possibility of change over time of the negotiation context. The method to describe preferences and the structural properties of the agents' utility spaces also play a fundamental role in the complexity of the negotiation scenario. This is specially true when optimization techniques are used to find high utility regions within an agent's utility space, or to find deals among different agents. A constraint-based preference space, for instance, may present discontinuities which make gradient based optimizers not applicable, while

Therefore, to study complexity in negotiation scenarios, we may find useful to characterize structural complexity of the agents' utility spaces, and to this end we may benefit from existing research on function characterization for optimization. In particular, some of the results in the analysis of *fitness landscapes* in evolutionary computation are specially useful, since they identify a set of landscape structural properties, such as modality (Horn & Goldberg, 1994), ruggedness, smoothness and neutrality (Vassilev et al., 2003), which are also interesting regarding search complexity within the solution space in a negotiation scenario. Furthermore, there are some techniques which allow to measure the complexity of a given space. Most of the approaches we can find in the literature are based on the correlation between different samples of the fitness function, like *fitness distance correlation* metrics (Tomassini et al., 2005), or stochastic models representing the correlation structure of the space (Hordijk, 1995). A metric which is easy to compute in most scenarios and allows to make quantitative evaluations about the complexity of a fitness or utility landscape is *correlation length* or *correlation distance*.

Ito et al., 2008; Klein et al., 2003; Lai et al., 2006) and economics (Raiffa, 1982).

differentiable utility functions contribute to a faster local optimization.

objective of maximizing social welfare.

negotiation.

different notions of social welfare in this kind of negotiations.

for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces

**1.1 Multi-attribute negotiation in complex utility spaces**

the negotiators should be considered as an integral part within the multiparty negotiation protocols.

In addition, in the last years there has been an increasing interest in complex negotiations (Klein et al., 2003; Lopez-Carmona et al., 2011). Complexity of a negotiation scenario may depend on several factors, like the cardinality of the solution space, the number of negotiating agents, the number of issues under negotiation, the degree of interdependency between the issues, and structural properties of the different agent preference landscapes, like ruggedness, modality or correlation length (Vassilev et al., 2003). Specially challenging are those scenarios involving highly rugged or highly auto-uncorrelated utility spaces, since traditional negotiation approaches (mostly intended for linear or quasi-concave utility functions) cannot be applied effectively to these scenarios.

We can find some successful research works in the literature addressing negotiation in complex utility spaces (Fatima et al., 2009; Hindriks et al., 2006; Klein et al., 2003). However, most of these approaches rely on monotonic utility spaces, binary valued issues, low-order dependencies, a fixed set of defined *a priori* solutions or simplification of the negotiation space, and there are very few works addressing auto-uncorrelated utility spaces with high-order issue interdependencies. One of the most relevant approaches in this field is (Ito et al., 2007). They propose a bidding mechanism, which is based on taking random samples of the contract space and applying simulated annealing to these samples to identify high utility regions for each agent, sending these regions as bids to a mediator, and then performing a search in the mediator to find overlaps between the bids of the different agents. Experiments show that this approach achieves high effectiveness (measured as high optimality rates and low failure rates for the negotiations) in the evaluation scenario they describe. However, this approach performs worse as the circumstances of the scenario turn harder (that is, when the utility functions present low autocorrelation). Furthermore, the bidding-based negotiation protocol presents some scalability concerns when the number of agents increases. Under these circumstances, the failure rate increases drastically, raising the need for an alternative approach.

Therefore, new approaches are needed if automated negotiation is to be applied to settings involving highly uncorrelated preference spaces. Here the use of heuristic optimization techniques to assist negotiation processes has been revealed as a promising approach. In fact, negotiation scenarios and optimization problems are often closely related, since there are many similarities in the ways both problem families are defined and addressed. For example, negotiating agents are usually utility optimizers, and negotiation mechanisms are often evaluated in terms of their ability to reach Pareto-optimal solutions. Multi-objective optimization has been widely used for negotiation support (Wierzbicki et al., 1993), and negotiation mechanisms have also been used to solve multiobjective optimization problems, usually by distributing the different objectives among negotiating agents (Shi et al., 2006). Therefore, some of the techniques studied in multiobjective optimization may be used in negotiation, and vice versa.

Apart from using optimization techniques, there is a trend in automated complex negotiation to use mediated protocols. The existence of a mediator in the negotiation allows agents to provide more information to facilitate a successful outcome for the negotiation without revealing too much information to their counterparts. Furthermore, mediated protocols allow to decouple agents' individual objectives (normally, to maximize their profit) from the social objective of maximizing social welfare.

In this chapter, we will discuss the application of nonlinear optimization techniques to mediated negotiations involving highly uncorrelated utility spaces, and the effect of the different notions of social welfare in this kind of negotiations.

#### **1.1 Multi-attribute negotiation in complex utility spaces**

2 Will-be-set-by-IN-TECH

the negotiators should be considered as an integral part within the multiparty negotiation

In addition, in the last years there has been an increasing interest in complex negotiations (Klein et al., 2003; Lopez-Carmona et al., 2011). Complexity of a negotiation scenario may depend on several factors, like the cardinality of the solution space, the number of negotiating agents, the number of issues under negotiation, the degree of interdependency between the issues, and structural properties of the different agent preference landscapes, like ruggedness, modality or correlation length (Vassilev et al., 2003). Specially challenging are those scenarios involving highly rugged or highly auto-uncorrelated utility spaces, since traditional negotiation approaches (mostly intended for linear or quasi-concave utility

We can find some successful research works in the literature addressing negotiation in complex utility spaces (Fatima et al., 2009; Hindriks et al., 2006; Klein et al., 2003). However, most of these approaches rely on monotonic utility spaces, binary valued issues, low-order dependencies, a fixed set of defined *a priori* solutions or simplification of the negotiation space, and there are very few works addressing auto-uncorrelated utility spaces with high-order issue interdependencies. One of the most relevant approaches in this field is (Ito et al., 2007). They propose a bidding mechanism, which is based on taking random samples of the contract space and applying simulated annealing to these samples to identify high utility regions for each agent, sending these regions as bids to a mediator, and then performing a search in the mediator to find overlaps between the bids of the different agents. Experiments show that this approach achieves high effectiveness (measured as high optimality rates and low failure rates for the negotiations) in the evaluation scenario they describe. However, this approach performs worse as the circumstances of the scenario turn harder (that is, when the utility functions present low autocorrelation). Furthermore, the bidding-based negotiation protocol presents some scalability concerns when the number of agents increases. Under these circumstances, the failure rate increases drastically, raising the need for an alternative

Therefore, new approaches are needed if automated negotiation is to be applied to settings involving highly uncorrelated preference spaces. Here the use of heuristic optimization techniques to assist negotiation processes has been revealed as a promising approach. In fact, negotiation scenarios and optimization problems are often closely related, since there are many similarities in the ways both problem families are defined and addressed. For example, negotiating agents are usually utility optimizers, and negotiation mechanisms are often evaluated in terms of their ability to reach Pareto-optimal solutions. Multi-objective optimization has been widely used for negotiation support (Wierzbicki et al., 1993), and negotiation mechanisms have also been used to solve multiobjective optimization problems, usually by distributing the different objectives among negotiating agents (Shi et al., 2006). Therefore, some of the techniques studied in multiobjective optimization may be used in

Apart from using optimization techniques, there is a trend in automated complex negotiation to use mediated protocols. The existence of a mediator in the negotiation allows agents to provide more information to facilitate a successful outcome for the negotiation without revealing too much information to their counterparts. Furthermore, mediated protocols allow

functions) cannot be applied effectively to these scenarios.

protocols.

approach.

negotiation, and vice versa.

Multi-attribute negotiation may be seen as an interaction between two or more agents with the goal of reaching an agreement about a range of issues which usually involves solving a conflict of interests between the agents. This kind of interaction has been widely studied in different research areas, such as game theory (Nash, 1950; Rosenschein & Zlotkin, 1994), distributed artificial intelligence (Ehtamo et al., 1999; Faratin et al., 1998; 2002; Gatti & Amigoni, 2005; Ito et al., 2008; Klein et al., 2003; Lai et al., 2006) and economics (Raiffa, 1982).

Though there has been an increasing interest in complex negotiations in the last years, little efforts have been made to study complexity itself within negotiation (apart from computational complexity, which has been thoroughly studied in many scenarios). Therefore, if we want to be able to assess complexity in negotiations, we need to resort to other knowledge areas. An area where many authors have dealt with complexity characterization and measurement is optimization. In fact, negotiation scenarios and optimization problems are often closely related, since there are many similarities in the ways both problem families are defined and addressed (Sawaragi et al., 1985; Shi et al., 2006; Wierzbicki et al., 1993). Therefore, some of the concepts studied in multiobjective optimization may be used in negotiation.

In the context of a multi-attribute negotiation, complexity may depend at least on the number of issues, the level of interdependency between the preferences on the issues, the size of the issue domain, and the possibility of change over time of the negotiation context. The method to describe preferences and the structural properties of the agents' utility spaces also play a fundamental role in the complexity of the negotiation scenario. This is specially true when optimization techniques are used to find high utility regions within an agent's utility space, or to find deals among different agents. A constraint-based preference space, for instance, may present discontinuities which make gradient based optimizers not applicable, while differentiable utility functions contribute to a faster local optimization.

Therefore, to study complexity in negotiation scenarios, we may find useful to characterize structural complexity of the agents' utility spaces, and to this end we may benefit from existing research on function characterization for optimization. In particular, some of the results in the analysis of *fitness landscapes* in evolutionary computation are specially useful, since they identify a set of landscape structural properties, such as modality (Horn & Goldberg, 1994), ruggedness, smoothness and neutrality (Vassilev et al., 2003), which are also interesting regarding search complexity within the solution space in a negotiation scenario. Furthermore, there are some techniques which allow to measure the complexity of a given space. Most of the approaches we can find in the literature are based on the correlation between different samples of the fitness function, like *fitness distance correlation* metrics (Tomassini et al., 2005), or stochastic models representing the correlation structure of the space (Hordijk, 1995). A metric which is easy to compute in most scenarios and allows to make quantitative evaluations about the complexity of a fitness or utility landscape is *correlation length* or *correlation distance*.

of the utility space where the actual outcome is expected to be (i.e. it is assumed that the

<sup>181</sup> Mediated Heuristic Approaches and Alternative Social Welfare Definitions

for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces

In (Robu et al., 2005) utility graphs are used to model issue interdependencies for binary-valued issues. Utility graphs are inspired by graph theory and probabilistic influence networks to derive efficient heuristics for non-mediated bilateral negotiations about multiple issues. The idea is to decompose highly non-linear utility functions in sub-utilities of clusters of inter-related items. They show how utility graphs can be used to model an opponent's preferences. In this approach agents need prior information about the maximal structure of the utility space to be explored. Authors argue that this prior information could be obtained through a history of past negotiations or the input of domain experts. However, our approach has the advantage that outcomes can be reached without any prior information and that it is

There are several proposals which employ genetic algorithms to learn opponent's preferences according to the history of the counter-offers based upon stochastic approximation. In (Choi et al., 2001) a system based on genetic-algorithms for electronic business is proposed. In this work the utility functions are restricted to take a product combination form (i.e. utility of an outcome is the product of the utility values of the different issues). The objective function used is based on the comparison of the changes of consecutive offers. Small changes of an issue suggest that this issue is more important. For each new population, the protocol enforces that the generated candidates cannot be better than the previous offer. Unlike other negotiation models based on genetic algorithms, this proposal adapts to the environment by dynamically modifying its mutation rate. (Lau et al., 2004) have also reported a negotiation mechanism for non-mediated automated negotiations based on genetic algorithms. The fitness function relies on three aspects: an agent's own preference, the distance of a candidate offer from the previous opponent's offer, and time pressure. In this work agents' preferences are quantified by a linear aggregation of the issue valuations. However, rugged and discontinuous preference spaces are not explored. In (Chou et al., 2007) a genetic algorithm is proposed which is based on a joint elitism operation and a joint fitness operation. In the joint elitism operation an agent stores the latest offers received from the opponent. The joint fitness operation combines agent's own utility function and euclidean distance to the opponent's offer. In this work two different negotiation scenarios are considered. In the first one utility is defined as the weighted sum of the different issue values (i.e. issues are independent). The second scenario defines a utility function where there is a master issue and a set of slave issues. Utility is calculated as the weighted sum of the different issue values, but the weights of the

slave and master issues change according to the value of the master issue.

In (Yager, 2007) a mediated negotiation framework for multi-agent negotiation is presented. This framework involves a mediation step in which the individual preference functions are aggregated to obtain a group preference function. The main interest is focused on the implementation of the mediation rule where they allow a linguistic description using fuzzy logic. A notable feature of their approach is the inclusion of a mechanism rewarding the agents for being open to alternatives other than simply their most preferred. The negotiation space and utility values are assumed to be arbitrary (i.e. preferences can be uncorrelated). However, the set of possible solutions is defined *a priori* and is fixed. Moreover, the preference function needs to be provided to the mediation step in the negotiation process, and pareto-optimality is not considered. Instead, a stopping rule is considered, which determines when the rounds

region is known *a priori* by the agents).

not restricted to binary-valued issues.

of mediation stop.

Correlation distance is defined as the minimum distance *ψ* which makes correlation fall below a given threshold (usually 0.5). This measure gives an idea of the distance we can move throughout the solution space while keeping a certain correlation between samples (Manderick et al., 1991). Throughout this work, we will use the terms "correlated" or "uncorrelated" to refer to utility spaces in terms of their correlation lengths.

(Klein et al., 2003) present the first negotiation protocols specific for complex preference spaces. They propose a simulated annealing-based approach, a refined version based on a parity-maintaining annealing mediator, and an unmediated version of the negotiation protocol. Of great interest in this work are the positive results about the use of simulated annealing as a way to regulate agent decision making, along with the use of an expressive bid language to allow the mediator to improve its proposals. However, this expressiveness is somewhat limited, with only four possible valuations which allow the mediator to decide which contract to use as a parent for mutation, but not in which direction to mutate it. On the other hand, the performed experiments only consider the bilateral negotiation scenario, though authors claim that the multiparty generalization is simple. Finally, the family of negotiation protocols they propose are specific for binary issues and binary dependencies. Higher-order dependencies and continuous-valued issues, common in many real-world contexts, are known to generate more challenging utility landscapes which are not considered in their work.

(Luo et al., 2003) propose a fuzzy constraint based framework for multi-attribute negotiations. In this framework a buyer agent defines a set of fuzzy constraints to describe its preferences. The proposals of the buyer agent are a set of hard constraints which are extracted from the set of fuzzy constraints. The seller agent responds with an offer or with a relaxation request. The buyer then decides whether to accept or reject an offer, or to relax some constraints by priority from the lowest to highest. In (Lopez-Carmona et al., 2006; 2007) an improvement to Luo's model is presented. They devise an expressive negotiation protocol where proposals include a valuation of the different constraints, and seller's responses may contain explicit relaxation requests. It means that a seller agent may suggest the specific relaxation of one or more constraints. The relaxation suggested by a seller agent is based on utility and viability criteria, which improves the negotiation process. Though these constraint-based works model discontinuous preference spaces, the operators used to compute utility and the utility spaces defined yield monotonic preference spaces, which are far from the complex preference spaces covered in our work.

Another interesting approach to solve the computational cost and complexity of negotiating interdependent issues is to simplify the negotiation space. (Hindriks et al., 2006) propose a weighted approximation technique to simplify the utility space. They show that for smooth utility functions the application of this technique results in an outcome that closely matches the outcome based on the original interdependent utility structure. The method is evaluated for a number of randomly generated utility spaces with interdependent issues. Experiments show that this approach can achieve reasonably good outcomes for utility spaces with simple dependencies. However, an approximation error that deviates negotiation outcomes from the optimal solutions cannot be avoided, and this error may become larger when the approximated utility functions become more complex. Authors acknowledge as a necessary future work to study which kind of functions can be approximated accurately enough using this mechanism. Another limitation of this approach is that it is necessary to estimate a region 4 Will-be-set-by-IN-TECH

Correlation distance is defined as the minimum distance *ψ* which makes correlation fall below a given threshold (usually 0.5). This measure gives an idea of the distance we can move throughout the solution space while keeping a certain correlation between samples (Manderick et al., 1991). Throughout this work, we will use the terms "correlated" or

(Klein et al., 2003) present the first negotiation protocols specific for complex preference spaces. They propose a simulated annealing-based approach, a refined version based on a parity-maintaining annealing mediator, and an unmediated version of the negotiation protocol. Of great interest in this work are the positive results about the use of simulated annealing as a way to regulate agent decision making, along with the use of an expressive bid language to allow the mediator to improve its proposals. However, this expressiveness is somewhat limited, with only four possible valuations which allow the mediator to decide which contract to use as a parent for mutation, but not in which direction to mutate it. On the other hand, the performed experiments only consider the bilateral negotiation scenario, though authors claim that the multiparty generalization is simple. Finally, the family of negotiation protocols they propose are specific for binary issues and binary dependencies. Higher-order dependencies and continuous-valued issues, common in many real-world contexts, are known to generate more challenging utility landscapes which are not considered

(Luo et al., 2003) propose a fuzzy constraint based framework for multi-attribute negotiations. In this framework a buyer agent defines a set of fuzzy constraints to describe its preferences. The proposals of the buyer agent are a set of hard constraints which are extracted from the set of fuzzy constraints. The seller agent responds with an offer or with a relaxation request. The buyer then decides whether to accept or reject an offer, or to relax some constraints by priority from the lowest to highest. In (Lopez-Carmona et al., 2006; 2007) an improvement to Luo's model is presented. They devise an expressive negotiation protocol where proposals include a valuation of the different constraints, and seller's responses may contain explicit relaxation requests. It means that a seller agent may suggest the specific relaxation of one or more constraints. The relaxation suggested by a seller agent is based on utility and viability criteria, which improves the negotiation process. Though these constraint-based works model discontinuous preference spaces, the operators used to compute utility and the utility spaces defined yield monotonic preference spaces, which are far from the complex preference spaces

Another interesting approach to solve the computational cost and complexity of negotiating interdependent issues is to simplify the negotiation space. (Hindriks et al., 2006) propose a weighted approximation technique to simplify the utility space. They show that for smooth utility functions the application of this technique results in an outcome that closely matches the outcome based on the original interdependent utility structure. The method is evaluated for a number of randomly generated utility spaces with interdependent issues. Experiments show that this approach can achieve reasonably good outcomes for utility spaces with simple dependencies. However, an approximation error that deviates negotiation outcomes from the optimal solutions cannot be avoided, and this error may become larger when the approximated utility functions become more complex. Authors acknowledge as a necessary future work to study which kind of functions can be approximated accurately enough using this mechanism. Another limitation of this approach is that it is necessary to estimate a region

"uncorrelated" to refer to utility spaces in terms of their correlation lengths.

in their work.

covered in our work.

of the utility space where the actual outcome is expected to be (i.e. it is assumed that the region is known *a priori* by the agents).

In (Robu et al., 2005) utility graphs are used to model issue interdependencies for binary-valued issues. Utility graphs are inspired by graph theory and probabilistic influence networks to derive efficient heuristics for non-mediated bilateral negotiations about multiple issues. The idea is to decompose highly non-linear utility functions in sub-utilities of clusters of inter-related items. They show how utility graphs can be used to model an opponent's preferences. In this approach agents need prior information about the maximal structure of the utility space to be explored. Authors argue that this prior information could be obtained through a history of past negotiations or the input of domain experts. However, our approach has the advantage that outcomes can be reached without any prior information and that it is not restricted to binary-valued issues.

There are several proposals which employ genetic algorithms to learn opponent's preferences according to the history of the counter-offers based upon stochastic approximation. In (Choi et al., 2001) a system based on genetic-algorithms for electronic business is proposed. In this work the utility functions are restricted to take a product combination form (i.e. utility of an outcome is the product of the utility values of the different issues). The objective function used is based on the comparison of the changes of consecutive offers. Small changes of an issue suggest that this issue is more important. For each new population, the protocol enforces that the generated candidates cannot be better than the previous offer. Unlike other negotiation models based on genetic algorithms, this proposal adapts to the environment by dynamically modifying its mutation rate. (Lau et al., 2004) have also reported a negotiation mechanism for non-mediated automated negotiations based on genetic algorithms. The fitness function relies on three aspects: an agent's own preference, the distance of a candidate offer from the previous opponent's offer, and time pressure. In this work agents' preferences are quantified by a linear aggregation of the issue valuations. However, rugged and discontinuous preference spaces are not explored. In (Chou et al., 2007) a genetic algorithm is proposed which is based on a joint elitism operation and a joint fitness operation. In the joint elitism operation an agent stores the latest offers received from the opponent. The joint fitness operation combines agent's own utility function and euclidean distance to the opponent's offer. In this work two different negotiation scenarios are considered. In the first one utility is defined as the weighted sum of the different issue values (i.e. issues are independent). The second scenario defines a utility function where there is a master issue and a set of slave issues. Utility is calculated as the weighted sum of the different issue values, but the weights of the slave and master issues change according to the value of the master issue.

In (Yager, 2007) a mediated negotiation framework for multi-agent negotiation is presented. This framework involves a mediation step in which the individual preference functions are aggregated to obtain a group preference function. The main interest is focused on the implementation of the mediation rule where they allow a linguistic description using fuzzy logic. A notable feature of their approach is the inclusion of a mechanism rewarding the agents for being open to alternatives other than simply their most preferred. The negotiation space and utility values are assumed to be arbitrary (i.e. preferences can be uncorrelated). However, the set of possible solutions is defined *a priori* and is fixed. Moreover, the preference function needs to be provided to the mediation step in the negotiation process, and pareto-optimality is not considered. Instead, a stopping rule is considered, which determines when the rounds of mediation stop.

<sup>0</sup> <sup>1</sup> <sup>2</sup> <sup>3</sup> <sup>4</sup> <sup>5</sup> <sup>0</sup>

y

x

<sup>183</sup> Mediated Heuristic Approaches and Alternative Social Welfare Definitions

Issue 1

this chapter, we will focus on the iterative negotiation protocol and the mechanisms used for

Nonlinear agent preferences can be described by using different categories of functions, like K-additive utility functions (Chevaleyre et al., 2004), bidding languages (Nisan, 2006), or weighted constraints (Ito et al., 2008). In this section we focus on nonlinear utility spaces generated by means of weighted constraints. In these cases, agents' utility functions are described by defining a set of constraints. Each constraint represents a region with one or more dimensions, and has an associated utility value. The number of dimensions of the space is given by the number of issues *n* under negotiation, and the number of dimensions of each constraint must be lesser or equal than *n*. The utility yielded by a given potential solution (contract) in the utility space for an agent is the sum of the utility values of all the constraints that are satisfied by that contract. Figure 1 shows a very simple example of an agent's utility space for two issues and three constraints: a unary constraint *C*1 and two binary constraints *C*2 and *C*3. The utility values associated to the constraints are also shown in the figure. In this example, contract *x* would yield a utility value for the agent *u*(*x*) = 15, since it satisfies both *C*1 and *C*2, while contract *y* would yield a utility value *u*(*y*) = 5, because it only satisfies *C*1. More formally, we can define the issues under negotiation as a finite set of variables *x* = {*xi*|*i* = 1, ..., *n*}, and a contract (or a possible solution to the negotiation problem) as a vector

*<sup>i</sup>* |*i* = 1, ..., *n*} defined by the issues' values. Issues take values from the domain of

*<sup>i</sup>* ] defines the minimum and maximum values for each issue to satisfy the

*<sup>i</sup>* <sup>∈</sup> *<sup>I</sup><sup>c</sup>*

*<sup>i</sup>* |*i* = 1, ..., *n*}, where

*<sup>i</sup>* ∀*i*. For notation simplicity, we denote

Agent utility space is defined as a set of constraints *C* = {*ck*|*k* = 1, ..., *l*}. Each constraint is given by a set of intervals which define the region where a contract must be contained to

this as *s* ∈ *x*(*ck*), meaning that *s* is in the set of contracts that satisfy *ck*. An agent's utility

satisfy the constraint. In this way a constraint *<sup>c</sup>* is defined as *<sup>c</sup>* <sup>=</sup> {*I<sup>c</sup>*

constraint. Each constraint *ck* has an associated utility value *u*(*ck*).

A contract *s* satisfies a constraint *c* if and only if *x<sup>s</sup>*

1

deal identification at the mediator.

*<sup>s</sup>* <sup>=</sup> {*x<sup>s</sup>*

*Ic*

integers [0, *X*].

*<sup>i</sup>* = [*xmin*

*<sup>i</sup>* , *<sup>x</sup>max*

**2.1 Constraint-based nonlinear utility spaces**

2

3

Issue 2

4

C2:10

for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces

C3:12 C1:5

Fig. 1. Example of a utility space with two issues and three constraints.

5

(Fatima et al., 2009) analyze bilateral multi-issue negotiation involving nonlinear utility functions. They consider the case where issues are divisible and there are time constraints in the form of deadlines and discounts. They show that it is possible to reach Pareto-optimal agreements by negotiating all the issues together, and that finding an equilibrium is not computationally easy if the agents' utility functions are nonlinear. In order to overcome this complexity they investigate two solutions: approximating nonlinear utilities with linear ones; and using a simultaneous procedure where the issues are discussed in parallel but independently of each other. This study shows that the equilibrium can be computed in polynomial time. An important part of this work is the complexity analysis and estimated approximation error analysis performed over the proposed approximated equilibrium strategies. Heuristic approaches have generally the drawback of the lack of a solid mathematical structure which guarantees their viability, which raises the need of an exhaustive experimental evaluation. An adequate complexity analysis and establishing a bound over the approximation error contribute to give heuristic approaches part of the technical soundness they usually lack. Among the limitations of the proposal, we can point out that this work is focused on symmetric agents where the preferences are distributed identically, and the utility functions are separable in nonlinear polynomials of a single variable. This somewhat limits the complexity of the preference space.

Finally, combinatorial auctions (Giovannucci et al., 2010; Hunsberger & Grosz, 2000; Sakurai et al., 2000; Sandholm, 2002; Volgenant, 2002; Xia et al., 2005) can enable large-scale collective decision making in nonlinear domains, but only of a very limited type (i.e. negotiations consisting solely of resource allocation decisions). Multi-attribute auctions, wherein buyers advertise their utility functions, and sellers compete to offer the highest-utility bid (Bichler et al., 1999; Parkes & Kalagnanam, 2005; Teich et al., 2006) are also aimed at a fundamentally limited problem (a purchase negotiation with a single buyer) and require full revelation of preference information.

In summary, in the existing research nearly all the models which assume issue interdependency rely on monotonic utility spaces, binary valued issues, low-order dependencies, or a fixed set of defined *a priori* solutions. Simplification of the negotiation space has also been reported as a valid approach for simple utility functions, but it cannot be used with higher-order issue dependencies which make utility spaces very uncorrelated. Therefore, new approaches are needed if automated negotiation is to be applied to settings involving highly uncorrelated preference spaces. One of the most promising avenues for research in this area is the use of mediated protocols, and the use of heuristics based on optimization mechanisms to make these protocols converge to a suitable agreement. We will see in the following some of these approaches, with special emphasis in the way they use alternative social welfare notions to impact the final outcome of the negotiations.

#### **2. An auction-based approach for negotiations in complex, constraint-based utility spaces**

In this section we describe a negotiation model specifically designed for complex utility spaces, which was first described in (Marsa-Maestre et al., 2009). The model relies on the use of a constraint-based agent preference structure, a mediated, auction-based interaction protocol capable to support multiple iterations and expressive feedback at each iteration, and a set of heuristic mechanisms for the negotiating agents and the mediator. For the purpose of

Fig. 1. Example of a utility space with two issues and three constraints.

this chapter, we will focus on the iterative negotiation protocol and the mechanisms used for deal identification at the mediator.

#### **2.1 Constraint-based nonlinear utility spaces**

6 Will-be-set-by-IN-TECH

(Fatima et al., 2009) analyze bilateral multi-issue negotiation involving nonlinear utility functions. They consider the case where issues are divisible and there are time constraints in the form of deadlines and discounts. They show that it is possible to reach Pareto-optimal agreements by negotiating all the issues together, and that finding an equilibrium is not computationally easy if the agents' utility functions are nonlinear. In order to overcome this complexity they investigate two solutions: approximating nonlinear utilities with linear ones; and using a simultaneous procedure where the issues are discussed in parallel but independently of each other. This study shows that the equilibrium can be computed in polynomial time. An important part of this work is the complexity analysis and estimated approximation error analysis performed over the proposed approximated equilibrium strategies. Heuristic approaches have generally the drawback of the lack of a solid mathematical structure which guarantees their viability, which raises the need of an exhaustive experimental evaluation. An adequate complexity analysis and establishing a bound over the approximation error contribute to give heuristic approaches part of the technical soundness they usually lack. Among the limitations of the proposal, we can point out that this work is focused on symmetric agents where the preferences are distributed identically, and the utility functions are separable in nonlinear polynomials of a single

Finally, combinatorial auctions (Giovannucci et al., 2010; Hunsberger & Grosz, 2000; Sakurai et al., 2000; Sandholm, 2002; Volgenant, 2002; Xia et al., 2005) can enable large-scale collective decision making in nonlinear domains, but only of a very limited type (i.e. negotiations consisting solely of resource allocation decisions). Multi-attribute auctions, wherein buyers advertise their utility functions, and sellers compete to offer the highest-utility bid (Bichler et al., 1999; Parkes & Kalagnanam, 2005; Teich et al., 2006) are also aimed at a fundamentally limited problem (a purchase negotiation with a single buyer) and require full

In summary, in the existing research nearly all the models which assume issue interdependency rely on monotonic utility spaces, binary valued issues, low-order dependencies, or a fixed set of defined *a priori* solutions. Simplification of the negotiation space has also been reported as a valid approach for simple utility functions, but it cannot be used with higher-order issue dependencies which make utility spaces very uncorrelated. Therefore, new approaches are needed if automated negotiation is to be applied to settings involving highly uncorrelated preference spaces. One of the most promising avenues for research in this area is the use of mediated protocols, and the use of heuristics based on optimization mechanisms to make these protocols converge to a suitable agreement. We will see in the following some of these approaches, with special emphasis in the way they use

alternative social welfare notions to impact the final outcome of the negotiations.

**2. An auction-based approach for negotiations in complex, constraint-based**

In this section we describe a negotiation model specifically designed for complex utility spaces, which was first described in (Marsa-Maestre et al., 2009). The model relies on the use of a constraint-based agent preference structure, a mediated, auction-based interaction protocol capable to support multiple iterations and expressive feedback at each iteration, and a set of heuristic mechanisms for the negotiating agents and the mediator. For the purpose of

variable. This somewhat limits the complexity of the preference space.

revelation of preference information.

**utility spaces**

Nonlinear agent preferences can be described by using different categories of functions, like K-additive utility functions (Chevaleyre et al., 2004), bidding languages (Nisan, 2006), or weighted constraints (Ito et al., 2008). In this section we focus on nonlinear utility spaces generated by means of weighted constraints. In these cases, agents' utility functions are described by defining a set of constraints. Each constraint represents a region with one or more dimensions, and has an associated utility value. The number of dimensions of the space is given by the number of issues *n* under negotiation, and the number of dimensions of each constraint must be lesser or equal than *n*. The utility yielded by a given potential solution (contract) in the utility space for an agent is the sum of the utility values of all the constraints that are satisfied by that contract. Figure 1 shows a very simple example of an agent's utility space for two issues and three constraints: a unary constraint *C*1 and two binary constraints *C*2 and *C*3. The utility values associated to the constraints are also shown in the figure. In this example, contract *x* would yield a utility value for the agent *u*(*x*) = 15, since it satisfies both *C*1 and *C*2, while contract *y* would yield a utility value *u*(*y*) = 5, because it only satisfies *C*1.

More formally, we can define the issues under negotiation as a finite set of variables *x* = {*xi*|*i* = 1, ..., *n*}, and a contract (or a possible solution to the negotiation problem) as a vector *<sup>s</sup>* <sup>=</sup> {*x<sup>s</sup> <sup>i</sup>* |*i* = 1, ..., *n*} defined by the issues' values. Issues take values from the domain of integers [0, *X*].

Agent utility space is defined as a set of constraints *C* = {*ck*|*k* = 1, ..., *l*}. Each constraint is given by a set of intervals which define the region where a contract must be contained to satisfy the constraint. In this way a constraint *<sup>c</sup>* is defined as *<sup>c</sup>* <sup>=</sup> {*I<sup>c</sup> <sup>i</sup>* |*i* = 1, ..., *n*}, where *Ic <sup>i</sup>* = [*xmin <sup>i</sup>* , *<sup>x</sup>max <sup>i</sup>* ] defines the minimum and maximum values for each issue to satisfy the constraint. Each constraint *ck* has an associated utility value *u*(*ck*).

A contract *s* satisfies a constraint *c* if and only if *x<sup>s</sup> <sup>i</sup>* <sup>∈</sup> *<sup>I</sup><sup>c</sup> <sup>i</sup>* ∀*i*. For notation simplicity, we denote this as *s* ∈ *x*(*ck*), meaning that *s* is in the set of contracts that satisfy *ck*. An agent's utility

Fig. 2. State diagram of the protocol.

(Marsa-Maestre, 2009).

it an amount of time or a number of iterations).

transitions involving different agents, by the exchanged locutions.

4. *Adaptive Bidding*: Each agent creates a new set of bids *B<sup>j</sup>* taking into account the received feedback. Steps 2 to 4 are repeated until a solution is found or a given limit is reached (be

<sup>185</sup> Mediated Heuristic Approaches and Alternative Social Welfare Definitions

for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces

For the formal description of the interaction protocol, we have employed an specification framework based on *formal dialogue games* (Amgoud et al., 2000; Hitchcock et al., 2001; Lopez-Carmona, 2006; McBurney & Parsons, 2002), where the interaction protocol is modeled by defining three key elements. A set of locutions establishes which communications are permitted between the agents. A set of decision mechanisms allows agents to advance through the different stages of the protocol and to invoke the different locutions as needed. Finally, operational semantics indicates how the dialogue game evolves depending on the outputs of the different mechanisms and the locutions exchanged by the agents. The detailed specification of the protocol is beyond the scope of this discussion, and can be found in

For the purpose of this chapter, to have a clear understanding of the role of the different elements within the protocol and the relationships between them will suffice. The dialogue may be seen as a set of states, where these states are defined by the different mechanisms, and where transition rules govern the transitions within the states when a certain locution is passed between agents or a certain output is passed between different mechanisms of the same agent. Figure 2 shows a state diagram which represents the link between transition rules, decision mechanisms, and locutions. The states represent the decision mechanisms, and the transitions are specified by the outputs of the mechanisms firing them and, in case of

for a contract *<sup>s</sup>* is defined as *<sup>u</sup>*(*s*) = <sup>∑</sup>*ck*∈*C*|*s*∈*x*(*ck*) *<sup>u</sup>*(*ck*), that is, the sum of the utility values of all constraints satisfied by *s*. This kind of utility functions produces nonlinear utility spaces, with high points where many constraints are satisfied, and lower regions where few or no constraints are satisfied (Ito et al., 2008). For the purpose of this work, we make the following additional assumptions, relaxing some of the common limitations of previous approaches, which allows us to tackle more realistic scenarios:


The proposed negotiation protocol and decision mechanisms, which are described in the next sections, are specifically designed to address this negotiation setting.

#### **2.2 Expressive, iterative interaction protocol**

In highly uncorrelated utility spaces, one of the main problems of basic auction-based negotiation is that it usually relies on a one-shot protocol: agents send their bids to the mediator, the mediator searches for solutions and the negotiation ends. If a solution has been found, the negotiation is successful. Otherwise, the only possibility is to repeat the process until it is successful. In scenarios where high-utility regions are "wide" this is hardly a problem, since agent bids usually cover a wide range of the solution space, and the probability for the mediator to find a successful deal is high. In highly uncorrelated utility spaces, however, high utility regions agents send as bids tend to be narrower, and thus it is very likely that a single iteration of the protocol achieves no solution. In these cases, it would be desirable that agents had a way to "learn" from previous interactions to issue bids which are more likely to result in a deal. For this to be possible, two mechanisms are needed: a mechanism for the mediator to provide feedback to the agents, and a mechanism which allows the agents to use this feedback in the generation of new bids.

Our proposal for an iterative, expressive protocol may be briefly described as consisting of the following steps:


Fig. 2. State diagram of the protocol.

8 Will-be-set-by-IN-TECH

for a contract *<sup>s</sup>* is defined as *<sup>u</sup>*(*s*) = <sup>∑</sup>*ck*∈*C*|*s*∈*x*(*ck*) *<sup>u</sup>*(*ck*), that is, the sum of the utility values of all constraints satisfied by *s*. This kind of utility functions produces nonlinear utility spaces, with high points where many constraints are satisfied, and lower regions where few or no constraints are satisfied (Ito et al., 2008). For the purpose of this work, we make the following additional assumptions, relaxing some of the common limitations of previous approaches,

• We assume that the number of issues and the domains of the issues are such that they make

• We assume that the utility spaces of an agent is highly uncorrelated, and so no *a priori* assumptions may be made about where high utility contracts may be located. Therefore, agents may need to resort to local nonlinear optimization techniques to identify such

• We assume knowledge about other agent's preferences not to be common (i.e. agents do not know their opponent preference structures, neither they can compute opponent's

• We assume that the negotiation setting is competitive, and that agents may be unwilling to reveal too much information about their preferences to the other negotiating agents. The proposed negotiation protocol and decision mechanisms, which are described in the next

In highly uncorrelated utility spaces, one of the main problems of basic auction-based negotiation is that it usually relies on a one-shot protocol: agents send their bids to the mediator, the mediator searches for solutions and the negotiation ends. If a solution has been found, the negotiation is successful. Otherwise, the only possibility is to repeat the process until it is successful. In scenarios where high-utility regions are "wide" this is hardly a problem, since agent bids usually cover a wide range of the solution space, and the probability for the mediator to find a successful deal is high. In highly uncorrelated utility spaces, however, high utility regions agents send as bids tend to be narrower, and thus it is very likely that a single iteration of the protocol achieves no solution. In these cases, it would be desirable that agents had a way to "learn" from previous interactions to issue bids which are more likely to result in a deal. For this to be possible, two mechanisms are needed: a mechanism for the mediator to provide feedback to the agents, and a mechanism which allows the agents to use

Our proposal for an iterative, expressive protocol may be briefly described as consisting of the

the solution space which, in general, contains only contracts the agent would accept as

2. *Deal Identification*: The mediator employs a search method to find overlaps between agents'

3. *Feedback*: If no satisfying solution is found, the mediator creates feedback statements *ρ<sup>j</sup>* for

*<sup>b</sup>* bids. Each bid represents a region in

exhaustive search within the utility space of the agents intractable.

sections, are specifically designed to address this negotiation setting.

which allows us to tackle more realistic scenarios:

high-utility contracts.

utility for a given contract).

**2.2 Expressive, iterative interaction protocol**

this feedback in the generation of new bids.

solutions to the negotiation problem.

each agent and distribute them.

1. *Bidding*: Each agent *Aj* generates a set *<sup>B</sup><sup>j</sup>* with *<sup>n</sup><sup>j</sup>*

bids. If a solution is found, the protocol ends.

following steps:

4. *Adaptive Bidding*: Each agent creates a new set of bids *B<sup>j</sup>* taking into account the received feedback. Steps 2 to 4 are repeated until a solution is found or a given limit is reached (be it an amount of time or a number of iterations).

For the formal description of the interaction protocol, we have employed an specification framework based on *formal dialogue games* (Amgoud et al., 2000; Hitchcock et al., 2001; Lopez-Carmona, 2006; McBurney & Parsons, 2002), where the interaction protocol is modeled by defining three key elements. A set of locutions establishes which communications are permitted between the agents. A set of decision mechanisms allows agents to advance through the different stages of the protocol and to invoke the different locutions as needed. Finally, operational semantics indicates how the dialogue game evolves depending on the outputs of the different mechanisms and the locutions exchanged by the agents. The detailed specification of the protocol is beyond the scope of this discussion, and can be found in (Marsa-Maestre, 2009).

For the purpose of this chapter, to have a clear understanding of the role of the different elements within the protocol and the relationships between them will suffice. The dialogue may be seen as a set of states, where these states are defined by the different mechanisms, and where transition rules govern the transitions within the states when a certain locution is passed between agents or a certain output is passed between different mechanisms of the same agent. Figure 2 shows a state diagram which represents the link between transition rules, decision mechanisms, and locutions. The states represent the decision mechanisms, and the transitions are specified by the outputs of the mechanisms firing them and, in case of transitions involving different agents, by the exchanged locutions.

**2.3 A probabilistic deal identification mechanism**

agents, the maximum number of bids per agent is *na*

uncorrelated utility spaces, where bids are narrower.

*Pdeal* =

*<sup>l</sup>* is the volume of each *l*-th bid.

*E*[*u<sup>k</sup>* ] = ⎡ ⎣ *nk b* ∑ *l*=1 *uk <sup>l</sup>* · *<sup>v</sup><sup>k</sup> l* ⎤ ⎦ ⎡ ⎣ ∏ *n<sup>k</sup> bp* ∑ *j*=1

∏ *n<sup>k</sup> bp* ∑ *j*=1

for the issues (assuming all issues have the same domain size), and *n<sup>k</sup>*

In a similar way, we can see that the *expected utility* for agent *k* is given by

expected social welfare the mediator should try to maximize <sup>∑</sup>*n<sup>k</sup>*

(−1) *j*+1

*bp* <sup>=</sup> <sup>∑</sup>*n<sup>k</sup> b <sup>l</sup>*=<sup>1</sup> *<sup>v</sup><sup>k</sup>*

a successful negotiation).

negotiation protocol is given by

by agent *k* bids. This is given by *n<sup>k</sup>*

way bid utility and bid volume.

agent *k* and *v<sup>k</sup>*

where *u<sup>k</sup>*

worst case scenario, this means searching through a total of *nna*

for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces

number *nbc* of randomly chosen bid combinations, where *nbc* <sup>&</sup>lt; *<sup>n</sup>na*

Scalability is identified as one of the main drawbacks in a bidding based negotiation protocol (Ito et al., 2007). Once agents have placed their bids, the mediator performs an exhaustive search for overlaps between the bids using a breadth-first algorithm with branch cutting. In a

<sup>187</sup> Mediated Heuristic Approaches and Alternative Social Welfare Definitions

the number of bids per agent, and *na* is the number of negotiating agents. In the experiments, the authors limit the number of combinations to 6, 400, 000. This means that, for 4 negotiating

as the number of agents increases. For example, for 10 agents, the limit is 4 bids per agent, which drastically reduces the probability of reaching a deal. This is specially true for highly

To address this scalability limitation, we propose to perform a probabilistic search in the mediator instead of an exhaustive search. This means that the mediator will try a certain

as a performance parameter in the mediator, which limits the computational cost of the deal identification phase. Of course, restricting the search for solutions to a limited number of combinations may cause the mediator to miss good deals. Taking this into account, the random selection of combinations should be biased to maximize the probability of finding a good deal. Intuitively, in such complex scenarios, a mediator with no knowledge of the agents' preferences could try to achieve that by balancing the utility of the agents' bids (to maximize agent's social welfare) and the volume of those bids (to maximize the probability of

More formally, we may find mathematic expressions for the deal probability and the expected utility in a negotiation using the auction-based protocol. The deduction of these expressions can be found in (Lopez-Carmona et al., 2010). For the purpose of this section, the final expressions will suffice. In particular, deal probability for a single run of the auction-based

> �∏ *n<sup>k</sup> bp j*

where *<sup>n</sup><sup>a</sup>* is the number of negotiating agents, *<sup>n</sup>* is the number of issues, <sup>|</sup>*D*<sup>|</sup> is the domain size

*contracts* for agent *k*, that is, an indication of the portion of the solution space which is covered

� �

*<sup>l</sup>* , where *<sup>n</sup><sup>k</sup>*

�∏ *n<sup>k</sup> bp j*

*<sup>l</sup>* is the utility for the *l*-th bid of agent *k*. According to this expression, to maximize

1

� (−1)


*j*+1

*b <sup>l</sup>*=<sup>1</sup> *<sup>u</sup><sup>k</sup> <sup>l</sup>* · *<sup>v</sup><sup>k</sup>*

⎤

*n*(*na*−1)*j*

*n*(*na*−1)

�*j*


*<sup>b</sup>* bid combinations, where *nb* is

*<sup>b</sup>* . In this way, *nbc* acts

, (1)

*bp* is the *number of bidden*

⎦ , (2)

*<sup>l</sup>* , balancing in this

*<sup>b</sup>* is the number of bids issued by

<sup>√</sup><sup>6400000</sup> <sup>=</sup> 50. This limit becomes harder

The protocol considers the existence of nine decision mechanisms: four for the negotiating agents (labeled *Ax* in the diagram) and five for the mediator (labeled *Mx*). Mechanisms *M1*, *A1*, *M2*, and *M5* focus on the opening and closing of the negotiation dialogues, and are not of much interest for this specific work. Therefore, in the rest of the paper we will assume simple implementations for these mechanisms. On the other hand, mechanisms *A2*, *A3*, *M3*, and *M4* play key roles in the negotiation stage of the protocol, and will be further discussed in the following section. Before addressing in detail the mechanisms directly related to the negotiation stage, we will briefly review the different mechanisms considered by the protocol:


For the negotiation model to be complete, it is necessary to define the negotiation mechanisms which will implement the agent strategies. Since this chapter is concerned mainly with mediated approaches and their use of heuristic methods and alternative social welfare notions, in the rest of this section we will focus in the mechanisms used to identify deals within the mediator (*M3*). Mechanisms used for bidding can be found in (Lopez-Carmona et al., 2010).

#### **2.3 A probabilistic deal identification mechanism**

10 Will-be-set-by-IN-TECH

The protocol considers the existence of nine decision mechanisms: four for the negotiating agents (labeled *Ax* in the diagram) and five for the mediator (labeled *Mx*). Mechanisms *M1*, *A1*, *M2*, and *M5* focus on the opening and closing of the negotiation dialogues, and are not of much interest for this specific work. Therefore, in the rest of the paper we will assume simple implementations for these mechanisms. On the other hand, mechanisms *A2*, *A3*, *M3*, and *M4* play key roles in the negotiation stage of the protocol, and will be further discussed in the following section. Before addressing in detail the mechanisms directly related to the negotiation stage, we will briefly review the different mechanisms considered by the protocol: *M1: Start Negotiation.* Determines the initial feasibility of the negotiation. In the following, we will assume that all negotiations proposed by the mediator are feasible, and thus this

*A1: Assess Negotiation.* Decides whether the negotiating agent participates or not in the proposed negotiation. We will assume that not participating in the negotiation has zero utility for the agent, and so it will always participate. Thus, the output of this mechanism

*M2: Confirm Dialogue.* Confirms the feasibility of the negotiation once the responses from all agents have been received. Since the negotiation is defined as feasible in the output of the mechanism *M1: Start Negotiation* and all agents are willing to participate in the dialog, the

*A2: Generate Bids.* Generates a set of bids to send to the mediator. This mechanism, along with mechanism *A3: Assess Feedback*, constitutes the core of the agent negotiation strategy. Therefore, in the following sections we will address different strategies for bid generation. *M3: Identify Deals.* Searches for solutions to the negotiation problem once the bids from the different agents have been received. The deal identification strategy at the mediator greatly influences important properties of the model such as social welfare maximization, deal

*M4: Generate feedback.* Builds feedback statements sent to the negotiating agents to try to influence the generation of new bids at each protocol iteration. This mechanisms influences

*A4: Assess Solution.* Decides whether the negotiating agent accept or rejects a solution found by the mediator. We will assume that negotiating agents have a *reservation value* which remains unchanged during the negotiation. A negotiating agent will accept any solution

*M5: Confirm Solution.* Confirms the success or failure of a negotiation once the responses from the agents have been received. We will assume that a negotiation is successful if *all*

For the negotiation model to be complete, it is necessary to define the negotiation mechanisms which will implement the agent strategies. Since this chapter is concerned mainly with mediated approaches and their use of heuristic methods and alternative social welfare notions, in the rest of this section we will focus in the mechanisms used to identify deals within the mediator (*M3*). Mechanisms used for bidding can be found in (Lopez-Carmona et al.,

the ability of the model to refine the search for solutions in successive iterations. *A3: Assess Feedback.* This mechanism complements the previous one in the task to provide the model with the possibility for iterative improvement. It is used by negotiating agents to generate a new set of bids at each protocol iteration, taking into account the feedback

proposed by the mediator which is above its reservation value.

negotiating agents accept the solution proposed by the mediator.

mechanism will always yield as output *feasible\_negotiation*.

output of this message will always be *feasible\_negotiation*.

will always be *enter\_dialogue*.

probability and performance.

received from the mediator.

2010).

Scalability is identified as one of the main drawbacks in a bidding based negotiation protocol (Ito et al., 2007). Once agents have placed their bids, the mediator performs an exhaustive search for overlaps between the bids using a breadth-first algorithm with branch cutting. In a worst case scenario, this means searching through a total of *nna <sup>b</sup>* bid combinations, where *nb* is the number of bids per agent, and *na* is the number of negotiating agents. In the experiments, the authors limit the number of combinations to 6, 400, 000. This means that, for 4 negotiating agents, the maximum number of bids per agent is *na* <sup>√</sup><sup>6400000</sup> <sup>=</sup> 50. This limit becomes harder as the number of agents increases. For example, for 10 agents, the limit is 4 bids per agent, which drastically reduces the probability of reaching a deal. This is specially true for highly uncorrelated utility spaces, where bids are narrower.

To address this scalability limitation, we propose to perform a probabilistic search in the mediator instead of an exhaustive search. This means that the mediator will try a certain number *nbc* of randomly chosen bid combinations, where *nbc* <sup>&</sup>lt; *<sup>n</sup>na <sup>b</sup>* . In this way, *nbc* acts as a performance parameter in the mediator, which limits the computational cost of the deal identification phase. Of course, restricting the search for solutions to a limited number of combinations may cause the mediator to miss good deals. Taking this into account, the random selection of combinations should be biased to maximize the probability of finding a good deal. Intuitively, in such complex scenarios, a mediator with no knowledge of the agents' preferences could try to achieve that by balancing the utility of the agents' bids (to maximize agent's social welfare) and the volume of those bids (to maximize the probability of a successful negotiation).

More formally, we may find mathematic expressions for the deal probability and the expected utility in a negotiation using the auction-based protocol. The deduction of these expressions can be found in (Lopez-Carmona et al., 2010). For the purpose of this section, the final expressions will suffice. In particular, deal probability for a single run of the auction-based negotiation protocol is given by

$$P\_{dead} = \sum\_{j=1}^{\prod n\_{bp}^k} (-1)^{j+1} \binom{\prod n\_{bp}^k}{j} \left(\frac{1}{|D|^{n(n^a-1)}}\right)^j,\tag{1}$$

where *<sup>n</sup><sup>a</sup>* is the number of negotiating agents, *<sup>n</sup>* is the number of issues, <sup>|</sup>*D*<sup>|</sup> is the domain size for the issues (assuming all issues have the same domain size), and *n<sup>k</sup> bp* is the *number of bidden contracts* for agent *k*, that is, an indication of the portion of the solution space which is covered by agent *k* bids. This is given by *n<sup>k</sup> bp* <sup>=</sup> <sup>∑</sup>*n<sup>k</sup> b <sup>l</sup>*=<sup>1</sup> *<sup>v</sup><sup>k</sup> <sup>l</sup>* , where *<sup>n</sup><sup>k</sup> <sup>b</sup>* is the number of bids issued by agent *k* and *v<sup>k</sup> <sup>l</sup>* is the volume of each *l*-th bid.

In a similar way, we can see that the *expected utility* for agent *k* is given by

$$E[u^k] = \left[\sum\_{l=1}^{n\_b^k} u\_l^k \cdot v\_l^k \right] \left[\sum\_{j=1}^{\prod n\_{bp}^k} \binom{\prod n\_{bp}^k}{j} \frac{(-1)^{j+1}}{|D|^{n(n^a-1)j}}\right],\tag{2}$$

where *u<sup>k</sup> <sup>l</sup>* is the utility for the *l*-th bid of agent *k*. According to this expression, to maximize expected social welfare the mediator should try to maximize <sup>∑</sup>*n<sup>k</sup> b <sup>l</sup>*=<sup>1</sup> *<sup>u</sup><sup>k</sup> <sup>l</sup>* · *<sup>v</sup><sup>k</sup> <sup>l</sup>* , balancing in this way bid utility and bid volume.

• *Uniform combination selection*: The simplest approach is to make a totally random selection of bids for each combination. Intuitively, the main drawback of this approach is that, as discussed above, reducing the search space arbitrarily may lead us to discard high social

<sup>189</sup> Mediated Heuristic Approaches and Alternative Social Welfare Definitions

• *Tournament selection of bid combinations*: It seems reasonable to bias combination selection by means of the quality factor *Q*, that is, making the selection favor bids with high quality factor. A fairly straightforward way to do this is by means of *tournament selection*. For each bid combination, a subset of *nt* bids is randomly drawn from each agent, and the bid

• *Roulette selection of bid combinations*: In this case, to select a bid for an agent *j*, a set of labels

*<sup>i</sup>* corresponding to bid *b*

proportional to the bid quality factor. Once the label set has been generated, a random

to label *kj*-th in set *L<sup>j</sup>* is selected. In this way, the probability for each bid to be selected is

is drawn from a probability distribution favoring lower numbers, and the *kj*-th bid is selected. Some probability distribution functions which may be used are the negative binomial distribution, the geometric distribution (a particular case of the previous one),

• *Ordinal probabilistic selection*: For each agent *j*, the bid set *B<sup>j</sup>* is sorted in quality factor

As we can easily see, by using this strategy for deal identification, the mediator is favoring those bids combinations which maximize the sum of quality factors, rather than maximizing the sum of utilities. In this sense, the mediator is using an alternative form of social welfare, which not only cares about individual agent utility, but also about deal probability. Experiments performed in (Marsa-Maestre et al., 2009) show that using this Q-based social welfare notion allows for better performance and scalability in complex utility spaces. However, the protocol and deal identification mechanism are prone to some strategic stability problems, which may make individually rational agents agents to try to bid narrow, high utility bids to maximize their expected utility in case of agreement, but severely impacting deal probability. These stability issues can also be lowered by means of alternative social

**2.4 Addressing stability issues through the use of alternative social welfare notions**

In (Marsa-Maestre et al., 2009), once a set of viable solutions has been found, the mediator chooses as the solution the one which maximizes social welfare. However, as discussed in (Lopez-Carmona et al., 2010), when the mediator uses the classic social welfare option an agent's attitude greatly influences the final utility value for this agent if an agreement is reached. Once all valid intersections have been found, the final outcome is selected using a function which depends on the utility values the outcome gives to the agents. Selfish, risk-willing or highly competitive agents, which give more importance to utility against volume in the bid generation process, will have, on average, higher utility bids, and thus their expected utility conditioned to the existence of a deal will be higher. Taking this into account, the preferred strategy of an agent may be to take a selfish attitude, as we inferred

*Q*<sup>0</sup> descending order. Once the set has been sorted, a random number *kj* ∈

*j*

is generated for each agent *j*, and the bid corresponding

*<sup>i</sup>* is repeated a number of times

 1, *n<sup>j</sup> B* 

maximizing quality factor is selected from each subset to form the combination.

*j*

for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces

welfare solutions.

number *kj* ∈

*L<sup>j</sup>* is created, where each label *l*

proportional to its quality factor *Q*0.

or the positive normal distribution.

welfare, as we will see in the following section.

 1, <sup>∑</sup>*n<sup>j</sup> B <sup>l</sup>*=<sup>1</sup> *Q*<sup>0</sup> *b j i* 

We formally represent this through the following definitions:

**Definition 1.** *Volume of a bid. The* volume *of a given region b within the solution space D is defined as the cardinality of the set of contracts contained within the region defined by the bid.*

$$v(b) = |r| \, , \, with \, b \subset D$$

**Definition 2.** *Quality factor. The* quality factor *of a given region b within the solution space D (be it a constraint or a bid) is defined as*

$$Q\_0(b) = \mu(b) \cdot v(b)\_{\prime}$$

*where u*(*b*) *and v*(*b*) *are, respectively, the utility and volume of the bid b.*

Intuitively, it can be seen that the quality factor defined above allows an agent to balance bid utility (to maximize its own benefit) and bid volume (to maximize deal probability). Taking this into account, the parameter used to bias the random selection in the mechanism is *Q*0, so that higher-*Q*<sup>0</sup> bids have more probability of being selected for bid combinations at the mediator.

The mechanism is formally shown in Algorithm 1. We can see that the number of analyzed bid combinations is limited to *nbc* (1), and that the function *combine*\_*bids*(...) selects the bid combinations to analyze (2). Limiting bid combinations at the mediator allows us to remove the limit on the bids issued by the agents, which increases the probability of finding good potential deals. However, it may be convenient to keep a less strict limit in the number of bids sent to the mediator, in order to keep communication load of the negotiation process within reasonable bounds.

#### **Algorithm 1:** Probabilistic deal identification

#### **Input**:

```
A: set of negotiating agents;
    na = |A|: number of negotiating agents
    B: set of bids issued by every agent;
    U: declared utilities for every agent's bids
    Q0: declared quality factors for every agent's bids;
    sw: social welfare function
    nbc: maximum number of bid combinations at the mediator
  Output:
    sf : final potential deal
  n = 0;
  S = ∅;
1 while n < nbc do
2 s = combine_bids(A, na, B, U, Q0);
      if s �= ∅ then
           S = S ∪ s;
      n = n + 1;
  end
  sf = arg {maxs∈S sw (s, U)};
```
The core of this mechanism from an strategic point of view is the probabilistic selection function *combine*\_*bids*(*A*, *naB*, *U*, *Q*0), which generates a bid combination selecting a bid from each agent. This function is the one which ultimately determines which bid combinations are tested at the mediator. Different algorithms may be used to generate these combinations. Among the multiple possibilities, we have considered the following ones:

12 Will-be-set-by-IN-TECH

**Definition 1.** *Volume of a bid. The* volume *of a given region b within the solution space D is defined*

*v*(*b*) = |*r*| *, with b* ⊂ *D* **Definition 2.** *Quality factor. The* quality factor *of a given region b within the solution space D (be*

*Q*0(*b*) = *u*(*b*)· *v*(*b*),

Intuitively, it can be seen that the quality factor defined above allows an agent to balance bid utility (to maximize its own benefit) and bid volume (to maximize deal probability). Taking this into account, the parameter used to bias the random selection in the mechanism is *Q*0, so that higher-*Q*<sup>0</sup> bids have more probability of being selected for bid combinations at the

The mechanism is formally shown in Algorithm 1. We can see that the number of analyzed bid combinations is limited to *nbc* (1), and that the function *combine*\_*bids*(...) selects the bid combinations to analyze (2). Limiting bid combinations at the mediator allows us to remove the limit on the bids issued by the agents, which increases the probability of finding good potential deals. However, it may be convenient to keep a less strict limit in the number of bids sent to the mediator, in order to keep communication load of the negotiation process within

The core of this mechanism from an strategic point of view is the probabilistic selection function *combine*\_*bids*(*A*, *naB*, *U*, *Q*0), which generates a bid combination selecting a bid from each agent. This function is the one which ultimately determines which bid combinations are tested at the mediator. Different algorithms may be used to generate these combinations.

Among the multiple possibilities, we have considered the following ones:

*as the cardinality of the set of contracts contained within the region defined by the bid.*

We formally represent this through the following definitions:

*where u*(*b*) *and v*(*b*) *are, respectively, the utility and volume of the bid b.*

*it a constraint or a bid) is defined as*

mediator.

**Input**:

**Output**:

*n* = 0; *S* = ∅; **<sup>1</sup> while** *n* < *nbc* **do**

**end**

reasonable bounds.

*A*: set of negotiating agents;

*sw*: social welfare function

**<sup>2</sup>** *s* = *combine*\_*bids*(*A*, *na*, *B*, *U*, *Q*0);

*sf* : final potential deal

**if** *s* �= ∅ **then** *S* = *S* ∪ *s*; *n* = *n* + 1;

*sf* = arg {max*s*∈*<sup>S</sup> sw* (*s*, *U*)};

**Algorithm 1:** Probabilistic deal identification

*nbc*: maximum number of bid combinations at the mediator

*na* = |*A*|: number of negotiating agents *B*: set of bids issued by every agent; *U*: declared utilities for every agent's bids *Q*0: declared quality factors for every agent's bids;


As we can easily see, by using this strategy for deal identification, the mediator is favoring those bids combinations which maximize the sum of quality factors, rather than maximizing the sum of utilities. In this sense, the mediator is using an alternative form of social welfare, which not only cares about individual agent utility, but also about deal probability. Experiments performed in (Marsa-Maestre et al., 2009) show that using this Q-based social welfare notion allows for better performance and scalability in complex utility spaces. However, the protocol and deal identification mechanism are prone to some strategic stability problems, which may make individually rational agents agents to try to bid narrow, high utility bids to maximize their expected utility in case of agreement, but severely impacting deal probability. These stability issues can also be lowered by means of alternative social welfare, as we will see in the following section.

#### **2.4 Addressing stability issues through the use of alternative social welfare notions**

In (Marsa-Maestre et al., 2009), once a set of viable solutions has been found, the mediator chooses as the solution the one which maximizes social welfare. However, as discussed in (Lopez-Carmona et al., 2010), when the mediator uses the classic social welfare option an agent's attitude greatly influences the final utility value for this agent if an agreement is reached. Once all valid intersections have been found, the final outcome is selected using a function which depends on the utility values the outcome gives to the agents. Selfish, risk-willing or highly competitive agents, which give more importance to utility against volume in the bid generation process, will have, on average, higher utility bids, and thus their expected utility conditioned to the existence of a deal will be higher. Taking this into account, the preferred strategy of an agent may be to take a selfish attitude, as we inferred

for different number of selfish agents, thus studying the effect of possible coalitions or coincidences. Results showed that the mechanism based on average volume provided not enough improvement in stability. For all cases median utility results were higher for selfish agents, thus maintaining the incentive for agents to deviate from the socially optimal strategy. The mechanism based on average quality factor, however, significantly mitigated the gain for selfish agents, removing the incentive to choose the previously individually optimal strategy. Another threat to mechanism stability is strategic revelation of information. In incomplete information scenarios (Jonker et al., 2007), since the agents' beliefs about the preferences of a given agent may influence the decision mechanisms they use, an agent may use as a strategy to lie about its own preferences in order to manipulate the decision mechanisms of the rest of the agents to its own benefit. This raises an additional concern to mechanism design (Weiss, 1999). It would be desirable to design protocols which are not prone to be manipulated through insincere revelation of information. *Incentive-compatibility* is defined as the property of a negotiation mechanism which makes telling the truth the best strategy for any agent,

<sup>191</sup> Mediated Heuristic Approaches and Alternative Social Welfare Definitions

for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces

In the negotiation model we are dealing with, information revealed to the mediator is the set of agents' bids. These bids represent regions within the solution space. Each offer has an associated utility value, a volume, and an associated quality factor value. Since bid volume is directly related to the region represented by the bid, it does not seem feasible to fake it, since it can be easily checked by the mediator. Quality factor may be faked, but since the mediator is very likely to recompute it, this strategy is also harmless. Finally, agents may fake bid utility. Insincere information revelation about bid utility may generally occur in two ways: exaggerating upward or downward the utility values of *all* bids, or exaggerating the utility values of *some* bids with respect to the others. Exaggerating all bids is not profitable with the proposed deal identification mechanisms, since bid selection at the mediator is performed independently for each agent. This means that the bids from different agents do not compete among each other to be selected as part of a solution. In contrast, the different bids of a single agent compete among themselves. Taking this into account, an agent could try to exaggerate the utility value of its preferred bids, thus trying to increase the probability of the mediator choosing those preferred bids to form deals. As far as social welfare is concerned, this is a problem if the set of exaggerated bids is small with respect to the total set of bids, since that would reduce the number of effective bids considered by the mediator, thus reducing deal

Therefore, we seek for mechanisms which counter this effect, incentivizing sincere revelation of information. A possibility to achieve this is to normalize the utility values assigned by the agents to their bids, thus lowering the absolute differences in utility. We propose three

• *Normalization to maximum utility* : obtained by dividing each agent's bid utility by the

*un* (*bi*) <sup>=</sup> *<sup>u</sup>* (*bi*)

max*bj*∈*<sup>B</sup> <sup>u</sup>*

Using this normalization mechanism we can avoid the manipulation of the final deal by exaggerating upwards the utility values of the preferred offers. It does not prevent,

 *bj* . (5)

assuming the rest of the agents also tell the truth.

different possibilities regarding utility normalization:

maximum utility value issued by that agent:

probability.

in the previous section. The problem is that, in complex utility spaces, having all agents taking such attitudes could lead to very narrow offers at the mediator, which would make deal probability (given by Equation 1) decrease drastically. This may lead the protocol to negotiation failures, with zero social welfare, thus resulting in situations of infinite Expected Price of Anarchy (Papadimitriou, 2001) , turning the negotiation model unstable.

To improve the strategic stability of the negotiation, the mechanisms should be modified to incentivize the adoption of socially optimal strategies. A feasible way to do that is to modify the social welfare notion at the mediator to reward in the selection of the final solution to those agents which have made wider bids. This can be done by using a generalized or asymmetrical version of the Nash product, similar to the ones used in (Kalai, 1977) to model agents' power of commitment. In particular, we propose a modification of the Nash product which we have called *weighted product by average volume*:

**Definition 3. Weighted product by average volume** *The* weighted product by average volume *of a solution to a negotiation problem among na agents is the product of the utilities the solution gives to every agent i, weighting each utility u<sup>i</sup>* (*s*) *by an adjustment factor equal to the ratio between the average volume of the bids issued by the agent v*¯*<sup>i</sup> and the maximum average volume of the bids of one of the agents:*

$$sw\_{\mathcal{V}}(s, \mathcal{U}) = \prod\_{i=1}^{n\_d} \left(\mathfrak{u}^i(s)\right)^{\frac{\mathfrak{g}^i}{\max\_{1 \le j \le n\_d} \mathfrak{g}^j}}\,\,\,\,\tag{3}$$

*where u<sup>i</sup>* (*s*) *is the utility of the solution s for agent i, and v*¯*<sup>i</sup> is the average volume of the bids issued by agent i.*

In this way, the utility for those agents who have issued widest bids (which, on average, will be the ones using more socially oriented strategies) will be given more weight in the selection of the final solution than those of the more selfish agents. An interesting effect of this metric is that a rational agent could issue some high volume, low utility bids to try to compensate for its high-utility, low volume bids. To counter this effect, we propose to consider bid utility and bid volume jointly, using a *product weighted by average quality factor*:

**Definition 4. Weighted product by average quality factor** *The* weighted product by average quality factor *of a solution to a negotiation problem among na agents is the product of the utilities that solution gives to every agent i, weighting each utility u<sup>i</sup>* (*s*) *by an adjustment factor equal to the ratio between the average quality factor of the bids issued by the agent Q*¯*<sup>i</sup>* <sup>0</sup> *and the maximum average quality factor of the bids of one of the agents:*

$$sw\_{\tilde{Q}\_0}(s, lI) = \prod\_{i=1}^{n\_d} \left(\mu^i(s)\right)^{\frac{Q\_0^{l'}}{\max\_{1 \le j \le \mu\_d Q\_0^{l'}}}} \,, \tag{4}$$

*where Q*¯ 0 *<sup>i</sup> is the average quality factor of the bids issued by agent i.*

In (Lopez-Carmona et al., 2010), experiments were conducted for the different social welfare notions, comparing the utility obtained by a *selfish* agent, using its individually optimal strategy, against the utility obtained by the agent when using the corresponding socially optimal strategy, assuming the rest of the agents are also using that socially optimal strategy. Experiments were made for utility spaces with different correlation lengths. Furthermore, since the model is designed for multi agent negotiations, experiments were performed 14 Will-be-set-by-IN-TECH

in the previous section. The problem is that, in complex utility spaces, having all agents taking such attitudes could lead to very narrow offers at the mediator, which would make deal probability (given by Equation 1) decrease drastically. This may lead the protocol to negotiation failures, with zero social welfare, thus resulting in situations of infinite Expected

To improve the strategic stability of the negotiation, the mechanisms should be modified to incentivize the adoption of socially optimal strategies. A feasible way to do that is to modify the social welfare notion at the mediator to reward in the selection of the final solution to those agents which have made wider bids. This can be done by using a generalized or asymmetrical version of the Nash product, similar to the ones used in (Kalai, 1977) to model agents' power of commitment. In particular, we propose a modification of the Nash product which we have

**Definition 3. Weighted product by average volume** *The* weighted product by average volume *of a solution to a negotiation problem among na agents is the product of the utilities the solution gives to every agent i, weighting each utility u<sup>i</sup>* (*s*) *by an adjustment factor equal to the ratio between the average volume of the bids issued by the agent v*¯*<sup>i</sup> and the maximum average volume of the*

*where u<sup>i</sup>* (*s*) *is the utility of the solution s for agent i, and v*¯*<sup>i</sup> is the average volume of the bids issued by*

In this way, the utility for those agents who have issued widest bids (which, on average, will be the ones using more socially oriented strategies) will be given more weight in the selection of the final solution than those of the more selfish agents. An interesting effect of this metric is that a rational agent could issue some high volume, low utility bids to try to compensate for its high-utility, low volume bids. To counter this effect, we propose to consider bid utility and

**Definition 4. Weighted product by average quality factor** *The* weighted product by average quality factor *of a solution to a negotiation problem among na agents is the product of the utilities that solution gives to every agent i, weighting each utility u<sup>i</sup>* (*s*) *by an adjustment factor equal to the ratio*

> *na* ∏ *i*=1 *u<sup>i</sup>* (*s*)

In (Lopez-Carmona et al., 2010), experiments were conducted for the different social welfare notions, comparing the utility obtained by a *selfish* agent, using its individually optimal strategy, against the utility obtained by the agent when using the corresponding socially optimal strategy, assuming the rest of the agents are also using that socially optimal strategy. Experiments were made for utility spaces with different correlation lengths. Furthermore, since the model is designed for multi agent negotiations, experiments were performed

 *<sup>v</sup>*¯*<sup>i</sup>* max1≤*j*≤*na <sup>v</sup>*¯*<sup>j</sup>*

> *Q*¯ 0 *i* max1≤*j*≤*na <sup>Q</sup>*¯ 0 *j*

, (3)

<sup>0</sup> *and the maximum average quality*

, (4)

*na* ∏ *i*=1 *u<sup>i</sup>* (*s*)

Price of Anarchy (Papadimitriou, 2001) , turning the negotiation model unstable.

*swV*¯(*s*, *U*) =

bid volume jointly, using a *product weighted by average quality factor*:

*between the average quality factor of the bids issued by the agent Q*¯*<sup>i</sup>*

*swQ*¯ 0 (*s*, *U*) =

*<sup>i</sup> is the average quality factor of the bids issued by agent i.*

*factor of the bids of one of the agents:*

called *weighted product by average volume*:

*bids of one of the agents:*

*agent i.*

*where Q*¯ 0 for different number of selfish agents, thus studying the effect of possible coalitions or coincidences. Results showed that the mechanism based on average volume provided not enough improvement in stability. For all cases median utility results were higher for selfish agents, thus maintaining the incentive for agents to deviate from the socially optimal strategy. The mechanism based on average quality factor, however, significantly mitigated the gain for selfish agents, removing the incentive to choose the previously individually optimal strategy.

Another threat to mechanism stability is strategic revelation of information. In incomplete information scenarios (Jonker et al., 2007), since the agents' beliefs about the preferences of a given agent may influence the decision mechanisms they use, an agent may use as a strategy to lie about its own preferences in order to manipulate the decision mechanisms of the rest of the agents to its own benefit. This raises an additional concern to mechanism design (Weiss, 1999). It would be desirable to design protocols which are not prone to be manipulated through insincere revelation of information. *Incentive-compatibility* is defined as the property of a negotiation mechanism which makes telling the truth the best strategy for any agent, assuming the rest of the agents also tell the truth.

In the negotiation model we are dealing with, information revealed to the mediator is the set of agents' bids. These bids represent regions within the solution space. Each offer has an associated utility value, a volume, and an associated quality factor value. Since bid volume is directly related to the region represented by the bid, it does not seem feasible to fake it, since it can be easily checked by the mediator. Quality factor may be faked, but since the mediator is very likely to recompute it, this strategy is also harmless. Finally, agents may fake bid utility. Insincere information revelation about bid utility may generally occur in two ways: exaggerating upward or downward the utility values of *all* bids, or exaggerating the utility values of *some* bids with respect to the others. Exaggerating all bids is not profitable with the proposed deal identification mechanisms, since bid selection at the mediator is performed independently for each agent. This means that the bids from different agents do not compete among each other to be selected as part of a solution. In contrast, the different bids of a single agent compete among themselves. Taking this into account, an agent could try to exaggerate the utility value of its preferred bids, thus trying to increase the probability of the mediator choosing those preferred bids to form deals. As far as social welfare is concerned, this is a problem if the set of exaggerated bids is small with respect to the total set of bids, since that would reduce the number of effective bids considered by the mediator, thus reducing deal probability.

Therefore, we seek for mechanisms which counter this effect, incentivizing sincere revelation of information. A possibility to achieve this is to normalize the utility values assigned by the agents to their bids, thus lowering the absolute differences in utility. We propose three different possibilities regarding utility normalization:

• *Normalization to maximum utility* : obtained by dividing each agent's bid utility by the maximum utility value issued by that agent:

$$
\mu\_{\boldsymbol{\eta}}\left(\boldsymbol{b}\_{\boldsymbol{i}}\right) = \frac{\boldsymbol{u}\left(\boldsymbol{b}\_{\boldsymbol{i}}\right)}{\max\_{\boldsymbol{b}\_{\boldsymbol{j}}\in B} \boldsymbol{u}\left(\boldsymbol{b}\_{\boldsymbol{j}}\right)}.\tag{5}
$$

Using this normalization mechanism we can avoid the manipulation of the final deal by exaggerating upwards the utility values of the preferred offers. It does not prevent,

aggregation, or simply the system may need to implement more sophisticated forms of

<sup>193</sup> Mediated Heuristic Approaches and Alternative Social Welfare Definitions

for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces

We propose to use other mediation rules to improve the negotiation processes where either a quorum is not necessary or simply such quorum is not possible. For example, a solution may be acceptable if *most* of the agents support it. To incorporate these notions into our negotiation framework we will use a more general class of aggregation rules. The idea is to use a *quantifier guided aggregation*, which allows a natural language expression of the quantity of agents that need to agree on an acceptable solution. As we shall see the *Ordered Weighted Averaging* (OWA)

An aggregation operator *<sup>M</sup>* : *<sup>S</sup><sup>n</sup>* <sup>→</sup> *<sup>G</sup>*,(*S*, *<sup>G</sup>* <sup>∈</sup> [0, 1]) is called an OWA operator of dimension *<sup>n</sup>* if it has an associated weighting vector *<sup>W</sup>* = [*w*1*w*<sup>2</sup> ... *wn*] such that *wt* <sup>∈</sup> [0, 1] and <sup>∑</sup>*<sup>n</sup>*

Note that in the definition of OWA we have used the notation *M* to identify the aggregation operator with the mediation rule, *S<sup>n</sup>* to make reference to the preferences of the agents, and *G* to define the group preference. In the OWA aggregation the weights are not directly associated with a particular argument but with the ordered position of the arguments. If *ind* is an index function such that *ind*(*t*) is the index of the *t*th largest argument, then we can express

*<sup>i</sup>* for all *<sup>i</sup>* then *<sup>M</sup>*(*Si*,..., *Sn*) <sup>≥</sup> *<sup>M</sup>*(*S*<sup>ˆ</sup>

Under these conditions the OWA operator is a mean operator. The form of the aggregation is dependent upon the associated weighting vector. We have a number of special cases of weighting vector are worth noting. The vector *W*<sup>∗</sup> defined such that *w*<sup>1</sup> = 1 and *wt* = 0 for all *t* �= 1 gives us the aggregation *Maxi*[*Si*]. Thus, it provides the largest possible aggregation. The vector *W*<sup>∗</sup> defined such that *wn* = 1 and *wt* = 0 for all *t* �= 1 gives the aggregation *Mini*[*Si*]. The weighting vector *Wave* defined such that *wt* = 1/*n* gives us the average

*<sup>i</sup>*=<sup>1</sup> *Si*. Finally, an interesting family of OWA operators are the E-Z OWA operators. There are two families. In the first family we have *wt* = 1/*q* for *t* = 1 to *q*, and *wt* = 0 for *t* = *q* + 1 to *n*. Here we are taking the average of the *q* largest arguments. The other family defines *wt* = 0

In the preceding, we have seen how the OWA operators can be used to compute the group preference for different alternatives. However, our aim is to define consesus policies in the form of a linguistic agenda for our mediation mechanisms. For example, the mediator should make decisions regarding the exploration of the negotiation space, i.e. expansion and

1. Commutativity: The indexing of the arguments is irrelevant

softening of the original *min* and *max* mediation rules by modifying *q*.

4. Boundedness: *Maxi*[*Si*] ≥ *M*(*Si*,..., *Sn*) ≥ *Mini*[*Si*]

contraction of the mesh, following mediation rules like

*<sup>t</sup>*=<sup>1</sup> *wtbt* where *bt* is the *t*th largest element of the aggregates

*<sup>i</sup>*,..., *S*ˆ *n*)

*<sup>t</sup>*=<sup>1</sup> *wtSind*(*t*). It can be shown that OWA aggregation has the following

*<sup>n</sup>*−*<sup>q</sup>* for *<sup>t</sup>* <sup>=</sup> *<sup>q</sup>* <sup>+</sup> 1 to *<sup>n</sup>*. We can see that this operator can provide a

*<sup>t</sup>*=<sup>1</sup> *wt* =

operator (Yager, 1996) will provide a tool to model this kind of softer mediation rule.

aggregation.

**3.1 OWA operators**

*M*(*S*1,..., *Sn*) = ∑*<sup>n</sup>*

2. Monotonicity: If *Si* <sup>≥</sup> *<sup>S</sup>*<sup>ˆ</sup>

for *t* = 1 to *q*, and *wt* = <sup>1</sup>

**3.2 Quantifier guided aggregation**

3. Idempotency: *M*(*S*,..., *S*) = *S*

{*S*1,..., *Sn*}.

properties:

1 *<sup>n</sup>* <sup>∑</sup>*<sup>n</sup>*

1 and where *M*(*S*1,..., *Sn*) = ∑*<sup>n</sup>*

however, downward exaggerations, that is, to assign an extremely low value to the bids which are less profitable for the agent.

• *Bounded maximum-minimum normalization*: Attempts to prevent the manipulation of the negotiation model through upwards or backwards exaggerations. It is given by the expression

$$
\mu\_n \left( b\_i \right) = \mathfrak{u}'\_{\min} + \frac{\mathfrak{u} \left( b\_i \right) - \mathfrak{u}\_{\min}}{\mathfrak{u}\_{\max} - \mathfrak{u}\_{\min}} \left( \mathfrak{u}'\_{\max} - \mathfrak{u}'\_{\min} \right),
\tag{6}
$$

where *umax* = max*bj*∈*<sup>B</sup> <sup>u</sup> bj* , *umin* = min*bj*∈*<sup>B</sup> <sup>u</sup> bj* and *u*� *min* and *u*� *max* are parameters chosen by the mediator. In this way, a utility mapping from the interval [*umin*, *umax*] to the interval *u*� *min*, *u*� *max* is performed for all bids, putting an upper bound *<sup>u</sup>*� *max u*� *min* to the ratio between the utilities of an agent's bids.

• *Ordinal normalization:* obtained by ordering the different bids of an agent according to their utility or quality factor, and mapping this order to a monotonically increasing succession of utility values, regardless of the original utility values. For instance, if *B* is the set of bids for an agent, in ascendent order of utility, and taking the arithmetic succession *s* = {1, 2, . . . , *nB*} as the mapping function, the normalized bid utility values would be of the form

$$
u\_n \left( b\_i \right) = s\_i = i.\\
$$

To evaluate the effect of the described mechanisms over the incentive-compatibility of the model, in (Lopez-Carmona et al., 2010) we also conducted experiments comparing the utility obtained by an *insincere agent* with the utility obtained being sincere, assuming the rest of the agents are sincere. The behavior of the insincere agent was modeled by exaggerating the utility of a portion of the agent's highest utility bids. We considered different degrees of exaggeration for the insincere agent: the insincere agent exaggerated 75%, 50%, 25% and 12.5% of its bids. Results showed that all proposed normalization techniques reduced the incentive for the insincere agent to exaggerate, but only bounded maximum-minimum normalization made the expected gain for the insincere agent negligible, thus effectively removing the incentive to exaggerate, improving incentive-compatibility of the model.

#### **3. Using social welfare to incentivize different consensus policies**

So far we have seen how social welfare notions may be used to incentivize different kind of behaviors from the negotiating agents. However, social welfare notions can also be used to directly influence the negotiation outcome, looking for specific kinds of agreements. In (Hoz et al., 2011), a consensus policy mediated framework (CPMF) is defined for negotiations with complex utility spaces. The negotiation framework is based on the use of a mediator, which iterates through a series of contracts trying to optimize social welfare throughout the use of Mesh-adaptive Generalized Pattern Search (Lewis et al., 2000) and an iterative, expressive protocol which gathers the different negotiating agents valuations for the contracts in the mesh. This framework is specially interesting for the scope of this chapter, because the framework explicitly allows the mediator to pursue optimization objectives different than classic social welfare, thus allowing for the enforcement of different consensus policies.

The most widespread consensus policy found in the automated negotiation literature is social welfare maximization, which suggests desire to satisfy *all* the agents. However, the requirement to satisfy all the agents may not be suitable for multi-agent preference aggregation, or simply the system may need to implement more sophisticated forms of aggregation.

We propose to use other mediation rules to improve the negotiation processes where either a quorum is not necessary or simply such quorum is not possible. For example, a solution may be acceptable if *most* of the agents support it. To incorporate these notions into our negotiation framework we will use a more general class of aggregation rules. The idea is to use a *quantifier guided aggregation*, which allows a natural language expression of the quantity of agents that need to agree on an acceptable solution. As we shall see the *Ordered Weighted Averaging* (OWA) operator (Yager, 1996) will provide a tool to model this kind of softer mediation rule.

#### **3.1 OWA operators**

16 Will-be-set-by-IN-TECH

• *Bounded maximum-minimum normalization*: Attempts to prevent the manipulation of the negotiation model through upwards or backwards exaggerations. It is given by the

which are less profitable for the agent.

between the utilities of an agent's bids.

*un* (*bi*) = *u*�

 *bj*  *min* +

, *umin* = min*bj*∈*<sup>B</sup> <sup>u</sup>*

expression

interval

form

where *umax* = max*bj*∈*<sup>B</sup> <sup>u</sup>*

*u*� *min*, *u*� *max* 

however, downward exaggerations, that is, to assign an extremely low value to the bids

*u* (*bi*) − *umin umax* − *umin*

chosen by the mediator. In this way, a utility mapping from the interval [*umin*, *umax*] to the

• *Ordinal normalization:* obtained by ordering the different bids of an agent according to their utility or quality factor, and mapping this order to a monotonically increasing succession of utility values, regardless of the original utility values. For instance, if *B* is the set of bids for an agent, in ascendent order of utility, and taking the arithmetic succession *s* = {1, 2, . . . , *nB*} as the mapping function, the normalized bid utility values would be of the

*un* (*bi*) = *si* = *i*.

To evaluate the effect of the described mechanisms over the incentive-compatibility of the model, in (Lopez-Carmona et al., 2010) we also conducted experiments comparing the utility obtained by an *insincere agent* with the utility obtained being sincere, assuming the rest of the agents are sincere. The behavior of the insincere agent was modeled by exaggerating the utility of a portion of the agent's highest utility bids. We considered different degrees of exaggeration for the insincere agent: the insincere agent exaggerated 75%, 50%, 25% and 12.5% of its bids. Results showed that all proposed normalization techniques reduced the incentive for the insincere agent to exaggerate, but only bounded maximum-minimum normalization made the expected gain for the insincere agent negligible, thus effectively removing the

So far we have seen how social welfare notions may be used to incentivize different kind of behaviors from the negotiating agents. However, social welfare notions can also be used to directly influence the negotiation outcome, looking for specific kinds of agreements. In (Hoz et al., 2011), a consensus policy mediated framework (CPMF) is defined for negotiations with complex utility spaces. The negotiation framework is based on the use of a mediator, which iterates through a series of contracts trying to optimize social welfare throughout the use of Mesh-adaptive Generalized Pattern Search (Lewis et al., 2000) and an iterative, expressive protocol which gathers the different negotiating agents valuations for the contracts in the mesh. This framework is specially interesting for the scope of this chapter, because the framework explicitly allows the mediator to pursue optimization objectives different than classic social welfare, thus allowing for the enforcement of different consensus policies.

The most widespread consensus policy found in the automated negotiation literature is social welfare maximization, which suggests desire to satisfy *all* the agents. However, the requirement to satisfy all the agents may not be suitable for multi-agent preference

incentive to exaggerate, improving incentive-compatibility of the model.

**3. Using social welfare to incentivize different consensus policies**

 *u*� *max* − *u*�

 *bj* and *u*�

is performed for all bids, putting an upper bound *<sup>u</sup>*�

*min* 

*min* and *u*�

, (6)

*max u*� *min*

*max* are parameters

to the ratio

An aggregation operator *<sup>M</sup>* : *<sup>S</sup><sup>n</sup>* <sup>→</sup> *<sup>G</sup>*,(*S*, *<sup>G</sup>* <sup>∈</sup> [0, 1]) is called an OWA operator of dimension *<sup>n</sup>* if it has an associated weighting vector *<sup>W</sup>* = [*w*1*w*<sup>2</sup> ... *wn*] such that *wt* <sup>∈</sup> [0, 1] and <sup>∑</sup>*<sup>n</sup> <sup>t</sup>*=<sup>1</sup> *wt* = 1 and where *M*(*S*1,..., *Sn*) = ∑*<sup>n</sup> <sup>t</sup>*=<sup>1</sup> *wtbt* where *bt* is the *t*th largest element of the aggregates {*S*1,..., *Sn*}.

Note that in the definition of OWA we have used the notation *M* to identify the aggregation operator with the mediation rule, *S<sup>n</sup>* to make reference to the preferences of the agents, and *G* to define the group preference. In the OWA aggregation the weights are not directly associated with a particular argument but with the ordered position of the arguments. If *ind* is an index function such that *ind*(*t*) is the index of the *t*th largest argument, then we can express *M*(*S*1,..., *Sn*) = ∑*<sup>n</sup> <sup>t</sup>*=<sup>1</sup> *wtSind*(*t*). It can be shown that OWA aggregation has the following properties:


Under these conditions the OWA operator is a mean operator. The form of the aggregation is dependent upon the associated weighting vector. We have a number of special cases of weighting vector are worth noting. The vector *W*<sup>∗</sup> defined such that *w*<sup>1</sup> = 1 and *wt* = 0 for all *t* �= 1 gives us the aggregation *Maxi*[*Si*]. Thus, it provides the largest possible aggregation. The vector *W*<sup>∗</sup> defined such that *wn* = 1 and *wt* = 0 for all *t* �= 1 gives the aggregation *Mini*[*Si*]. The weighting vector *Wave* defined such that *wt* = 1/*n* gives us the average 1 *<sup>n</sup>* <sup>∑</sup>*<sup>n</sup> <sup>i</sup>*=<sup>1</sup> *Si*. Finally, an interesting family of OWA operators are the E-Z OWA operators. There are two families. In the first family we have *wt* = 1/*q* for *t* = 1 to *q*, and *wt* = 0 for *t* = *q* + 1 to *n*. Here we are taking the average of the *q* largest arguments. The other family defines *wt* = 0 for *t* = 1 to *q*, and *wt* = <sup>1</sup> *<sup>n</sup>*−*<sup>q</sup>* for *<sup>t</sup>* <sup>=</sup> *<sup>q</sup>* <sup>+</sup> 1 to *<sup>n</sup>*. We can see that this operator can provide a softening of the original *min* and *max* mediation rules by modifying *q*.

#### **3.2 Quantifier guided aggregation**

In the preceding, we have seen how the OWA operators can be used to compute the group preference for different alternatives. However, our aim is to define consesus policies in the form of a linguistic agenda for our mediation mechanisms. For example, the mediator should make decisions regarding the exploration of the negotiation space, i.e. expansion and contraction of the mesh, following mediation rules like

<sup>0</sup> 1/5 2/5 3/5 4/5 <sup>1</sup> <sup>0</sup>

<sup>195</sup> Mediated Heuristic Approaches and Alternative Social Welfare Definitions

w1 <sup>y</sup>

0 1

ALL ANY At least α

0 1

QZ β

<sup>y</sup> Linear

0 1

y

0 1

α

0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1

<sup>β</sup> <sup>α</sup> QZ

i/5

Fig. 3. Example of how to obtain the weights from a quantifier for *n* = 5 agents.

0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1

Fig. 4. Functional form of typical quantifiers: all, any, at least, linear, piecewise linear *QZ<sup>β</sup>*

1

w5

for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces

w4

w3 w2

Q(y)

0 1

0 1

0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1

and piecewise linear *QZ<sup>α</sup>* .

Q(y)

Q(y)

*Most* agents must be satisfied by the contract, *at least α* agents must be satisfied by the contract, *many* agents must be satisfied, . . .

The above statements are examples of *quantifier guided aggregations*. (Zadeh, 1983) suggested a formal representation of these linguistic quantifiers using fuzzy sets. He suggested that any relative linguistic quantifier can be expressed as a fuzzy subset *Q* of the unit interval *I* = [0, 1]. In this representation for any proportion *y* ∈ *I*, *Q*(*y*) indicates the degree to which *y* satisfies the concept expressed by the term *Q*. In most applications of the quantifier guided aggregation we use a special case class of these liguistic quantifiers, called *Regular Increasing Monotone* (RIM) quantifiers. These types of quantifiers have the property that as more agents are satisfied our overall satisfaction can't decrease. Formally, these quantifiers are characterized in the following way: 1) *Q*(0) = 0, 2) *Q*(1) = 1 and 3) *Q*(*x*) ≥ *Q*(*y*) if *x* > *y*. Examples of this kind of quantifier are *all*, *most*, *many*, *at least α*. Two examples of RIM quantifiers are *all* which is represented by *Q*<sup>∗</sup> where *Q*∗(1) = 1 and *Q*∗(*x*) = 0 for all *x* �= 1, and *any* which is defined as *Q*∗(0) = 0 and *Q*∗(*x*) = 1 for all *x* �= 0.

The question now is how to obtain the OWA operator to satisfy a quantifier guided aggregation. Again assume we have a collection of *n* agents. These agents have their preferences represented as fuzzy subsets over the set of alternatives in the mesh {*S*1(*x*+*o*(*k*)),..., *Sn*(*x*+*o*(*k*))}. Under the quantifier guided mediation approach a group mediation protocol is expressed in terms of a linguistic quantifier *Q* indicating the proportion of agents whose agreement if necessary for a solution to be acceptable. The basic form of the mediation rule in this approach is that *Q* agents must be satisfied by the contract, where *Q* is a quantifier.

The formal procedure used to implement this mediation rule is described in the following. The quantifier *Q* is used to generate an OWA weighting vector *W* of dimension *n*. This weighting vector is then used in an OWA aggregation to determine the group support for the contract. For each contract in the mesh the argument of this OWA aggregation is the degree of support for that contract by each of the agents, *Si*(*xej*(*k*)), *i* = 1, . . . , *n*. Thus, the process used in the quantifier guided aggregation is as follows:


The procedure used for generating the weights from the quantifier is to divide the unit interval into *n* equally spaced intervals and then to compute the length of the mapped intervals using *Q*

$$w\_t = Q(\frac{t}{n}) - Q(\frac{t-1}{n}) \text{ for } t = 1, \dots, n \dots$$

Because of the nondecreasing nature of *Q* it follows that *wt* ≥ 0. Furthermore from the regularity of *Q*, *Q*(1) = 1 and *Q*(0) = 0, it follows that ∑*<sup>t</sup> wt* = 1. Thus we can see that the weights generated are an acceptable class of OWA weights.

In Figure 3 we show an example of a RIM linguistic quantifier and illustrate the process of determining the weights from the quantifier. We see that the weights depend on the number of agents as well as the form of *Q*. In Figure 4 we show the functional form for the quantifiers *all*, *any*, *Q*∗, *Q*∗, *at least α percent*, *linear quantifier*, *piecewise QZ<sup>β</sup>* and *piecewise QZ<sup>α</sup>* .

18 Will-be-set-by-IN-TECH

*Most* agents must be satisfied by the contract, *at least α* agents must be satisfied by the contract,

The above statements are examples of *quantifier guided aggregations*. (Zadeh, 1983) suggested a formal representation of these linguistic quantifiers using fuzzy sets. He suggested that any relative linguistic quantifier can be expressed as a fuzzy subset *Q* of the unit interval *I* = [0, 1]. In this representation for any proportion *y* ∈ *I*, *Q*(*y*) indicates the degree to which *y* satisfies the concept expressed by the term *Q*. In most applications of the quantifier guided aggregation we use a special case class of these liguistic quantifiers, called *Regular Increasing Monotone* (RIM) quantifiers. These types of quantifiers have the property that as more agents are satisfied our overall satisfaction can't decrease. Formally, these quantifiers are characterized in the following way: 1) *Q*(0) = 0, 2) *Q*(1) = 1 and 3) *Q*(*x*) ≥ *Q*(*y*) if *x* > *y*. Examples of this kind of quantifier are *all*, *most*, *many*, *at least α*. Two examples of RIM quantifiers are *all* which is represented by *Q*<sup>∗</sup> where *Q*∗(1) = 1 and *Q*∗(*x*) = 0 for all *x* �= 1,

The question now is how to obtain the OWA operator to satisfy a quantifier guided aggregation. Again assume we have a collection of *n* agents. These agents have their preferences represented as fuzzy subsets over the set of alternatives in the mesh {*S*1(*x*+*o*(*k*)),..., *Sn*(*x*+*o*(*k*))}. Under the quantifier guided mediation approach a group mediation protocol is expressed in terms of a linguistic quantifier *Q* indicating the proportion of agents whose agreement if necessary for a solution to be acceptable. The basic form of the mediation rule in this approach is that *Q* agents must be satisfied by the contract, where *Q* is

The formal procedure used to implement this mediation rule is described in the following. The quantifier *Q* is used to generate an OWA weighting vector *W* of dimension *n*. This weighting vector is then used in an OWA aggregation to determine the group support for the contract. For each contract in the mesh the argument of this OWA aggregation is the degree of support for that contract by each of the agents, *Si*(*xej*(*k*)), *i* = 1, . . . , *n*. Thus, the process used in the

2. For each contract *xej*(*k*) in *x*+*o*(*k*) calculate the overall group support *G*(*xej*(*k*)) =

The procedure used for generating the weights from the quantifier is to divide the unit interval into *n* equally spaced intervals and then to compute the length of the mapped intervals using

*t* − 1

Because of the nondecreasing nature of *Q* it follows that *wt* ≥ 0. Furthermore from the regularity of *Q*, *Q*(1) = 1 and *Q*(0) = 0, it follows that ∑*<sup>t</sup> wt* = 1. Thus we can see that

In Figure 3 we show an example of a RIM linguistic quantifier and illustrate the process of determining the weights from the quantifier. We see that the weights depend on the number of agents as well as the form of *Q*. In Figure 4 we show the functional form for the quantifiers

*all*, *any*, *Q*∗, *Q*∗, *at least α percent*, *linear quantifier*, *piecewise QZ<sup>β</sup>* and *piecewise QZ<sup>α</sup>* .

*<sup>n</sup>* )for*<sup>t</sup>* <sup>=</sup> 1, . . . , *<sup>n</sup>* .

and *any* which is defined as *Q*∗(0) = 0 and *Q*∗(*x*) = 1 for all *x* �= 0.

*many* agents must be satisfied, . . .

a quantifier.

*Q*

quantifier guided aggregation is as follows:

*M*(*S*1(*xej*(*k*)),..., *Sn*(*xej*(*k*))).

1. Use *Q* to generate a set of OWA weights, *w*1,..., *wn*.

*wt* = *Q*(

the weights generated are an acceptable class of OWA weights.

*t <sup>n</sup>* ) <sup>−</sup> *<sup>Q</sup>*(

Fig. 3. Example of how to obtain the weights from a quantifier for *n* = 5 agents.

Fig. 4. Functional form of typical quantifiers: all, any, at least, linear, piecewise linear *QZ<sup>β</sup>* and piecewise linear *QZ<sup>α</sup>* .

**4. Concluding remarks**

and incentive compatibility.

situations where unanimous agreements are not viable.

In this chapter we have studied how different notions of social welfare may be used to influence the outcome of mediated negotiations. First, we have presented a model for automated negotiation with complex, constraint based utility spaces, where using social welfare notions which take into account both bid utility and deal probability provide an improvement of the performance and scalability of the negotiation protocol as the complexity of the utility space increases. Results show that there is a significant advantage in using the quality factor *Q*<sup>0</sup> and the proposed mechanisms in highly uncorrelated utility spaces, both in terms of optimality and failure rate. We have also seen how social welfare notions can address strategic issues within our model, allowing to address important problems such as stability

<sup>197</sup> Mediated Heuristic Approaches and Alternative Social Welfare Definitions

for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces

The model definition has been divided into three parts: the agent preference model, the interaction protocol and the decision mechanisms. To describe the agent preference model we have chosen a representation based on weighted constraints, since this allows to easily generate utility spaces of adjustable complexity, which is an advantage for the evaluation of the proposed mechanisms. Regarding the interaction protocol, an auction-based, expressive, iterative approach has been chosen. The use of the mediator allows to reduce information revelation between negotiating agents, and also facilitates to separate the individual agent strategies (which for rational agents tend to be maximize their own profit) from the social strategies. The interaction protocol is closely related to the one-shot, auction-based protocol described in (Ito et al., 2007), with the difference that our model considers the possibility to perform multiple iterations and provides the mediator with expressive capabilities to send feedback to the agents at each iteration, in order to refine their bids. There are other works in the literature suggesting the use of expressive negotiation protocols in multi-agent negotiations. (Li et al., 2009b; Lopez-Carmona et al., 2010; Marsa-Maestre et al., 2008) use gradient information to bias the search for solutions in linear unmediated negotiation, and (Lopez-Carmona et al., 2007) uses relax requirements in bilateral buyer-seller negotiations. Finally, we have explored the hypothesis that the consensus type by which an agreement meets in some specific manner the concerns of all the negotiators should be considered in the construction of multiparty negotiation protocols. We argue that there exist situations where an unanimous agreement is not possible or simply the rules imposed by the system may not seek such unanimous agreement. Thus, we have used a particular aggregation of the agent preferences by the mediator, which allows to define different social welfare notions (which we call consensus policies) based on OWA and quantifier guided aggregation. Results show that CPMF efficiently manages negotiations following predefined consensus policies and solves

Though the conducted experiments have yielded satisfactory results, there is still plenty of research to be done in this area. We believe that the results presented opens the door to a new set of negotiation policies where social welfare notions, be it as behavior adherence metrics or as consensus policies, will play an important role. We are also interested in dynamic adjustments for the social welfare notions, so that at different steps in the negotiation the consensus policies may be adapted to the specific negotiation dynamics. Finally, the challenge of enforcing alternative social welfare notions without the aid of a mediator still remains open.

The quantifiers *all*, *any* and *at least α* describe the consensus policy using a natural language verbal description. However, more generally any function *Q* : [0, 1] → [0, 1] such that *Q*(*x*) ≥ *Q*(*y*) for *x* ≥ *y*, *Q*(1) = 1 and *Q*(0) = 0 can be seen to be an appropriate form for generating mediation rules or consensus policies. Thus there are two techniques to generating these quantifier based mediation rules. One possibility is to start with a linguistic expression and then obtain *Q*. The second approach is to allow the mediation rule to be directly expressed in terms of a function *Q*. One important characteristic of this second method is that we can easily introduce into our mediation a number of formal properties that are not very easily expressed using a verbal description of the quantifier. The linear quantifier *Q*(*y*) = *y* for instance generates *wt* = 1/*n*, and thus, all the agents get the same weight. The *QZ<sup>β</sup>* quantifier it is required that at least *β* agents are satisfied to initiate a *Q* linear improvement. *QZ<sup>α</sup>* initiates the *Q* linear improvement with the first satisfied agent, and once there are *α* agents satisfied there is no improvement in *Q* if more agents are satisfied.

One feature which distinguishes the different types of mediation rules is the power of an individual agent to eliminate an alternative. For example, in the case of *all* this power is complete. In order to capture this idea the *Value Of Individual Disapproval* (VOID)

$$VOID(Q) = 1 - \int\_0^1 Q(y)dy$$

measures this power. For the *all*, *any, at least α* and *linear* quantifiers the VOID measures are respectively 1, 0, *<sup>α</sup>* and 0.5. For the *QZ<sup>β</sup>* quantifier *VOID*(*QZ<sup>β</sup>* ) = <sup>1</sup> <sup>2</sup> <sup>+</sup> *<sup>β</sup>* <sup>2</sup> ) and therefore *VOID*(*QZ<sup>β</sup>* ) <sup>∈</sup> [0.5, 1]. The *QZ<sup>α</sup>* quantifier gets *VOID*(*QZ<sup>α</sup>* ) = *<sup>α</sup>* <sup>2</sup> and *VOID*(*QZ<sup>α</sup>* ) ∈ [0, 0.5].

Another family of quantifiers are those defined by *Qp*(*y*) = *y<sup>p</sup>* for *p* > 0. In this case *VOID*(*Qp*) = <sup>1</sup> <sup>−</sup> <sup>1</sup> <sup>0</sup> *<sup>r</sup>pdr* <sup>=</sup> *<sup>p</sup> <sup>p</sup>*+<sup>1</sup> . For this quantifier we can easily obtain the OWA weights with

$$w\_t = \left(\frac{t}{n}\right)^p - \left(\frac{t-1}{n}\right)^p \dots$$

For *Qp* we see that as *p* increases we get closer to the *min* and that as *p* gets closer to zero we get the *max*.

#### **3.3 Impact of consensus policies on automated negotiations**

In (Hoz et al., 2011), we performed experiments to test the performance of the protocol under a proof of concept scenario and a complex negotiation scenario for 5 different consensus policies defined by the corresponding VOID degrees: 0, 0.25, 0.5, 0.75 and 0.95, using the quantifier *Qp*(*y*) = *yp*. We also defined a contrast experiment where the consensus policy based mediation process is deactivated, such that the group preference evaluation is limited to compute the sum of agents' valuations for a given contract (i.e. the winner contract is that with the highest sum of valuations), which corresponds to classic social welfare. Experiments showed that social welfare maximization tends to achieve poorer agreements in complex utility spaces, while consensus policies which try to satisfy only a subset of the agents achieve higher satisfaction degrees for the agents. Using different consensus policies may allow, for instance, to perform simultaneous partial agreements among subsets of the negotiating agents, which may be of key importance in some scenarios, like group formation negotiations.

#### **4. Concluding remarks**

20 Will-be-set-by-IN-TECH

The quantifiers *all*, *any* and *at least α* describe the consensus policy using a natural language verbal description. However, more generally any function *Q* : [0, 1] → [0, 1] such that *Q*(*x*) ≥ *Q*(*y*) for *x* ≥ *y*, *Q*(1) = 1 and *Q*(0) = 0 can be seen to be an appropriate form for generating mediation rules or consensus policies. Thus there are two techniques to generating these quantifier based mediation rules. One possibility is to start with a linguistic expression and then obtain *Q*. The second approach is to allow the mediation rule to be directly expressed in terms of a function *Q*. One important characteristic of this second method is that we can easily introduce into our mediation a number of formal properties that are not very easily expressed using a verbal description of the quantifier. The linear quantifier *Q*(*y*) = *y* for instance generates *wt* = 1/*n*, and thus, all the agents get the same weight. The *QZ<sup>β</sup>* quantifier it is required that at least *β* agents are satisfied to initiate a *Q* linear improvement. *QZ<sup>α</sup>* initiates the *Q* linear improvement with the first satisfied agent, and once there are *α* agents satisfied

One feature which distinguishes the different types of mediation rules is the power of an individual agent to eliminate an alternative. For example, in the case of *all* this power is

measures this power. For the *all*, *any, at least α* and *linear* quantifiers the VOID measures

Another family of quantifiers are those defined by *Qp*(*y*) = *y<sup>p</sup>* for *p* > 0. In this case

For *Qp* we see that as *p* increases we get closer to the *min* and that as *p* gets closer to zero we

In (Hoz et al., 2011), we performed experiments to test the performance of the protocol under a proof of concept scenario and a complex negotiation scenario for 5 different consensus policies defined by the corresponding VOID degrees: 0, 0.25, 0.5, 0.75 and 0.95, using the quantifier *Qp*(*y*) = *yp*. We also defined a contrast experiment where the consensus policy based mediation process is deactivated, such that the group preference evaluation is limited to compute the sum of agents' valuations for a given contract (i.e. the winner contract is that with the highest sum of valuations), which corresponds to classic social welfare. Experiments showed that social welfare maximization tends to achieve poorer agreements in complex utility spaces, while consensus policies which try to satisfy only a subset of the agents achieve higher satisfaction degrees for the agents. Using different consensus policies may allow, for instance, to perform simultaneous partial agreements among subsets of the negotiating agents, which may be of key importance in some scenarios, like group formation negotiations.

 1 0

 *<sup>t</sup>* <sup>−</sup> <sup>1</sup> *n*

*<sup>p</sup>* .

*Q*(*y*)*dy*

*<sup>p</sup>*+<sup>1</sup> . For this quantifier we can easily obtain the OWA weights

<sup>2</sup> <sup>+</sup> *<sup>β</sup>*

<sup>2</sup> and *VOID*(*QZ<sup>α</sup>* ) ∈ [0, 0.5].

<sup>2</sup> ) and therefore

complete. In order to capture this idea the *Value Of Individual Disapproval* (VOID)

*VOID*(*Q*) = 1 −

are respectively 1, 0, *<sup>α</sup>* and 0.5. For the *QZ<sup>β</sup>* quantifier *VOID*(*QZ<sup>β</sup>* ) = <sup>1</sup>

*wt* =

 *t n <sup>p</sup>* −

*VOID*(*QZ<sup>β</sup>* ) <sup>∈</sup> [0.5, 1]. The *QZ<sup>α</sup>* quantifier gets *VOID*(*QZ<sup>α</sup>* ) = *<sup>α</sup>*

**3.3 Impact of consensus policies on automated negotiations**

<sup>0</sup> *<sup>r</sup>pdr* <sup>=</sup> *<sup>p</sup>*

*VOID*(*Qp*) = <sup>1</sup> <sup>−</sup> <sup>1</sup>

with

get the *max*.

there is no improvement in *Q* if more agents are satisfied.

In this chapter we have studied how different notions of social welfare may be used to influence the outcome of mediated negotiations. First, we have presented a model for automated negotiation with complex, constraint based utility spaces, where using social welfare notions which take into account both bid utility and deal probability provide an improvement of the performance and scalability of the negotiation protocol as the complexity of the utility space increases. Results show that there is a significant advantage in using the quality factor *Q*<sup>0</sup> and the proposed mechanisms in highly uncorrelated utility spaces, both in terms of optimality and failure rate. We have also seen how social welfare notions can address strategic issues within our model, allowing to address important problems such as stability and incentive compatibility.

The model definition has been divided into three parts: the agent preference model, the interaction protocol and the decision mechanisms. To describe the agent preference model we have chosen a representation based on weighted constraints, since this allows to easily generate utility spaces of adjustable complexity, which is an advantage for the evaluation of the proposed mechanisms. Regarding the interaction protocol, an auction-based, expressive, iterative approach has been chosen. The use of the mediator allows to reduce information revelation between negotiating agents, and also facilitates to separate the individual agent strategies (which for rational agents tend to be maximize their own profit) from the social strategies. The interaction protocol is closely related to the one-shot, auction-based protocol described in (Ito et al., 2007), with the difference that our model considers the possibility to perform multiple iterations and provides the mediator with expressive capabilities to send feedback to the agents at each iteration, in order to refine their bids. There are other works in the literature suggesting the use of expressive negotiation protocols in multi-agent negotiations. (Li et al., 2009b; Lopez-Carmona et al., 2010; Marsa-Maestre et al., 2008) use gradient information to bias the search for solutions in linear unmediated negotiation, and (Lopez-Carmona et al., 2007) uses relax requirements in bilateral buyer-seller negotiations.

Finally, we have explored the hypothesis that the consensus type by which an agreement meets in some specific manner the concerns of all the negotiators should be considered in the construction of multiparty negotiation protocols. We argue that there exist situations where an unanimous agreement is not possible or simply the rules imposed by the system may not seek such unanimous agreement. Thus, we have used a particular aggregation of the agent preferences by the mediator, which allows to define different social welfare notions (which we call consensus policies) based on OWA and quantifier guided aggregation. Results show that CPMF efficiently manages negotiations following predefined consensus policies and solves situations where unanimous agreements are not viable.

Though the conducted experiments have yielded satisfactory results, there is still plenty of research to be done in this area. We believe that the results presented opens the door to a new set of negotiation policies where social welfare notions, be it as behavior adherence metrics or as consensus policies, will play an important role. We are also interested in dynamic adjustments for the social welfare notions, so that at different steps in the negotiation the consensus policies may be adapted to the specific negotiation dynamics. Finally, the challenge of enforcing alternative social welfare notions without the aid of a mediator still remains open.

Heiskanen, P., Ehtamo, H. & Hamalainen, R. P. (2001). Constraint proposal method

<sup>199</sup> Mediated Heuristic Approaches and Alternative Social Welfare Definitions

Hindriks, K., Jonker, C. & Tykhonov, D. (2006). Eliminating interdependencies between issues

Hindriks, K. V., Jonker, C. M. & Tykhonov, D. (2009). A multi-agent environment for

Hitchcock, D., McBurney, P. & Parsons, S. (2001). A framework for deliberation dialogues,

Horn, J. & Goldberg, D. E. (1994). Genetic algorithm difficulty and the modality of fitness landscapes, *Foundations of Genetic Algorithms 3*, Morgan Kaufmann, pp. 243–269. Hoz, E. de la, Lopez-Carmona, M. A., Klein, M. & Marsa-Maestre, I. (2011). Consensus

Hunsberger, L. & Grosz, B. J. (2000). A combinatorial auction for collaborative planning,

Ito, T., Klein, M. & Hattori, H. (2007). Multi-issue negotiation protocol for agents: Exploring

Ito, T., Klein, M. & Hattori, H. (2008). A multi-issue negotiation protocol among agents with nonlinear utility functions, *Journal of Multiagent and Grid Systems* 4(1): 67–83. Jennings, N. R. (2001). An agent-based approach for building complex software systems,

Jonker, C., Robu, V. & Treur, J. (2007). An agent architecture for multi-attribute negotiation

Kalai, E. (1977). Nonsymmetric nash solutions and replications of 2-person bargaining,

Kersten, G. E. & Noronha, S. J. (1998). Rational agents, contract curves, and

Klein, M., Faratin, P. & Bar-Yam, Y. (2002). Using an annealing mediator to solve the

Klein, M., Faratin, P., Sayama, H. & Bar-Yam, Y. (2003). Protocols for negotiating complex

Kraus, S., Sycara, K. & Evenchick, A. (1998). Reaching agreements through argumentation: A

logical model and implementation, *Artificial Intelligence* 1-2: 1–69.

Hordijk, W. (1995). A measure of landscapes, *Working Papers 95-05-049*, Santa Fe Institute.

*on Principles and Practice of Multi Agents (PRIMA 2011)*, pp. 159–173.

*Operational Research* 133(1): 44–61.

*in Computer Science*, Springer Berlin, pp. 301–316.

*Argumentation (OSSA-2001)*, Windsor, Canada.

*Multi-Agent Programming:*, Springer US, pp. 333–363.

for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces

URL: *http://ideas.repec.org/p/wop/safiwp/95-05-049.html*

*Multi-Agent Systems, International Conference on* 0: 0151.

*Intelligence (IJCAI07)*, pp. 1347–1352.

*Communications of the ACM* 44(4): 35–41.

URL: *http://dx.doi.org/10.1007/BF01774658*

28(3): 326–338.

UK, pp. 194–202.

*Systems* 15: 221–252. 10.1007/s10458-006-9009-y. URL: *http://dx.doi.org/10.1007/s10458-006-9009-y*

*International Journal of Game Theory* 6(3): 129–133.

contracts, *IEEE Intelligent Systems* 18(6): 32–38.

for computing pareto solutions in multi-party negotiations, *European Journal of*

for multi-issue negotiation, *Cooperative Information Agents X*, Vol. 4149 of *Lecture Notes*

negotiation, *in* A. El Fallah Seghrouchni, J. Dix, M. Dastani & R. H. Bordini (eds),

*in* H. V. Hansen (ed.), *Proceedings of the 4th Conference Ontario Society Study of*

policy based multi-agent negotiation, *Proceedings of the 14th International Conference*

nonlinear utility spaces, *Proceedings of the 20th International Joint Conference on Artificial*

using incomplete preference information, *Autonomous Agents and Multi-Agent*

inefficient compromises, *IEEE Transactions on Systems, Man, and Cybernetics, Part A*

prisoners' dilemma in the negotiation of complex contracts, *AAMAS '02: Revised Papers from the Workshop on Agent Mediated Electronic Commerce on Agent-Mediated Electronic Commerce IV, Designing Mechanisms and Systems*, Springer-Verlag, London,

#### **5. Acknowledgments**

This work has been supported by the Spanish Ministry of Education and Science grants: TIN2008-06739-C04-04, T2C2 research project, and the mobility grant JC2010-003.

#### **6. References**


22 Will-be-set-by-IN-TECH

This work has been supported by the Spanish Ministry of Education and Science grants:

Amgoud, L., Maudet, N. & Parsons, S. (2000). Modelling dialogues using argumentation, *in*

Beer, M., D'Inverno, M., Luck, M., Jennings, N., Preist, C. & Schroeder, M. (1999). Negotiation in multi-agent systems, *Knowledge Engineering Review* 14(3): 285–290. Bichler, M., Kaukal, M. & Segev, A. (1999). Multi-attribute auctions for electronic procurement. Chevaleyre, Y., Endriss, U., Estivie, S. & Maudet, N. (2004). Multiagent resource allocation

Choi, S. P. M., Liu, J. & Chan, S.-P. (2001). A genetic agent-based negotiation system, *Computer*

Chou, T.-C., Fu, L.-C. & Liu, K.-P. (2007). E-negotiation of dependent multiple issues by

Ehtamo, H., Hamalainen, R. P., Heiskanen, P., Teich, J., Verkama, M. & Zionts, S. (1999).

Faratin, P., Sierra, C. & Jennings, N. R. (1998). Negotiation decision functions for autonomous

Faratin, P., Sierra, C. & Jennings, N. R. (2002). Using similarity criteria to make issue trade-offs

Fatima, S., Wooldridge, M. J. & Jennings, N. R. (2004). An agenda based framework for

Fatima, S., Wooldridge, M. & Jennings, N. R. (2009). An analysis of feasible solutions

Gatti, N. & Amigoni, F. (2005). An approximate pareto optimal cooperative negotiation

Giovannucci, A., Cerquides, J., Endriss, U. & Rodríguez-Aguilar, J. (2010). A graphical

Guttman, R. H., Moukas, A. G. & Maes, P. (1998). Agent-mediated electronic commerce: A

He, M., Jennings, N. R. & Leung, H.-F. (2003). On agent-mediated electronic commerce, *IEEE*

*Multi-Agent Systems* 20: 342–368. 10.1007/s10458-009-9085-x.

*Transactions on Knowledge and Data Engineering* 15(4): 985–1003.

survey, *The Knowledge Engineering Review* 13(2): 147–159.

URL: *http://dx.doi.org/10.1007/s10458-009-9085-x*

for multi-issue negotiation involving nonlinear utility functions, *AAMAS '09: Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems*, International Foundation for Autonomous Agents and Multiagent Systems,

model for multiple, *IAT '05: Proceedings of the IEEE/WIC/ACM International Conference on Intelligent Agent Technology*, IEEE Computer Society, Washington, DC, USA,

formalism for mixed multi-unit combinatorial auctions, *Autonomous Agents and*

E. Durfee (ed.), *Proceedings of the 4th International Conference on Multi-agent Systems*

with k-additive utility functions, *Proceedings of the DIMACS-LAMSADE Workshop on*

using a joint search strategy, *ICRA'07: IEEE International Conference on Robotics and*

Generating pareto solutions in a two-party setting: constraint proposal methods,

TIN2008-06739-C04-04, T2C2 research project, and the mobility grant JC2010-003.

*(ICMAS-2000)*, Boston, USA, pp. 31–38.

*Automation*, pp. 1298–1303.

Richland, SC, pp. 1041–1048.

pp. 565–571.

*Management Science* 45(12): 1697–1709.

*Computer Science and Decision Theory*, pp. 83–100.

*Networks* 37(2): 195 – 204. Electronic Business Systems.

agents, *Robotics and Autonomous Systems* 24(3-4): 159–182.

in automated negotiations, *Artificial Intelligence* 142(2): 205–237.

multi-issues negotiation, *Artificial Intelligent Journal* 152(1): 1–45.

**5. Acknowledgments**

**6. References**


Manderick, B., de Weger, M. & Spiessens, P. (1991). The genetic algorithm and the structure

<sup>201</sup> Mediated Heuristic Approaches and Alternative Social Welfare Definitions

Marsa-Maestre, I. (2009). *Contribución a la negociación automática en espacios de utilidad complejos*,

Marsa-Maestre, I., Lopez-Carmona, M. A. & Velasco, J. R. (2008). Improving trade-offs in

Marsa-Maestre, I., Lopez-Carmona, M. A., Velasco, J. R. & de la Hoz, E. (2009). Effective

Marsa-Maestre, I., Lopez-Carmona, M. A., Velasco, J. R., Ito, T., Fujita, K. & Klein, M.

McBurney, P. & Parsons, S. (2002). Games that agents play: A formal framework for dialogues between autonomous agents, *Logic, Language and Information* 11(3): 315–334.

Papadimitriou, C. (2001). Algorithms, games, and the internet, *STOC '01: Proceedings of the*

Parkes, D. C. & Kalagnanam, J. (2005). Models for iterative multiattribute procurement

Robu, V., Somefun, D. J. A. & La Poutré, J. A. (2005). Modeling complex multi-issue

Rosenschein, J. S. & Zlotkin, G. (1994). *Rules of Encounter*, MIT Press, Cambridge MA, USA. Sakurai, Y., Yokoo, M. & Kamei, K. (2000). An efficient approximate algorithm for winner

Sandholm, T. (2002). Algorithm for optimal winner determination in combinatorial auctions,

Sawaragi, Y., Nakayama, H. & Tanino, T. (1985). *Theory of Multiobjective Optimization*,

Shi, C., Luo, J. & Lin, F. (2006). A multi-agent negotiation model applied in multi-objective optimization, *Agent Computing and Multi-Agent Systems* pp. 305–314. Sierra, C. (2004). Agent-mediated electronic commerce, *Autonomous Agents and Multi-Agent*

Teich, J. E., Wallenius, H., Wallenius, J. & Zaitsev, A. (2006). A multi-attribute e-auction

mechanism for procurement: Theoretical foundations, *European Journal of Operational*

*Algorithms*, San Diego, CA, pp. 1143–150.

Springer-Verlag, Berlin, Heidelberg, pp. 275–286.

for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces

Nash, J. F. (1950). The bargaining problem, *Econometrica* 18(2): 155–162.

Nisan, N. (2006). *Bidding languages for combinatorial auctions*, MIT Press, chapter 9.

Raiffa, H. (1982). *The Art and Science of Negotiation*, Harvard University Press.

*on Electronic commerce*, ACM, New York, NY, USA, pp. 30–37.

PhD thesis, Universidad de Alcalá.

*Systems (AAMAS 2009)*, pp. 1057–1064.

*Intelligence (IJCAI-09)*, pp. 214–219.

auctions, *Manage. Sci.* 51(3): 435–451.

*Artificial Intelligence* 135(1-2): 1 – 54.

pp. 749–753.

USA, pp. 280–287.

*2200291d55309bf2e*

Academic Press.

*Systems* 9: 285–301.

of the fitness landscape, *Proceedings of the Fourth International Conference on Genetic*

bilateral negotiations under complete and incomplete information settings, *PRIMA '08: Proceedings of the 11th Pacific Rim International Conference on Multi-Agents*,

bidding and deal identification for negotiations in highly nonlinear scenarios, *Proceedings of the VIII International Conference on Autonomous Agents and Multiagent*

(2009). Balancing utility and deal probability for negotiations in highly nonlinear utility spaces, *Proceedings of the Twenty-first International Joint Conference on Artificial*

*thirty-third annual ACM symposium on Theory of computing*, ACM, New York, NY, USA,

negotiations using utility graphs, *AAMAS '05: Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems*, ACM, New York, NY,

determination in combinatorial auctions, *EC '00: Proceedings of the 2nd ACM conference*

URL: *http://www.sciencedirect.com/science/article/B6TYF-4475Y1H-1/2/aafde6399310976*


24 Will-be-set-by-IN-TECH

Lai, G., Li, C. & Sycara, K. (2006). Efficient multi-attribute negotiation with incomplete

Lai, G., Li, C., Sycara, K. & Giampapa, J. (2004). Literature review on multiattribute

Lau, R. Y., Tang, M. & Wong, O. (2004). Towards genetically optimised responsive negotiation

Lewis, R. M., Torczon, V. & Trosset, M. W. (2000). Direct search methods: then and now, *Journal*

Li, M., Vo, Q. B. & Kowalczyk, R. (2009a). Searching for fair joint gains in agent-based

Li, M., Vo, Q. B. & Kowalczyk, R. (2009b). Searching for fair joint gains in agent-based

Autonomous Agents and Multiagent Systems, Richland, SC, pp. 1049–1056.

Lopez-Carmona, M. A. (2006). *Estrategias de negociación automática basadas en restricciones*

Lopez-Carmona, M. A., Marsa-Maestre, I., De La Hoz, E. & Velasco, J. R. (2011). A

Lopez-Carmona, M. A., Marsa-Maestre, I., Ibañez, G., Carral, J. A. & Velasco, J. R.

Lopez-Carmona, M. A., Marsa-Maestre, I., Klein, M. & Ito, T. (2010). Addressing

Lopez-Carmona, M. A. & Velasco, J. R. (2006). An expressive approach to fuzzy constraint

Lopez-Carmona, M. A., Velasco, J. R. & Marsa-Maestre, I. (2007). The agents' attitudes

Luo, X., Jennings, N. R., Shadbolt, N., Ho-Fung-Leung & Lee, J. H. M. (2003). A fuzzy

*difusas sobre sistemas multiagente*, PhD thesis, Universidad de Alcalá.

negotiations, *Technical Report CMU-RI-TR-04-66*, Robotics Institute, Carnegie Mellon

agents, *in* I. C. Society (ed.), *Proceedings of the IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT'04)*, IEEE Computer Society, Beijing, China,

negotiation, *in* Decker, Sichman, Sierra & Castelfranchi (eds), *Proc. of 8th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2009)*, Budapest, Hungary,

negotiation, *Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 2*, AAMAS '09, International Foundation for

region-based multi-issue negotiation protocol for nonmonotonic utility spaces,

(2010). Improving trade-offs in automated bilateral negotiations for expressive and

stability issues in mediated complex contract negotiations for constraint-based, non-monotonic utility spaces, *Journal of Autonomous Agents and Multiagent Systems*

based agent purchase negotiation, *Proceedings of the International Joint Conference on Autonomous Agents and Multi-agent Systems (AAMAS-2006)*, Hakodate, Japan,

in fuzzy constraint based automated purchase negotiations, *Multi-Agent Systems and Applications V*, Vol. 4696/2007 of *Lecture Notes in Artificial Intelligence*, Springer,

constraint based model for bilateral, multi-issue negotiations in semi-competitive

information, *Group Decision and Negotiation* 15(5): 511–528.

*of Computational and Applied Mathematics* 124: 191–207.

URL: *http://portal.acm.org/citation.cfm?id=1558109.1558159*

URL: *http://dx.doi.org/10.1111/j.1467-8640.2011.00377.x*

inexpressive scenarios, *J. Intell. Fuzzy Syst.* 21: 165–174. URL: *http://portal.acm.org/citation.cfm?id=1735086.1735088*

environments, *Artificial Intelligence* 148(1-2): 53–102.

*Computational Intelligence* 27(2): 166–217.

Published online: 1 – 51.

pp. 429–431.

pp. 246–255.

URL: *http://dx.doi.org/10.1007/s10726-006-9041-y*

University, Pittsburgh, USA.

pp. 295–301.

pp. 1049–1056.


**1. Introduction**

<sup>2</sup> See Fearon (1995).

the conflict persists over time is positive.

The presence of long-lasting conflicts in situations where reaching an agreement would benefit all parties is a pervasive phenomenon. This paradoxical fact, pointed out by Hicks (1932) in the context of strikes, has especially serious consequences when nations are engaged in war. One of the most common sources of conflict is bargaining over a territory, present in both secessionist movements as in international disputes1. This chapter aims to provide a rationale to the "Hicks paradox" in a context where two countries are engaged in open conflict for a territory. Although termination of the conflict (i.e. reaching a negotiated outcome) improves the welfare of both parties, it is shown that when bargainers act rationally, the probability that

**Social Welfare and the Emergence** 

**10**

**of Negotiations** 

*University of Murcia* 

*Spain* 

Francisco Candel-Sánchez

There are two main ways in which the inefficiencies associated to delay in bargaining are explained from a rational choice perspective: asymmetric information and dynamic commitment problems2. The former invokes the existence of some sort of incomplete information in the bargaining environment. For instance, states might prefer fighting instead of achieving a peaceful settlement when there is uncertainty about the other's cost of fighting. The latter resorts to dynamic commitment problems whereby the players may face incentives

We propose a game-theoretic approach to understand the relationship between conflict, social welfare and the likelihood that a negotiation process emerges. Although the problem we tackle here is applicable to several bargaining contexts, for expositional purposes we conduct our analysis through the example of an international dispute over territory3. In contrast with the standard view of conflict as a bargaining tool (where the threat of war enhances the bargaining power of the parties), we consider conflict as the status quo, and then focus on the strategic elements that influence conflict termination. Our contribution is then very related to the work of Wittman (1979), who envisions the end of war as a rational process. The key element in our approach is that an inefficient outcome may arise as a consequence of the parties' ability to make public statements. When engaged in conflict, strong public pronouncements against the other's territorial claims undermine the possibilities of reaching a mutually beneficial agreement. Why would then the parties make such statements? Our

<sup>3</sup> Other contexts where our analysis would apply are: bargaining between a buyer and a seller, firm-union

to renege on agreements. Our contribution belongs to the first category.

<sup>1</sup> For a survey of the theoretical work on bargaining and war, see Powell (2002).

negotiations, settlements out of court in legal disputes, etc.

*Research* 175(1): 90 – 100.

