Better, Faster, Cheaper: An Experimental Analysis of a Multiattribute Reverse AuctionMechanism with Restricted Information FeedbackAuthor(s): Ching-Hua Chen-Ritzo, Terry P. Harrison, Anthony M. Kwasnica, Douglas J.ThomasSource: Management Science, Vol. 51, No. 12 (Dec., 2005), pp. 1753-1762Published by: INFORMSStable URL: http://www.jstor.org/stable/20110465 .Accessed: 02/05/2011 04:54

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unlessyou have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and youmay use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at .http://www.jstor.org/action/showPublisher?publisherCode=informs. .

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printedpage of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

INFORMS is collaborating with JSTOR to digitize, preserve and extend access to Management Science.

http://www.jstor.org

MANAGEMENT SCIENCE int!? Vol. 51, No. 12, December 2005, pp. 1753-1762 DOI i0.l287/mnsc.l050.0433 ISSN 0025-19091 EissN 1526-55011051511211753 ? 2005 INFORMS

Better, Faster, Cheaper: An Experimental Analysis of a Multiattribute Reverse Auction Mechanism with

Restricted Information Feedback

Ching-Hua Chen-Ritzo, Terry P. Harrison, Anthony M. Kwasnica, Douglas J. Thomas

Smeal College of Business, The Pennsylvania State University, University Park, Pennsylvania 16802

(chenritzo@psu.edu, tharrison@psu.edu, kwasnica@psu.edu, dthomas@psu.edu}

The majority of reverse auctions for procurement use a single-attribute (price) format while providing con

straints on nonprice attributes such as quality and lead time. Alternatively, a buyer could choose to conduct

a multiattribute auction where bidders can specify both a price and levels of nonprice attributes. While such an auction may provide higher theoretical utility to the buyer, it is not clear that this theoretical improvement

will be realized given the increased complexity of the auction. In this research, we present an ascending auction

mechanism for a buyer whose utility function is known and dependent on three attributes. Motivated by a

supply chain procurement problem setting, we consider quality and lead time for the two attributes in addition to price. The auction mechanism provides the bidders with restricted feedback regarding the buyer's utility function. We explore, experimentally, the performance of this multiattribute auction mechanism as compared to a price-only auction mechanism. Compared with the price-only auction, we find that our mechanism design is effective in increasing both buyer utility and bidder (supplier) profits. Key words: auctions; experimental economics; supply chain management

History: Accepted by Sunil Chopra, operations and supply chain management; received January 2004. This paper was with the authors 3 months for 2 revisions.

1. Introduction Reverse auctions are auctions in which the auction eer, on behalf of a buyer, solicits bids from a group of potential sellers. They provide a promising alter native to buying goods or services at a fixed price or

through negotiation and can be easily implemented electronically. Covisint, FreeMarkets, General Electric (Kwasnica and Thomas 2002), and Mars (Hohner et al.

2003) are examples of firms that have attempted to use auctions for this purpose. While auctions are an excellent form of price discovery, there are usually other aspects in addition to price that can affect the value of the outcome. In such cases, if the auction

process focuses competition on price only, the buyer is forced to ignore these other aspects or to handle them outside of the auction by subsequent contractual

arrangements.

Presumably, the buyer would like to negotiate all dimensions of the contract with all potential sellers to discover the most valuable trade. However, it is

not clear whether a multiattribute auction mechanism can be designed that effectively achieves this, in the

ory and in practice. A multiattribute auction mecha nism should possess the following two characteristics: First, the buyer must be able to assess the value of a

multiattribute bid. Second, a bidder must be able to

bid on several attributes simultaneously. These chal

lenges raise two important questions: Is the multiat tribute auction superior to the price-only auction for the buyer/seller in theory? What about in practice? In this paper, we address these questions for a spe cific multiattribute auction mechanism that provides restricted information feedback to the bidders.

A number of researchers have considered dimen

sions other than price in the procurement setting from a purely theoretical perspective. Che (1993) and Branco (1997) study the optimal (buyer utility-maxi

mizing) auction when there are only two attributes, price and quality. In their models, however, bid der private information can still be represented by a one-dimensional type. This simplification allows them to apply powerful mechanism design tools first discussed by Myerson (1981). While there has been some recent promising work on multidimensional

mechanism design (Armstrong 2000, Williams 1999), the extension of one-dimensional optimal auctions to

higher dimensions is not straightforward. Recently, Parkes and Kalagnanam (2002), Beil and Wein (2003), and Asker and Cantillon (2004) have studied auctions

with the potential for many attributes. Parkes and

Kalagnanam (2002) look for efficient (total surplus maximizing) auctions when there may be many

1753

Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback 1754 Management Science 51(12), pp. 1753-1762, ?2005 INFORMS

attributes and products. Beil and Wein (2003) focus on a procurement process similar to ours, but examine a mechanism by which the buyer can learn the sup pliers' cost functions when suppliers are assumed to bid myopically. Asker and Cantillon (2004) study the general properties of multiattribute auctions when the

buyer's scoring function is linear in price. However, even strong theoretical properties may

not be sufficient to justify the widespread applica tion of an auction. Vickrey sealed-bid auctions, for

example, are rarely used in practice despite their attractive theoretical properties (Rothkopf et al. 1990). The English auction, on the other hand, is ubiqui tous.1 Numerous experimental studies have shown

that, even in simple price-only auctions, actual human behavior may differ systematically from what theory predicts (Kagel et al. 1987, Coppinger et al. 1980).2 Because experiments may provide a close parallel with actual auction performance, the laboratory has been successfully used to testbed auction designs in

complex settings (Banks et al. 1989, Brewer and Plott 1996, Kwasnica et al. 2005).

For the reasons given above, we turned to the

experimental laboratory to examine our multiattribute auction design. Our goal was to design an auc tion that, much like the traditional English auction,

not only had good theoretical properties, but also

performed well experimentally. Also, because both bidder (seller) and buyer behavior may matter in the overall performance of the auction, we isolated the performance of the auction with respect to bid der behavior. With this in mind, we developed and tested an auction design that explicitly values three

attributes?price, quality, and lead time?and can sig nal bidders on all three attributes simultaneously. Based on laboratory experiments with human sub

jects, the results are promising. In addition to hav ing desirable theoretical performance, our mechanism leads to increased buyer utility while maintaining seller profits relative to a price-only auction. We do not claim, however, that our auction design is opti mal. Rather, our contribution is to propose a the

oretically sound multiattribute auction design and show its viability through the use of human-subjects experiments.

We highlight the following three characteristics of this research. First, we do not fully reveal the

buyer's utility function. Second, we compare, at both the aggregate and individual level, observed bidder

behavior with the theoretically expected behavior in both a price-only and a three-attribute auction. Third,

our comparison of the single-attribute English auction to the three-attribute English auction is motivated by a

specific supply chain procurement setting. A differentiating feature of our auction design is

the type of information we provide the bidders about the buyer's utility function. Instead of revealing the

buyer's entire utility function, we only provide infor mation on the marginal utilities related to submitted bids. This information can be used to guide bidders toward their optimal bid. We made this design choice for two reasons. First, in many applications it may not be practical for a buyer to reveal their entire util

ity function, either for strategic reasons, or because

they are unable to express it. Second, the informa tion provided to the bidders is designed to be simple and intuitive. Previous experimental studies of com

plex auctions have demonstrated that simple infor mation can often be effective in generating efficient outcomes; Kwasnica et al. (2005) found that properly designed single-item prices in a combinatorial auc tion can improve efficiency despite the fact that, in

theory, these prices do not provide all the necessary information. It is possible that the limited informa tion we provide may actually improve auction per formance relative to fully revealing the buyer's utility function by simplifying the situation for the bidder.3

One of the objectives of this research is to investigate whether a reasonably successful multiattribute auc tion can be designed with information that is both

readily available for the buyer and simple for the bid ders to understand.

Bichler (2000) was one of the first to use experi ments to study multiattribute auctions. He examined

two-attribute auctions using human subjects experi ments and found that the auctions, in general, have modest gains over single-attribute auctions. It is pos sible that, due to the added complexity of the auc

tion, the gains observed with only two attributes

may disappear when more attributes are considered. Therefore, in our experiments, we consider an auc

tion where the buyer and bidders care about three attributes (i.e., price, quality, and lead time). In his experiments, Bichler revealed the entire scoring func tion used by the buyer. Strecker (2003) conducted three-attribute English auction experiments, where the buyer's scoring function was not at all revealed in one treatment and fully revealed in the other. In contrast to both these studies, we examine a multiat tribute auction that provides partial revelation of the

buyer's scoring function.

1 Cramton (1998) discusses the pros and cons of using an English auction rather than a sealed-bid mechanism. 2 See Kagel and Roth (1995) for a good, albeit somewhat outdated, review of the large experimental literature on simple auctions.

3 For example, if we published the buyer's utility function, bid ders would need to perform calculations to arrive at the optimal attribute bid levels. Assuming the bidders recognize that they need to do this, it is still questionable whether they would make the correct calculations.

Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback

Management Science 51(12), pp. 1753-1762, ?2005 INFORMS 1755

Both Strecker (2003) and Bichler (2000) provided bidders with tools for computing the buyer's util

ity for all possible bids, whereas the issue of what kind of information to provide bidders within a three attribute auction setting was studied by Koppius and van Heck (2003). They designed an experiment in

which they exposed subjects to different information architectures?a restricted architecture and an unre stricted architecture. In the restricted architecture, bid ders were informed of the current best bid and bidder. In the unrestricted architecture, they were also pro vided with a ranking of the most recent losing bids.

Koppius and van Heck (2003) found that both the effi ciency and Pareto optimality are greater in the unre stricted information architecture case. Although their

experiment addressed the effect of two specific types of information exchange on the aggregate outcomes of the multiattribute auction, they did not compare these results against the single-attribute auction case.

Moreover, the information presented to the bidders in both cases is actually very limited with respect to

guiding bidders toward their optimal bid. We elabo rate on this point in ?3.1.

This paper is organized as follows: In ?2, we describe and motivate the functional forms for buyer utility and bidder profit functions. In ?3, we present the auctions examined. In ?4, we describe the exper imental design. Results are presented in ?5. We con clude our work and discuss future directions in ?6.

2. The Environment There is a single buyer who is procuring a single indi visible object or contract from a set of three potential suppliers.4 The contract has three measured attributes of interest: price p, quality level q, and lead time Z. Lead time is typically thought of as the amount of time that the buyer must wait between order and

receipt of goods. Throughout, we use the subscripts 0 for the buyer and i for the bidders and the sub

scripts Q and L to denote parameters related to qual ity and lead time.

The buyer's utility function for the contract is linear in price, with v(q, I) representing the value of the two

nonprice attributes quality q and lead time I. In our

experiments, the buyer's utility function for a contract is given by

U(p, q, I) = v(q, l)-p = aQOqb& + aL0(l -

l**) -p, (1)

where aQ0 > 0 and bQ0 are quality-related constants and aL0 > 0 and bL0 are lead-time-related constants. The buyer always prefers higher quality. In our exper iments, we chose bQ0 such that the marginal value of quality is decreasing (0 < bQ0 < 1).

The profit function for bidder (supplier) i for an awarded contract is linear in price, with c{(q, I) rep resenting bidder i's cost as a function of quality and lead-time levels. In our experiments, bidders have cost functions that yield the following profit function:

ir,-(P, (2)

where aQi > 0 and bQi are quality-related constants and aLi > 0 and bLi are lead-time-related constants. Each bidder's cost increases with improvements in both quality and lead time. The marginal cost of

higher quality is increasing (bQi > 1). If the bidder does not win the auction, she will have a profit of zero.

Define the total welfare created by a contract between the buyer and bidder i at quality and lead time levels (q, I) by

WK?, /) = U{p, q, I) + fli(p, q, 1) = v(q, I) - Ci(q, I). (3) Let (q*, I*) be quality and lead-time levels that max imize Equation (3). Note that the buyer can select at

most one supplier, and that total welfare is unaffected

by the contract price. Thus, an efficient procurement contract is given by the supplier (i) and the corre sponding (q*, I*) that maximize total welfare.

3. The Auctions We consider two auction mechanisms. Both auctions

are generalizations of the standard reverse English auction to the multiattribute setting. The first is the

multiattribute (MA) auction that allows for bidding in all three attributes. We compare this auction to the price-only (PO) auction where reservation qual ity and lead-time levels are predetermined and bids are

only accepted on price. Only integer-valued prices are accepted, and quality and lead-time offers are

required to be integer values ranging between 1 and 10. The bidders were human subjects, whereas the buyer was represented by the computer. Other than a time constraint, bidders are free to place any

bid at any time. Before the auction ends, the provi sionally winning, or standing, bid is the one that max imizes the buyer's utility among all the bids placed since the start of the auction.5 Once a bid is placed, it cannot be cancelled.

For both the PO and MA auctions, a "soft" stopping rule was used. That is, after an initial time period, the

4 We recognize that there may be interesting questions related to

the incentive to select multiple suppliers at potentially different attribute levels. Bichler and Kalagnanam (2005) consider multi

sourcing for multiattribute auctions. We focus on the design of the auction when the buyer must select at most one supplier and the

suppliers can offer different levels of attributes. 5 In the event of tie bids, the bid placed earliest always wins.

Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback 1756 Management Science 51(12), pp. 1753-1762, ?2005 INFORMS

auction ended if no new standing bids were placed for k seconds. The purpose of extending the auction in this way was to eliminate the advantage that might

be provided to bidders by sniping (i.e., placing a bid in the last few seconds of the auction).6 In the PO auc tion, the initial time period was two minutes and /c = 45 seconds. In the MA auction, the initial time

period was five minutes and k = 90 seconds.

3.1. The Multiattribute Auction In the MA auction, bidders were required to iden

tify multiattribute bids that were not only profitable, but that would also improve the buyer's utility with

respect to the current standing bid. A critical element of the MA auction is the feedback provided to the bidders. Because we have in mind future applications where the buyer utility function may not be known with certainty, we opted against directly publishing the buyer's utility function. Instead, after each bid, simple information about the buyer's preferences was revealed to the bidder. First, the standing bid was

always displayed. Second, the bidder was given three

pieces of information related to the bid she placed: the price difference (PD), the marginal quality (MQ), and the marginal lead time (ML). The price differ ence is the amount that the bidder must lower her bid in the price attribute for it to become a winning bid

(assuming no changes in quality and lead-time lev els). The marginal quality and lead time are simply the buyer's marginal utility for improvements in q or Z at the bidder's most recently placed bid. Importantly, in our experiments, the MQ value combined with the

bidder's marginal cost provided her with sufficient information to eventually bid at q*. Similarly, ML and

marginal cost information will direct the bidder to /*. Both the information structures studied by Koppius and van Heck (2003) do not necessarily have this fea ture. Under Koppius and van Heck's restricted infor

mation condition, the bidder only receives feedback on the winning bid; this suggests how to beat the current bid but not how to find the optimal bid for each particular bidder. Under the unrestricted con

dition, the bidder observes more information in the form of the relative ranking of the bids. However, the usefulness of this information in supporting opti

mal bidding behavior depends on the actions of the other bidders.7 An advantage of our information feed back is that the bidder can independently determine her unique optimal attribute combination. It remains to be seen whether this information can be properly

used by the bidders; that issue necessitates our experi mental examination. Wfhile the above feedback is pro vided for every bid submitted in the MA auction, throughout the auction, only bids that beat the cur rent best bid are accepted.

When bidders can determine the optimal quality and lead-time combinations (q*,l*), it is straightfor

ward to show that it is a dominant strategy for them to bid at those levels. Therefore, consistent with the standard English auction, it is also a dominant strat

egy to continue lowering the price offered until they make zero profits. If all bidders follow their dominant

strategy, the auction will be efficient and the expected utility of the buyer will be equivalent to the second

highest welfare offer of the bidders. See Chen-Ritzo et al. (2005) for a formal discussion of these properties of the MA auction.

If the buyer had information about bidder costs, then he may find it in his interest to misrepresent his true utility function in the MA auction. Che (1993) shows that this is optimal in a simpler multiattribute

setting where bidder types are still one-dimensional. Of course, any misrepresentation will result in lost

efficiency. We did not examine buyer misrepresenta tion because our focus is on examining bidder behav ior in these auctions. Specifically, the focus of this

paper is on whether an auction can be designed that

provides sufficient decision support to allow the bid ders to bid as theory predicts in a complex, multi attribute setting. The human-subject bidders in our experiment knew that the utility or scoring func tion used by the auctioneer was programmed by the

experimenter. Therefore, issues of buyer misrepresen tation were not a concern.

3.2. The Price-Only Auction In the PO auction, all bids are simply a state

ment of a price at which the bidder is willing to

supply the product. Before the auction begins, the

buyer announces reservation levels for the nonprice attributes. Let qr and V be the reservations levels. All bids are then evaluated as if they were placed with qr and V in the other attributes. Bidders were expected to identify the least cost quality and lead-time combi

nation satisfying these reservation levels to determine their reservation price.

Given the properties of the bidders' cost functions, bidders would never voluntarily deliver the product at a quality or lead time better than the reserve levels.

Once the reserve levels are announced, the PO auction is a standard reverse English auction. It is a domi

nant strategy for each bidder to bid down to the point where, at the reserve levels, profits are zero, mean

ing that the auction will end at the second-lowest cost given the reserve levels. Because the reserve levels eliminate many bid choices for the bidders,

6 See Roth and Ockenfels (2002) for more on the strategy of sniping. 7 As an example, imagine the extreme condition where all bidders

placed bids with identical nonprice attributes. In that case, the rank

ing would provide no more information about potential improve ments than a simple price-only auction.

Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback

Management Science 51(12), pp. 1753-1762, ?2005 INFORMS 1757

it is straightforward that buyer utility will always be at least as high in the MA auction as it is in the PO auction (Chen-Ritzo et al. 2005).

To give the PO auction the best chance of suc

cess, we chose the reserve levels for lead time and

quality that maximized buyer utility assuming full

knowledge of the bidder profit functions. While it is unrealistic to expect that a buyer would have full information when setting the reserve levels, we selected this standard to provide a difficult test for the MA auction with which to compete. It is worth

while to note that the reserve levels (for quality and lead time) that maximize buyer utility are not neces sarily the efficient levels. Furthermore, even though total welfare in the MA auction is no smaller than

welfare in any PO auction, the profit of the winning bidder may be lower in the MA auction.

4. Design of the Experiment We considered four different treatments in our exper

iment: (1) The PO auction with inexperienced bidders (PO), (2) the PO auction with experienced bidders (P02), (3) the MA auction with inexperienced bid

ders (MA), and (4) the MA auction with experienced bidders (MA2). The response variables of interest were, primarily, the realized buyer utility and seller

profits. We recruited subjects for a total of 24 experi

mental sessions,8 where in each session we exposed subjects to one of the four treatments previously described. Each session consisted of several indepen dent auctions, or periods. The same three bidders

were matched with each other in all periods, and each

group of bidders participated in either the PO auc tion or the MA auction for all periods. We decided to study the auction institutions in separate sessions rather than using a within-subjects design where the same subjects would have participated in both the PO and MA auctions. This allowed us to isolate the

learning effects that we expected to observe for a

given auction type from potential order effects (i.e., variation due to the order in which subjects would have participated in the different auction types in a

within-subject treatment). Additionally, because our subject pool was relatively homogeneous, it is reason able to assume that any effects relating to differences between the subject groups would be small.

In every auction (or period), subjects were provided with information about their bid history in the cur rent auction, their costs for delivering the asset at all

possible lead-time and quality combinations, the time

remaining in the auction, the current best bid and their bid status (i.e., whether or not they were the

winning bidder). The costs of the bidders were dif ferent from period to period and were different from

bidder to bidder. The buyer's utility function was also different from period to period. To help bidders iden

tify profitable bids, they were provided with a tool built into the computer interface that would compute the profit associated with a particular bid.

A total of nine auctions (or periods) were com pleted in each experimental session, the first of those being a practice period for which subjects

were not paid any earnings. Each of four different cases (labeled Case 1 through Case 4) was repeated once during the eight nonpractice periods. A dif ferent case from these four was used in the prac tice period. The sequence of cases was kept constant across all sessions so as not to affect comparisons of results across sessions. Whenever a case was repeated

within a session, the matching of subjects to bid der types was changed so that no subject was the same type in both replicates of each case. This elim inated learning effects that might be associated with

playing the same bidder type more than once. The

subject-bidder type matchings were kept constant across all sessions. The exact parameters used in each case, experiment instructions, screen shots, and other information are available in the online supplement (http://mansci.pubs.informs.org/ecompanion.html).

All experimental sessions were conducted at the Smeal College of Business's Laboratory for Economic

Management and Auctions (LEMA), The Pennsylva nia State University, between March 2003 and April 2003. Any undergraduate or graduate student who had not previously participated in this experiment was considered to be eligible for treatments that

required inexperienced bidders. Only those students who had previously participated in the PO auc tion (MA auction) were eligible to participate in the PO auction (MA auction) treatments that required experienced bidders. The auctions were implemented using Z-Tree software (Fischbacher 1999).

In the PO auction sessions, each period lasted

approximately 5-10 minutes, and in the MA auc tion sessions, each period lasted approximately 10-15

minutes. The PO auction sessions took around 1-1.5 hours to complete, while the MA auction sessions took around 1.5-2 hours to complete, on average. The

exchange rate used in all treatments was 0.1 U.S. dol lars for every experimental dollar. A $7 show-up fee

was added to the total earnings of each subject at the end of the session. No incentive besides cash

was provided to subjects. Subjects earned, on aver age, $22 and $16 in the MA and PO auction sessions, respectively.

5. Results In this section, we compare the performance of the

MA auction and the PO auction for bidders with and 8 A session refers to a continuous period of time where a fixed

group of subjects is exposed to a single experimental treatment.

1758 Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback

Management Science 51(12), pp. 1753-1762, ?2005 INFORMS

Table 1 Expected and Observed Outcomes for Each Case and Treatment Condition

Case 1 Case 2 Case 3 Case 4

Expected MA (PO) outcome Price 130(550) 52(299) 561(294) 2,399(2,364)

Quality (reserve) 4(4) 3(2) 3(4) 10(9) Lead time (reserve) 10(5) 10(6) 3(7) 5(3)

Winner 3(2) 3(2) 2(3) 1(1) Buyer utility 285 (250) 98 (84) 333 (237) 1,986 (1,870) Seller profit 62(10) 46(4) 48(6) 88(49) Improvement in buyer utility 14% 17% 41% 6%

MA (PO) observed Average price 646.6 (559.5) 190.7 (297.2) 577.1 (381.9) 2,035.5 (2,350.6) % Expected quality 44% (N/A) 38% (N/A) 38% (N/A) 50% (N/A) % Expected lead time 25% (N/A) 56% (N/A) 44% (N/A) 36% (N/A) Average distance 4.33 (N/A) 2.45 (N/A) 2.52 (N/A) 2.45 (N/A) % Expected winner 25% (80%) 63% (70%) 75% (85%) 50% (45%) Average buyer utility 240.5 (241.5) 98.5 (85.9) 281.3 (249.1 ) 1,887.1 (1,884.4) Average seller profit 44.1 (-128.4) -6.1 (-64.6) 2.6 (-25.2) 115.4 (-580.5) Number of observations 16 (20) 15 (20) 16 (20) 14 (20)

MA2 (P02) observed Average price 126.7(630.2) 94.5(299.8) 595.3(393.8) 2,090.0(2,355.2) % Expected quality 100% (N/A) 33% (N/A) 67% (N/A) 67% (N/A) % Expected lead time 100% (N/A) 100% (N/A) 67% (N/A) 33% (N/A) Average distance 0.00 (N/A) 0.67 (N/A) 0.47 (N/A) 1.69 (N/A) % Expected winner 100% (83%) 100% (100%) 100% (100%) 67% (50%) Average buyer utility 288.3 (170.8) 94.2 (83.2) 300.7 (237.2) 1,863.7 (1,879.8) Average seller profit 58.1(87.6) 46.8(4.0) 62.0(5.8) 149.0(39.6) Number of observations 6(6) 6(6) 6(6) 6(6)

without previous experience. In Table 1, we present the expected outcome (i.e., the winning bid, win

ning bidder, buyer utility, and seller profit) and the observed average outcome for both auction institu tions, in each of the cases and treatments studied. The

number of observations collected for each treatment in the different institutions is also included in this table.9 Table 2 presents the results of the rank-sum tests that we use for statistically comparing certain outcomes between treatments.10

We begin by examining the utility that the buyer achieves at the end of the auction. In theory, because bidders (sellers) have more choice in their bidding under the MA auction, they should provide the buyer with higher utility. However, the added complexity of the MA auction may keep the buyer from fully real

izing these potential gains. Conclusion 1. The MA auction increases buyer

utility. In Table 1 we report the expected improvement

in buyer utility in the MA auction as compared to

the PO auction. Based on the expected improvements reported in Table 1 and the results reported for the first two sets of rank-sum tests in Table 2, the MA auc tion does result in significantly greater buyer utility

when the expected improvement from the MA auc tion is large (17% or greater). With both inexperienced and experienced bidders, we can reject the equal ity of the observed buyer utility, and conclude that the MA auction provided stochastically greater buyer utility in Cases 2 and 3. Case 1 has a somewhat lower

expected improvement (14%), and we find that buyer utility is greater from the MA auction only under

Table 2 Wilcoxon-Mann-Whitney Rank-Sum Test Results

Test(fy,) Casel Case 2 Case 3 Case 4

Buyer utility w = 325.0 w = 343.0 tv = 368.0 w = 272.0 MA>P0 p = 0.1802 p = 0.0682 p = 0.0112 p = 0.1764 Buyer utility w = 57.0 w = 53.5 iv = 51.0 w = 39.0

MA2>P02 p = 0.0023 p = 0.0079 p = 0.0311 p = 0.5000 Seller profit n/ = 262.0 w = 246.5 w = 355.0 w = 385.0

MA^PO p = 0.2858 p = 0.4414 p = 0.0616 p = 0.0000 Seller profit iv = 33.0 iv = 51.0 u/ = 52.0 w = 28.5

MA2^P02 p = 0.3751 p = 0.0557 p = 0.0435 p = 0.1025 Euclidean distance w = 220.0 w = 202.0 iv = 210.0 iv = 157.0

M/4>M/12 p = 0.0038 p = 0.0883 p = 0.0264 p = 0.2139

Note. The left-hand column represents the measured variable and the alter native hypothesis of the statistical test, w is the rank-sum test statistic and

p is the p-value.

9 The number of observations vary in the MA treatment due to a technical difficultly related to the recording of the results to a data file. 10 The Wilcoxon-Mann-Whitney rank-sum test is a powerful non

parametric substitute to the standard f-test when data has at least ordinal measurement (Siegel and Castellan 1988). When examining data generated from human subjects, it is typical to assume that the data do not meet the assumptions required for a f-test.

Chen-Rltzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback

Management Science 51(12), pp. 1753-1762, ?2005 INFORMS 1759

the experienced treatment. Case 4, on the other hand, has only a small expected difference between the MA and PO auctions at 6%. As the toughest test for the

MA auction, it is not surprising that we are never able to conclude that buyer utility is greater under the MA auction. The PO auction does not provide the

buyer with greater utility in any case.

Although we have seen that the MA auction can

significantly improve buyer utility, it is worthwhile to examine how well the auctions performed relative to the expected outcomes. Again, referring to Table 1, we find that on average, the MA auction achieved 91% and 95% of the expected buyer utility for the inexpe rienced and experienced treatments, respectively. The PO auction tends to yield slightly higher utility than

predicted. On average, the PO auction yields 101% of the buyer utility predicted. This difference disap pears in the experienced treatment, and the P02 auc tion only yields 92% of the predicted buyer utility.

The auctions' performances relative to the theoretical benchmarks may shed light on the fact that we did not always observe a significant increase in buyer util

ity in the MA auction relative to the PO auction. At least for the inexperienced treatments, the MA auction fails to achieve full expected buyer utility, whereas the PO auction tends to overshoot predictions slightly.

The winning bidder, quality, and lead time are fre

quently as predicted for the MA auction format. As an additional measure of the closeness of the observed

multiattribute bid to the expected bid, we report (Table 1) the Euclidean distance of the observed qual ity and lead-time levels from those expected in the

MA auction. With the exception of Case 1 for the MA treatment, this distance is usually small.

The MA auction also offers potential gains for the bidders. In all four cases, the theoretical seller profits are higher under the MA auction than the PO auction.

Although this may seem to contradict the previous result, the added flexibility of the MA auction often benefits both parties. We examine seller profitabil ity under the different auction institutions in Conclu sions 2 and 3.

Conclusion 2. The MA auction does not degrade, and occasionally increases, seller profits.

The third and fourth sets of rank-sum tests in Table 2 compare seller profits in the MA and PO auc tions. In no case can we conclude that seller profit is

higher under the PO auction. In some cases, 3 and 4 for inexperienced bidders and 2 and 3 for experienced bidders, seller profits actually increase significantly under the MA auction. These observations are con sistent with the theoretical predictions; for the cases studied the theory predicts higher seller profits. We did not study cases where the theory predicts lower seller profits.

As Table 1 shows, average seller profits are often

negative. This is because, occasionally, sellers placed bids that were not individually rational. In 11 out of 80 (14%) PO observations and 5 out of 62 (8%)

MA observations, the winning bidder ended up los

ing money. While it is impossible to determine the source of this behavior, bidders were not in any way forced to bid below their cost (at a loss) and were given checks to help avoid this behavior. We feel the most likely determinant of these losses was the

complex setting the bidders had to understand in a time-constrained, competitive environment, which

was our primary motivation for studying these auc tions experimentally.

Conclusion 3. The frequency of seller losses are not related to the auction institution.

Despite the somewhat greater prevalence of bid der losses in the PO auctions, there is no statisti

cally significant difference between the proportion of observations with losses under the different institu tions. A chi-squared test for differences in proportions yields a test statistic of 0.5807, which is not signifi cant at any reasonable level. The fact that there is little difference in the observed frequency of losses under the two institutions is actually somewhat surprising.

A priori we expected that bidder errors would be a

greater problem in the MA auction. In principal, the PO auction was no different from a standard reverse auction, where previous experiments rarely observed bidder losses. However, it appears that the added information of the multiattribute cost tables, despite its irrelevancy in the bidding, might have affected

bidder behavior. Consistent with other experimen tal studies of auctions, we have identified behavior that differs dramatically from the theory. The level of losses actually tends to be greater in the PO auction as well; losses averaged 1,641 experimental dollars in the PO auction versus 217 experimental dollars in the MA auction. Using a rank-sum test, we find that losses under the PO auction are stochastically larger (w = 57.0 and p < 0.0200). These high losses also affect the buyer utility results presented earlier because bid der losses tend to manifest themselves as increased

utility for the buyer. When observations with losses are removed, we find that the PO auction buyer utility as a percentage of expected drops from 101% to 93%. The decline in the MA auction is more modest from 91% to 88%. All buyer rank-sum utility comparisons presented earlier in support of Conclusion 1 still hold if observations with losses are dropped.

Experienced bidders never made losses in either the P02 or MA2 treatments. Therefore, it appears that losses might be more related to learning the interface and environment rather than some systematic prob lem with the auction setup. This also suggests our

next conclusion.

Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback 1760 Management Science 51(12), pp. 1753-1762, ?2005 INFORMS

Conclusion 4. Performance of the MA auction institution improves with experience.

While buyer utility and seller profits rarely change significantly from the inexperienced to the experi enced treatments (see Table 2), the auctions appear to perform more consistently with expectations when there are experienced bidders. When comparing the

MA and MA2 treatments, the proportion of times the predicted winner is observed increases from 53% to 92%. The proportion of correctly predicted qual ity and lead times increases from 42% to 67% and from 40% to 75%, respectively. These differences in

proportions are all significant at the 10% level; a chi

squared test comparing these proportions yielded test statistics of 9.485 (winner), 3.303 (quality), and 6.993 (lead time). If we measure the Euclidean distance

of the winning quality and lead-time combination from the expected outcome, we find that it decreases from 2.96 to 0.71 on average. In three of four cases, the distance drops significantly from inexperienced to

experienced treatments (see Table 2). As a result, as mentioned earlier, the observed buyer utility as a per centage of the predicted increases from 91% to 95%

with experience. The performance of the PO auction improves less

dramatically. While losses are no longer observed and bidders always provide the reservation quality and lead time,11 the proportion of times that the predicted

bidder wins only increases from 70% to 83%, which is not statistically significant (chi-squared test statis tic of 1.059). In addition, buyer utility actually goes down relative to predicted utility; buyer utility as a

percentage of predicted drops from 101% to 93%. The previous results have highlighted the aggre

gate performance of the MA and PO auctions. We can also examine individual bidding behavior and how it relates to the theoretical predictions. Bidders should

place bids at the optimal levels of quality and lead time (??*,/*) in the MA auction. Only 9% and 11% of all bids placed in the MA and MA2 treatments,

respectively, are at the predicted levels. This is not

particularly surprising once one considers the fact that bidders must bid to learn the buyer's utility func tion in the experiments. However, we would expect that bids near the end of the auction will be close to the optimal levels. When considering only the best

(highest utility) bid placed by each bidder, the propor tion of optimal bids increases to 19% (MA) and 31% (MA2). This difference in proportions is significant at the 10% level (chi-squared test statistic of 3.26), indicating that experience increases the incidence of

behavior consistent with the theory. Although this result tells us that bidding behavior, even in the

experienced treatment, is often different than the the

ory predicts, it may be that bidders are only mak

ing small errors in their bidding. Given the struc ture of the cost and utility functions utilized, there

may be only small gains from movements to the effi cient quality and lead-time levels. By measuring the Euclidean distance of each bidder's best bid from the predicted quality and lead-time levels, we have a

measure of distance from the predicted strategy. In the MA and MA2 treatments, the distance averaged 2.40 and 2.01, respectively. Even more telling is that only 3 out of 33 total bidders averaged bids within one unit distance of the optimal strategy.

Even though bidders are not always bidding the correct levels of quality and lead time, they may still be placing bids close, in utility terms, to the opti

mal bid. In theory, losing bidders should offer a bid that provides all the surplus, at the efficient quality and lead time, to the buyer. By examining the highest utility offers of the losing bidders, we can examine the

accuracy of this benchmark.12 We can examine when

losing bidders leave "money on the table," or they do not bid despite the fact that there was a profitable bid for them that would beat the final winning bid. In the

inexperienced treatment, 51% of losing bidder obser vations (there are two observations for each auction period) failed to bid when a profitable bid was avail able. The proportion falls to a more reasonable, but still large, 35% in the experienced treatment. A chi

squared test for difference in these proportions yields a test statistic of 2.70, which is significant at the 10% level. This suggests, along with the previous result, that bidders are often failing to realize the optimal

bidding strategy. This is in contrast to the relatively strong performance of the MA auction on an aggre gate level. To reconcile these results, it must be that the actual utility offered by these bidders is close to the theoretical prediction. We can measure this by cal

culating the utility of each losing bidder's best bid

(i.e., the bid that provided the buyer with the greatest utility) as a percentage of the maximal possible util ity that bidder could offer (i.e., provision of greater

buyer utility would result in bidder losses). Figure 1 shows the fraction of the maximum possible buyer utility that was offered by each losing bidder's best bid in the MA auctions.13 In the MA and MA2 treat

ments, respectively, the losing bidders offer 57% and

11 In the inexperienced treatment, three bidders provided either

quality and/or lead time in excess of the reservation amounts.

12 The bids of the winning bidders cannot be examined because they should stop bidding when others stop bidding. Thus, they will not

bid up to their zero profit point. 13 It is important to note that this measure is biased downward.

Even if a losing bidder is following the optimal strategy, she may

appear to be offering less than 100% under this measure. This is because the winning bidder may jump over the maximal utility offer of the losing bidder, making placement of such a bid unnec

essary. We decided to use this measure because it gives noncom

pliance with the theory the benefit of the doubt.

Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback Management Science 51(12), pp. 1753-1762, ?2005 INFORMS 1761

Figure 1 Fraction of Maximum Available Buyer Utility Offered by Losing Bids Only

1.0

0.8

?0.6

CO

0.2

0

x x\ Xxxx ***?**. xX xxx

x

x

X XX i-Xa )X?3? i-r^X-rX-Xr

X MA MA2

12 3 4 5

Period

78% of the maximal possible utility on average. Using a rank-sum test, we find that losing bidders leave sig nificantly more money on the table in the inexperi enced case than in the experienced case (w = 348 and p < 0.016).

Conclusion 5. Individual bidding behavior in the MA auction is different than the theory predicts, but

improves with experience. Given previous experimental results on bidder

behavior in auctions, it should not be entirely sur

prising that specific theoretical predictions for the MA auction do not do particularly well. Rather, a more appropriate comparison would be to the indi vidual bidding behavior in the PO auction. In the PO auction, theory predicts that losing bidders should bid up to their zero profit condition at the reserve

quality and lead time. Using the same standard as

presented earlier for the MA auction, we find that 31%

(PO) and 39% (P02) of all losing bidders left money on the table. When compared to the MA treatments, only the PO proportion is significantly smaller than the MA proportion (chi-squared test statistic of 7.68). In both cases, these proportions are indistinguishable from the MA2 proportion of bidders failing to make

profitable bids. If we calculate the utility of the los

ing bidders' best offer as a percentage of the maximal

utility offer available for that bidder, we find that, on

average, bidders offered 92% and 90% of the maxi mal utility offer for the PO and P02 auctions, respec

tively. If we compare the percentages of the maximum available utility offered by losing bids in the PO auc tions with those in the MA auctions, we find that, at the 5% level, only the P02 and MA2 auctions are not significantly different. Therefore, with experience, the percentage of the maximum available buyer util

ity offered by losing bids is consistent between the two auction types. The results of the rank-sum tests for these comparisons are provided in Table 3.

Conclusion 6. At least with experience, individ ual bidding behavior is similar in the MA and PO auctions.

Table 3 Wilcoxon-Mann-Whitney Rank-Sum Test Results

Comparing Proportions of Maximum Available Buyer Utility Offered by Losing Bids

Mann-Whitney p-value Test (HA) test statistic (w) (p)

PO^MA 1,075.5 0.0000 PO^MA2 684.0 0.0049 P02^MA 214.0 0.0144

P02^MA2 104.0 0.1120

Note. The left-most column represents the alternative hypothesis of the statistical test.

While individual bidding behavior is not exactly as

predicted in theory, it is relatively consistent across the two auction mechanisms. Individual behavior also

gets closer to the predicted behavior with experience in the MA auction.14 This suggests that the added

complexity of the auction environment does not lead to systematically different behavior. Variations from the theory are expected in both cases, and highlights the need to study auction design from experimental

perspective in addition to a purely theoretical devel

opment. The fact that bidders in both cases seem to behave similarly helps reconcile the superior aggre gate performance of the MA auction (compared to the PO auction) despite obvious deviations from the the oretical benchmark.

6. Conclusions and Future Work The results presented here are promising for the

application of MA auctions for procurement when several nonprice components matter. Providing bid ders with only marginal information regarding the

buyer's utility, we are able to construct a simple auc tion mechanism that clearly betters, experimentally, the PO auction format where quality and lead time are set in advance. The MA auction benefits both the

buyer and the sellers. It is only when the expected dif ferences are small that we do not observe significant improvements from the experimental MA auction.

The experimental results, by showing where real

ity differs from theory, provide potentially important insights into the MA auction design problem. Bidder

behavior was not perfect. It took experienced sub

jects to get an MA auction that closely matched, on an aggregate level, the dominant strategy predictions. Even then, individual bidding behavior was not par ticularly close to the theoretical predictions. The level of bidder losses was also not predicted. Despite these differences, the MA auction still consistently outper forms the PO auction in the lab. However, future

designs for more complex or realistic settings might

14 It is interesting to note that the same trend does not appear to be true in the PO auction.

Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback 1762 Management Science 51(12), pp. 1753-1762, ?2005 INFORMS

consider these deviations in an attempt to design a better MA auction.

This study has focused on the behavior of bidders in the multiattribute setting. When multiple attributes

matter, the decisions of the buyer should also be considered. An issue facing the real-world auction

designer is that the buyer may not know his utility function. Instead, he must learn about what he wants

by observing the different options available. Also, in

many supply chain auctions there are a great num ber of interdependent objects being procured simulta neously. For example, while a buyer may care about

quality and lead time, he may also care about which of two different products arrives first. These concerns

greatly complicate the multiattribute design problem and require the incorporation of combinatorial auc tion methods as discussed by Parkes and Kalagnanam (2002) and Kwasnica et al. (2005).

An electronic companion to this paper is available on the Management Science website (http://mansci. pubs.informs.org/ecompanion.html).

Acknowledgments The authors gratefully acknowledge support from the Cen ter for Supply Research (CSCR), the Institute for the Study of Business Markets (ISBM), and the Laboratory for Eco nomics Management and Auctions (LEMA) at the Smeal College of Business at The Pennsylvania State University They also thank an anonymous associate editor and two

anonymous referees. This article has benefitted from their

thoughtful input.

References

Armstrong, M. 2000. Optimal multi-object auctions. Rev. Econom. Stud. 67(3) 455-481.

Asker, J., E. Cantillon. 2004. Properties of scoring auctions. Unpub lished manuscript, Harvard University, Cambridge, MA.

Banks, J. S., J. O. Ledyard, D. P. Porter. 1989. Allocating uncer tain and unresponsive resources: An experimental approach.

RAND J. Econom. 20(1) 1-25. Beil, D. R., L. R. Wein. 2003. An inverse-optimization-based auction

mechanism to support a multiattribute RFQ process. Manage ment Sei 49(11) 1529-1545.

Bichler, M. 2000. An experimental analysis of multi-attribute auc tions. Decision Support Systems 29 249-268.

Bichler, M., J. Kalagnanam. 2005. Configurable offers and win ner determination in multi-attribute auctions. Eur. ]. Oper. Res. 160(2) 380-394.

Branco, F. 1997. The design of multidimensional auctions. RAND J. Econom. 28(1) 63-81.

Brewer, P. J., C. R. Plott. 1996. A binary conflict ascending price (BICAP) mechanism for the decentralized allocation of the right to use railroad tracks. Internat. J. Indust. Organ. 14(6) 857-886.

Che, Y.-K. 1993. Design competition through multidimensional auc tions. RAND J. Econom. 24(4) 668-680.

Chen-Ritzo, C.-H., T. P. Harrison, A. M. Kwasnica, D. J. Thomas. 2005. Better, faster, cheaper: A multi-attribute supply chain auction mechanism. Technical report, Smeal College of Busi ness Center for Supply Chain Research, The Pennsylvania State

University, University Park, PA.

Coppinger, V., V. L. Smith, J. Titus. 1980. Incentives and behavior in English, Dutch and sealed-bid auctions. Econom. Inquiry 43 1-22.

Cramton, P. 1998. Ascending auctions. Eur. Econom. Rev. 42 745-756.

Fischbacher, U. 1999. Z-Tree: A toolbox for readymade economic

experiments. Working paper no. 21, University of Zurich, Zurich, Switzerland.

Hohner, G., J. Rich, E. Ng, A. Davenport, J. Kalagnanam, H. S. Lee, C. An. 2003. Combinatorial and quantity-discount procurement auctions benefit Mars, Incorporated and its suppliers. Interfaces 33(1) 23-35.

Kagel, J. H., A. E. Roth, eds. 1995. The Handbook of Experimental Economics. Princeton University Press, Princeton, NJ.

Kagel, J. H., R. M. Harstad, D. Levin. 1987. Information impact and allocation rules in auctions with affiliated private values:

A laboratory study. Econometrica 55 1275-1304.

Koppius, O. R, E. van Heck. 2003. Information architecture and electronic market performance in multidimensional auctions.

Working paper, Erasmus University, Rotterdam, The Nether lands.

Kwasnica, A., D. Thomas. 2002. Case study: General electric trans

portation systems. W Grenoble, D. Thomas, eds. eBusiness and

Supply Chain Processes. The Supply Chain Council, Pittsburgh, PA, 40-44.

Kwasnica, A. M., J. O. Ledyard, D. Porter, C. DeMartini. 2005. A new and improved design for multi-object iterative auctions.

Management Sei. 51(3) 419-434.

Myerson, R. B. 1981. Optimal auction design. Math. Oper. Res. 6 58-73.

Parkes, D. C, J. Kalagnanam. 2002. Iterative multiattribute Vickrey auctions. Management Sei. 51(3) 435-451.

Roth, A., A. Ockenfels. 2002. Last-minute bidding and the rules for ending second-price auctions: Evidence from eBay and

Amazon auctions on the Internet. Amer. Econom. Rev. 92(4) 1093-1103.

Rothkopf, M. H., T. J. Teisberg, E. P. Kahn. 1990. Why are Vickrey auctions rare? /. Political Econom. 98 94-109.

Siegel, S., N. J. Castellan. 1988. Nonparametric Statistic for the Behav ioral Sciences. McGraw-Hill, New York.

Strecker, S. 2003. Preference revelation in multi-attribute reverse

english auctions: A laboratory study. Internat. Conf Inform. Sys tems (ICIS). Association for Information Systems, Seattle, WA, 271-282.

Williams, S. 1999. A characterization of efficient, Bayesian incentive

compatible mechanisms. Econom. Theory 14(1) 155-180.

Article Contentsp. 1753p. 1754p. 1755p. 1756p. 1757p. 1758p. 1759p. 1760p. 1761p. 1762

Issue Table of ContentsManagement Science, Vol. 51, No. 12 (Dec., 2005), pp. i-iv, 1733-1902Volume InformationFront MatterIn Memoriam: Perwez Shahabuddin, 1962-2005 [p. 1733-1733]Management Control for Market Transactions: The Relation between Transaction Characteristics, Incomplete Contract Design, and Subsequent Performance [pp. 1734-1752]Better, Faster, Cheaper: An Experimental Analysis of a Multiattribute Reverse Auction Mechanism with Restricted Information Feedback [pp. 1753-1762]The Effect of Asymmetric Bidder Size on an Auction's Performance: Are More Bidders Always Better? [pp. 1763-1776]Investment Decisions and Time Horizon: Risk Perception and Risk Behavior in Repeated Gambles [pp. 1777-1790]The Effects of Imprecise Probabilities and Outcomes in Evaluating Investment Options [pp. 1791-1803]The Effects of Financial Risks on Inventory Policy [pp. 1804-1815]Skewness Preference, Risk Aversion, and the Precedence Relations on Stochastic Changes [pp. 1816-1828]The Effect of Payoff Feedback and Information Pooling on Reasoning Errors: Evidence from Experimental Markets [pp. 1829-1843]Pricing and Resource Allocation in Caching Services with Multiple Levels of Quality of Service [pp. 1844-1859]Simple Models for Multiattribute Choice with Many Alternatives: When It Does and Does Not Pay to Face Trade-Offs with Binary Attributes [pp. 1860-1872]Tailored Supply Chain Decision Making under Price-Sensitive Stochastic Demand and Delivery Uncertainty [pp. 1873-1891]Back Matter