EXPERIMENTAL ECONOMICS IN VIRTUAL WORLDS:

APRIL 7, 2008

ROBERT BLOOMFIELD: Welcome, everybody, to Metanomics business and policy in the

Metaverse of Virtual Worlds. Today we have three academic researchers joining us today,

one of my favorite types of sessions. Keep in mind that Metanomics is really intended to

serve four distinct communities, as we describe it on our Facebook profile, and I encourage

you to check it out there and join our Facebook group Metanomics. The four communities

are first, Real World Enterprises seeking to understand how they can use Virtual Worlds to

achieve their Real World strategic goals; virtual entrepreneurs who are identifying ways to

serve both Real World and Virtual World clienteles; World developers and regulators who

are looking at the forces and policies that are going to affect the future of the Virtual World

industry. And then finally, academics who are trying to understand and educate these other

communities or who are using Virtual Worlds as a laboratory for their research.

Now it turns out that this last group is pretty small right now because Virtual Worlds

research is, frankly, a pretty gutsy move for most academics. I have tenure, as does

John Duffy, our guest today from the University of Pittsburg, so we can afford to take a flyer

on this. If it doesn’t pan out, we’ve lost nothing but a little time. It’s not like it’s a major career

risk.

Our two other guests are taking a bigger chance. Thomas Chesney is a lecturer from

Nottingham University Business School and Steve Atlas is just completing his Masters

degree at Tufts University and is entering a doctoral program in business this fall. So these

two are taking much more significant risks, and I, for one, applaud them for it. If you know of

others who are taking these types of risks, please do point me to them. I’d love to be able to

have a lot more shows like this one.

Now we come to our long list of thank-you’s and announcements before our show. First,

thank you to your sponsors SAP, Generali Group, Cisco Systems, Saxo Bank, Kelly

Services, Sun Microsystems and my own institution, Cornell’s Johnson Graduate School of

Management. Of course, a big thank you to SLCN which is using some new technology

today. So if you are seeing us at one of our event partners or if you are seeing this on

SLCN.TV, you probably are seeing my lips move, which is a new and very exciting thing that

I’ve never experienced in Second Life, only in There.com.

I’d like to thank JenzZa Misfit, of Rendezvous, for letting Metanomics hold this event here on

Muse Isle. Muse Isle’s also going to be the location of a new regular event: Metanomics

Rewind, during which we’ll have a special rebroadcast of Monday’s show every Tuesday,

3:00 P.M., Second Life time. So this is a chance to catch the show. Any shows that you

miss, you can see them the next day. I know a lot of you in the U.S. find this a difficult time.

So come along, see the show and watch it with other people who share your interests, and,

hopefully, you can make some new friends.

I’d also like to say hi to those of you who are watching from Outreach Amphitheater of the

New Media Consortium Educational Community sims. New Media Consortium, NMC, is a

community of hundreds of leading universities, colleges, museums and research centers. I

think many of you probably know that NMC has been in the news lately because their CEO,

Larry Johnson, testified to Congress last week on the state of Virtual Worlds, along with

Linden Lab CEO Philip Rosedale and some others. Now, in my book, Larry wins the prize

for the best opening line in congressional testimony. He says, “My name is Dr. Larry

Johnson, and I have an avatar.” Well, my name is Rob Bloomfield, and I have an avatar,

Beyers Sellers, and I am very glad to have you all here for this show.

I also want to tell you very quickly about the other shows that are coming up. Next Monday

we have Cory Ondrejka, a former chief technology officer of Linden Lab, along with

Wagner James Au, former resident journalist, and they’re going to discuss the history and

the future of Linden Lab, Second Life and the Metaverse. We’re going to take a broader

Metaverse view the following week with Mitch Wagner, of Information Week, and

Steve Prentiss, from Gartner Incorporated, who are both very well versed analysts not just

of Virtual Worlds but the larger tech community in which they reside. Finally, April 28th we’re

going to take a trip into a Virtual World called Sophia’s Garden based on Quaq and will be

joined by Quaq’s CEO, Greg Nuyens.

If you have ideas for guests or topics, please do let us know. You can email me,

rjb9@cornell.edu. Send an IM in Second Life to Beyers Sellers, or you can get us through

contact at metanomics.net, our website. You can also follow us on Twitter and send us

messages, replies, on Twitter.

Finally, I do want to encourage everyone to join the Metanomics Group in Second Life. You

can do that right now. We use that for backchat, which is already cruising along right now I

see. So I and my guests will be watching the backchat during the show so that we can

respond to your questions.

Okay, now on to the show. For many of you, the first question may be this: What is

experimental economics? Well, the idea’s really pretty simple. You set up a small economy

under controlled conditions, manipulate some key variables that economic theory says

should matter, and then you compare what happens to the predictions of economic theory.

Professor Vernon Smith won the Nobel Prize in 2002 for laying out the basic tenets of

induced value, which he argued makes it possible to test economic theory in laboratory

settings. What does induced value mean? Well, it means we’re trying to get the people in

the economy that we are creating to actually have the incentives that economic theory

assumes. So the first tenet here really is rewards. There have to be rewards of some value

that participants can pursue within this small economic setting.

The second one is non-satiation, which is a two dollar word, which, I guess, would make it a

700-Linden word or so--excuse me--550. But it said non-satiation basically says participants

have to prefer more reward to less. They can’t ever be satisfied, no matter how much they

earn, which we say is very much the case in economic theory. Saliency means that the

rewards and how the games actions and outcomes map into those rewards must be clear

and apparent to the people in the economy. And finally, dominance, that the rewards have

to be large enough relative to anything else the participants care about, that what the

participants in the experiment care about really is the rewards that we are identifying.

So people have been doing experimental economics for many decades and in a lot of

different fields. So two of our guests today, Steve Atlas and Thomas Chesney, are

conducting experiments in what’s called behavioral game theory. They basically construct

very simple economic interactions and use a blend of economics and psychology to predict

what’s going to happen.

Our third guest, John Duffy, uses experiments to study macroeconomics, the behavior of

entire economies, with a fair bit of emphasis on bank runs and monetary policy. Or maybe I

just emphasize that because my own interests lie in that direction. I typically use

experiments to study financial economics, how financial markets behave and, in particular,

how they respond to financial reporting regulations and trading regulations.

Okay. Now, Thomas, I’d like to start with you. So first, welcome to Metanomics.

THOMAS CHESNEY: Thank you very much.

ROBERT BLOOMFIELD: We’re delighted to have you on the show. Now you recently put

out a working paper in which you describe your experience and the results of a number of

games that you conducted in Second Life. Let’s start with the simplest. You ran some

experiments for the Dictator game and the Ultimatum game. And, for starters, can you just

describe those games for our listeners?

THOMAS CHESNEY: Certainly. The Ultimatum game and the Dictator game are used to

examine the behavior surrounding fairness and altruism. So if we start with the Ultimatum

game, we bring in participants, and we pair them up, and we say to one of the two in the

pair, “We’re going to give you a stake. We’re going to give you some money, and you have

to decide how you want to split it between yourself and your partner.” And, of course, you

don’t know who your partner is. And once you’ve made that decision and communicate it to

us, we go to the partner, and we say, “This is the split that has been suggested. Do you

want to accept it?” And, if the person number two thinks that there’s a fair split, they will

accept it. And, if they think it’s an unfair split, they will reject it. And, if they reject it, people

get nothing. You go away empty-handed. So this is to examine fairness. And with the

Dictator game version, you go to the first person and you say, “Here’s the stake. How do

you want to split it?” And whatever they decide, you take it to person two, and you just give

them that amount so they don’t have the option of rejecting it. And that examines a number

of things.

First of all, it looks at whether or not the behavior in the original Ultimatum game was

strategic or not, whether or not it was genuinely altruistic or if people were just splitting it in a

way they thought was fair just so that they could get their split.

So we did this, and we tried to make our Second Life games as close to the real versions as

possible. So we went to a number of papers, and we took the method that was used in

those papers, and we tried to recreate it as closely as possible in Second Life to see

whether or not behavior in Second Life was as it was in Real Life. Because there are a

number of theories in the Information Systems--and I’m from the Information Systems

discipline. There are a number of theories that say that, in this sort of environment, people

will act differently. For instance, I could act hyper-selfishly because you don’t get to see me

blush. I can cheat you, and it doesn’t matter because you don’t really know who I am. So we

expect it--

ROBERT BLOOMFIELD: So, Thomas, when you say “hyper-selfishly,” let’s talk about that

Dictator game for a minute.

THOMAS CHESNEY: Mm-hmm. Yeah.

ROBERT BLOOMFIELD: So I come into your experiment. I’m the dictator. You give me--I’m

sorry, how many Lindens was it?

THOMAS CHESNEY: We choose 3,000 Lindens because that’s round about ten dollars,

which is what the Real Life versions use.

ROBERT BLOOMFIELD: So I’m given 3,000 Lindens. I get to decide how much of it I keep

and how much I give to you. And there’s nothing keeping me from giving you a Linden, if I

want to, so I end up with 2,999 Lindens?

THOMAS CHESNEY: Yeah, mm-hmm.

ROBERT BLOOMFIELD: So now I guess the first question is: What do people do in the

traditional laboratory experiment? And why do you think that would be different in a Virtual

World?

THOMAS CHESNEY: Yeah. Okay. In the traditional version, people tend to give about

20 percent of their stake, so you wouldn’t keep the whole thing. You would hand over

whatever 20 percent of 3,000 is. And I guess perhaps the reason they do that is because

they are face to face with the experimenter, and they’re in the same room as their partner,

even though they might not know which one of the group is their partner. But, in this

environment, I guess you could argue that people might act more selfishly because they

don’t get to see the experimenter’s face. They don’t have to look me in the eye and say,

“No, I’m not giving anything,” or, “I’m only giving one Linden.” So that’s why we might expect

behavior here to be different from the Real World.

ROBERT BLOOMFIELD: Now you found it was really pretty similar.

THOMAS CHESNEY: Yes, exactly. Statistically, there was no difference so people act the

same here as they do in the lab, in the real lab.

ROBERT BLOOMFIELD: So what’s your takeaway from that? I mean is your takeaway that

it’s safe to conduct these types of experiments in Second Life rather than in the controls of a

laboratory setting?

THOMAS CHESNEY: Exactly. Yes. That’s one our main findings. We think that it’s a

methodological contribution so if, for instance, somebody wanted to do some behavioral

game experiments on the cheap, Second Life is an environment where you could do that.

So for instance, I might want to do experiments with Spanish people, and there’s a big

Spanish community within Second Life, and I can go there, and I can be confident, or I can

refer to my results to try and imply confidence that those Spanish people will act as they

would have if I had set up a real lab in Barcelona and run the experiment there.

ROBERT BLOOMFIELD: And so that would definitely save you a lot of money in travel

expenses, doing it in a Virtual World to--

THOMAS CHESNEY: Exactly. Yes, exactly.

ROBERT BLOOMFIELD: --reach a global community. Now the other way to think about

doing these things on the cheap is that you can actually pay people less. So you paid

people roughly the same amount in--well, since Linden Lab maintains the Linden Exchange

steady to the U.S. dollar, we’ll keep this in dollars. And so you mentioned that you chose

about ten dollars to be split because that’s what is done typically in lab studies.

Now our second guest, Steve Atlas, has actually used a much smaller amount of money. So

first, Steve, it’s a little funny to say this, but, welcome to Metanomics.

STEVE ATLAS: Thanks, Rob. It’s a pleasure to be here.

ROBERT BLOOMFIELD: So I say that a little sheepishly since, in fact, for those who don’t

know, Steve Atlas--Curric Vida--has been working with me on Metanomics as he has been

also working on his Masters thesis at Tufts University. So that welcome is a little

tongue-in-cheek.

Now you also ran, well, a slightly different type of experiment, but it’s still behavioral game

theory. Can you quickly describe the type of game that you had people play, and also

maybe emphasize a little bit more of how you conducted your experiment? Because I know

you saved a huge amount of money, which is important for a Masters student.

STEVE ATLAS: Yes. I replicated the classic Berg, Dickhaut and McCabe 1995 study on

trust and reciprocity on a graduate student budget, where I had offered subjects a much

smaller than usual amount to split between them. The way this experiment works is that

subject A gets offered a certain amount of money. In this case, it was a hundred Lindens or

around 35 cents. Subject A was offered to send a certain amount of that money to the other

subject, who they did not know, who was anonymous and [AUDIO GLITCH] the experiment

much later. And that amount that they sent was triple, and then recipient B had the

opportunity to give some amount of money back to the initial participant who had given them

the money. Now neoclassic economic theory will predict that knowing that player B is going

to send nothing back to player A because they have no incentive to take that loss, then

player A isn’t going to choose to trust and to send any money to player B in the first place.

However, as was shown in the original lab study, that isn’t the case. Player A chooses to

trust sending, on average, five dollars and 16 cents of the ten-dollar endowment, and

player B chooses to reciprocate, sending around four dollars and change back.

ROBERT BLOOMFIELD: Okay. I’m a little confused by the numbers. I thought you were

starting out with much smaller numbers.

STEVE ATLAS: I’m sorry to interrupt there. So basically what I was talking about in the

splitting of the ten dollars was the classic Berg, Dickhaut and McCabe study. What I did in

this experiment was to provide only a hundred Lindens in contrast, and this allowed, through

providing a smaller amount of money per data point, as well as having an automated data

collection system, we were able to collect a massive amount more data. Specifically, it was

nearly 1,500 observations.

ROBERT BLOOMFIELD: Okay. Actually let’s just run through this experiment ourselves

really quickly to make sure that everyone is seeing what we’re talking about. So let’s start

with I get a hundred Lindens. So the experimenter gives me a hundred Lindens. I can do

whatever I want with it. I could actually put it in my pocket, and I’m done, and I have a

hundred Lindens. But, in fact, I might decide I’m going to give some to you, and, if I do that,

they get increased, and then you decide how much you’re going to give to me. So let’s say I

give you 50. Now what’s your decision?

STEVE ATLAS: Then that 50 that the original person gave gets tripled, and the other

person decides how much of that tripled, now 150 Lindens--how much of that to give back to

the original player.

ROBERT BLOOMFIELD: Okay. So what are you going to give me?

STEVE ATLAS: I will give you 51 since I’m so generous and want to ensure that you’re

better off than you had been.

ROBERT BLOOMFIELD: Okay. So unlike the Dictator and Ultimatum games that Thomas

was describing, here we can actually make the pie bigger. The more I give you, the more

wealthy we both are in combination. But then you basically, at the tail end of your game, you

have a Dictator game. You can just keep it all, if you want, but, hopefully, for my sake

anyway, you’d feel guilty. So the economics would say I don’t give you anything because I

don’t believe you’re going to give me anything in return. But, in fact, what we usually see is

trust and reciprocity. And I know, boy, hundreds--literally, right--hundreds of experiments of

this trust and reciprocity game have been conducted in laboratories. In fact, even in small

cultures where anthropologists have had to hack through brush and jungle to do this. I admit

I’m forgetting the name of the guy from Emory University study.

THOMAS CHESNEY: Was it Heinrich?

ROBERT BLOOMFIELD: Yes. Yes, that’s right. It was Heinrich. So he actually has had a

team of anthropologists doing this study around the world, and results are reasonably

consistent. Now one of the interesting things about your study, Steve, is that you offered so

little money. Now your results, how would you compare them to what we see in the

literature?

STEVE ATLAS: In general, the results were very consistent. They appeared slightly lower,

but not in a statistically significant way. So in general, I would say that there was no

statistically different outcome when they’re asked the same question online. I’m sorry. What

is that?

ROBERT BLOOMFIELD: Oh, go ahead.

STEVE ATLAS: One of the interesting things that we were able to do, however, was to take

a look at the effect of age on how people make this trust and reciprocity decision. We did

find an interesting significant relationship between age and how people--or at least reported

age, and what they were going to say to this trust question. We found that 18- to

22-year-olds, which would be the age group that would be typically tested in a normal trust

experiment that would be done on undergraduates, actually, in fact, trusted less than the

older population. So this supports the oft cited concerns that there is some degree of

age-related bias by sampling undergraduates. What’s particularly interesting is that this

difference was in the opposite direction of what would be expected. Typically

undergraduates are seen as being naïve and might be trusting more, whereas, the

sophisticated response is to not be putting any money into this trusting account. However, in

this particular case, they were actually the ones who were trusting less than the older

population, which was an interesting and surprising finding.

ROBERT BLOOMFIELD: Now I’m seeing some questions coming in through the backchat,

and one of them is from Joia Sands. She just says quite simply, “I don’t see why they think

that behaviors would be different in a virtual environment versus Real Life.” So now Thomas

mentioned some of the issues with face-to-face interaction, though I know a number of

people have tried to deal with that in laboratory settings. Steve, why did you expect a

difference in your study?

STEVE ATLAS: Well, one of the differences is demographic in nature. The lab experiments

you will often have [AUDIO GLITCH] college students tested there, unless it is specially

designed to bring in others. College students are often used because of the convenience

and easy access because they’re very controlled, in a sense, that you’re going to have of

the same draw from the population. So they’re going stay about to the constant over time. In

a Virtual World, it’s just demographic [AUDIO GLITCH]

ROBERT BLOOMFIELD: Now, John Duffy, welcome to Metanomics.

JOHN DUFFY: Hi.

ROBERT BLOOMFIELD: Hi. One reason it’s great to have you on is that you actually

participated in Steve’s study. You created an avatar. You went in and just acted exactly as a

subject, and you wrote about it in your paper called Trust in Second Life. You expressed

some concerns about the study and about experimental work in Virtual Worlds in general.

You want to just tell us a little about that?

JOHN DUFFY: Yes. I’m interested in doing experimentation in Virtual Worlds. And so one

way to find out about it, I thought, was to go to some laboratories, and I visited several, and

Steve’s was one of the experiments that I participated in. I think it’s an interesting place in

which to do research. I have doubts that it will lead to any significantly different findings, and

that’s, I think, what both Thomas and Steve are showing us. But I do have some concerns

about the control that’s offered in Second Life relative to physical laboratories.

ROBERT BLOOMFIELD: What are the key concerns that you would have? Go ahead.

JOHN DUFFY: Well, I think there’s several issues. One is the recruitment of subjects, who

are already expressing a preference for a leisure activity. I think opportunity costs here

would be very high. Chesney, et al., for instance, say that the internet message boards

didn’t generate much interest. They had to basically teleport willing subjects to their

experiments just prior to the start time, and that leads to issues of association, lots of people

may be in the same session. They may behave more nicely to one another because they

know each other, these sorts of things.

But another issue is the reliability of data collected may be quite low. This is especially true

of demographic data such as data on gender, race, income and age, where it’s known that

pretending to be someone other than you are is quite the norm on Second Life.

The third issue is common knowledge, where experimenters seek to minimize any

non-strategic uncertainty by publicly reading instructions at the start of an experiment. This

not only informs subjects of the rules of play, but also makes the game structure public

knowledge, the closest thing possible to inducing common knowledge in the laboratory.

Having subjects read instructions anonymously, as I did in Steve Atlas’s trust game, doesn’t

really inspire confidence that the players I’m matched with all had the same instructions, nor

does it inspire confidence that there is no other player in the room with me in which to

engage in trust and reciprocity.

Another issue is collusion. There’s very little control over communication protocols. IM-ing is

possible between subjects. People can be logged in with multiple avatars. And finally, I

would say there’s demand effects, in the sense that you’re telling your choices to the

experimenter--well it's mostly to this program that’s running the experiment, and I think that

that changes the nature of interactions as was shown in the Chesney, et al., study.

ROBERT BLOOMFIELD: Yeah. I see you’re getting some people who are agreeing with

you. Venus Jervil is actually saying the verification of the data in general in Second Life is a

problem and something we’ve talked about before on this show with some of our marketing

research folks. Also, I do want to mention that, let’s see, CeAire Decosta is asking, “Why are

they calling the participants “players” and Second Life a game?” And I think this is probably

a good time to point out that what the panel members are doing here, well, in particular,

Thomas and Steve, is creating a game within Second Life. So rather than saying Second

Life is itself a game, you can think of this a little more like creating some type of game within

Second Life that people can play, and then they become players just like the giant snails

racing on SLCN’s giant snail races. You can call them the players or the racers. So

hopefully that will clear that up.

Now I was going to ask Steve if he would respond to those comments, but I see he has

taken a little trip, this trip that Second Life seems to be sending lots of us on the last few

days, and, hopefully, he will be back momentarily. Until we get there, Thomas, what’s your

reaction to John’s concerns about conducting experiments in Second Life?

THOMAS CHESNEY: Well, certainly, they all have validity, without question. We did take

steps to try and minimize these problems. For instance, at the start of all of our games,

everybody was given a note card, exactly the same with instructions on it. And then there

were questions at the end to make sure that they had understood those instructions, and the

game didn’t proceed until we had correct answers from everybody.

ROBERT BLOOMFIELD: If I can stop you on that one. John, correct me if I’m getting your

point wrong, but I think John’s concern was not that people would misunderstand the

instructions, but that, in theory, one person could be getting one set of instructions, another

person could be getting a very different set of instruction. And so typically in a laboratory

setting you would actually have someone stand up in front of the whole room and read

something aloud, and everyone can see everyone hearing the same thing. Is there a way?

JOHN DUFFY: That’s the issue.

ROBERT BLOOMFIELD: Is there a way, really, to deal with this in a Virtual World or, for

that matter, any online environment?

THOMAS CHESNEY: Well, for instance, we could use voice like we’re doing now, and

somebody could read the instructions out to everybody so that everybody knows they are

getting the same experience. No, we didn’t want to do that because we didn’t want to

introduce factors such as tone or voice that somebody could say might be suggestive,

although that wouldn’t really matter for reading out the instructions. Or, alternatively, you

could put the instructions up on a poster on the wall, and then everybody knows that they’ve

read the same instructions. So perhaps there are ways around it, but, yes, that is an issue,

and it was an issue with our games. We could have been giving out different instructions to

different people, and they wouldn’t have known it.

We did take steps to minimize collusion. We programmed some lab chairs and, as soon as

you sat in them, it would alert us whether or not you had IM’d somebody. So we couldn’t

prevent collusion, but we could detect it. And the point about lies, yeah, absolutely. The

demographics that we collected--the people could have been lying to us. Now we should

point out that that could happen in a real lab as well. Yes, you might have more control over

it in a real lab, but still you could be lied to. So yeah, it’s not a perfect substitute for a real

lab, but there are advantages and disadvantages.

ROBERT BLOOMFIELD: Oh, go ahead.

THOMAS CHESNEY: That’s me. I’m done.

ROBERT BLOOMFIELD: Okay. Steve, are you back with us?

STEVE ATLAS: Yes, I am back, and I did hear Professor Duffy’s--

ROBERT BLOOMFIELD: I’m not a hundred percent sure of this, but I believe that you were

here long enough to hear John describe his concerns. And then right when I was going to

ask you of your responses, you impolitely left. So would you like to react to any of them?

STEVE ATLAS: Yes, actually I would. What an inconvenient time for a crash to take place. I

did hear what Professor Duffy had to say, and I’ve been very interested to hear his critique

of my work so far. Ultimately, there are special considerations and concerns that come into

play when conducting experiments online. One of the things in particular that got pointed out

were concerns about collusion and about subjects knowing who they were partnered with.

But one of the ways that we had control for this in our experiment was by having a delay

mechanism in play, in the chair, so that subjects really had no way of knowing who they

were going to be partnered with. The way that we did it is that we pulled several player A’s

in a row and then pulled player B’s later on so that players didn’t know who they were

actually going to be partnered with, and there was anonymity preserved there. Whether or

not subjects believed they knew who they were partnered with falsely is a different concern,

a different issue.

And the reality is that there are a lot of other concerns that are at play when analyzing this

type of research, but it’s also important to note that these methods are still in their nascent

stages. So I think the solution is to try to identify exactly what the problems with initial

studies are and try to improve the infrastructure so that we can collect better data as time

goes on. I think that there are certain unique benefits of conducting research in Virtual

Worlds, that aren’t necessarily lost even if there are some data quality and control

considerations. At the very least, we know that Virtual Worlds provide the ability to run pilot

studies in a much more cost-effective way before spending a significant amount, more

resources to do it in a real lab at a higher cost. Secondly, it would allow access to

conducting experiments to individuals who might not otherwise be able to get the significant

amount of funding necessary to [AUDIO GLITCH] physical lab.

ROBERT BLOOMFIELD: Now about how much in total did this thesis research cost you?

STEVE ATLAS: In total, including the false start of creating lab infrastructure that was

supposed to work in World of Warcraft that ultimately didn’t pan out, even with a couple of

false starts, this was all conducted in under a thousand dollars. And this is about 1,500 data

points that were collected under that budget.

ROBERT BLOOMFIELD: Yes. And so if we were to think about the numbers more

consistent--well, it seems like the primary savings that you had was probably with the size of

the stakes. So you had 750 groups? Is that it or pairs of subjects?

STEVE ATLAS: Yes, that’s correct.

ROBERT BLOOMFIELD: So at ten dollars a pair even, all of a sudden you’re talking $7,500

so that’s quite a significant savings. It does seem like it would be nice if we could figure out

how to deal with John’s concerns while keeping the money-saving aspects here.

One of the big concerns that has come up that I guess we haven’t really talked about too

much is, I guess, what you might call experiment farming. So if someone really wanted to try

to exploit the fact that you’re giving away free Lindens, they could either be creating alts or

just telling their friends to do it and don’t pay attention to what you’re doing. Just click

around and get some money and go away. Games deal with that problem all the time. Do

you have any specific thoughts, Steve, on how you can avoid that sort of experiment

farming?

STEVE ATLAS: What’s really interesting is that, in these Virtual Worlds, participating in an

experiment can actually be a relatively high paying job relative to what other options are out

there. Why exactly participants are willing to accept such low stakes for their time clearly

has something to do with the fact that they’re getting some enjoyment out of use of

experimenting something new and different. But the reality is that, if you’re either offering

even a small amount of money for very little time or a large amount of money for a lot of

time, that’s such an attractive offer that, even if you insert controls as we did that prevented

more than--basically that player to participate under the same avatar more than once, what

users were doing since they’re very savvy is that they were logging out and then logging

back in under the guise of a different avatar and participating again. So that can result in

another source of bias and problems with the data that we wound up collecting. Ultimately,

it’s worse with the more you pay and the less you’re asking in return. So I think there have

got to be some ways of getting around that kind of thing, and I’m not sure if they’ve been

discovered to their full extent yet.

JOHN DUFFY: Can I jump in and say something?

ROBERT BLOOMFIELD: Oh, yes. John, please.

JOHN DUFFY: I think as a possible solution is to get away from these two-person games

that have been studied in the laboratory where there are issues of collusion and

communication and move towards larger-scale games that really take advantage of the

scale of Virtual Worlds like Second Life. And here what I have in mind are experiments

involving asset pricing or voting where we really can’t do justice to the environments we’re

hoping to study theoretically or in the laboratory, without some input from the decisions of

many hundreds, maybe thousands, of participants. And in those settings, you would be

much less concerned about issues of collusion communication because the payoffs from

those activities would be much reduced.

ROBERT BLOOMFIELD: Yeah. I think that you get at a point that I think is even larger than

just making collusion harder, which is that given that both Thomas and Steve are conducting

experiments to replicate what was done in a small laboratory naturally these are

experiments that are well suited to the laboratory. And so it does seem to me that, while it’s

nice to save a little money, that’s probably not the most important thing that we can be doing

when we come to Virtual Worlds to conduct research. And especially, John, when I think of

the types of research that you do typically. You’ve been studying macroeconomics in a very

small laboratory setting, and, boy! maybe Virtual Worlds give you the opportunity to study

macroeconomics with very large groups of people that are really going to bring into play

some of the forces that you just can’t get at in these small groups. So let me ask John:

Since you have so many different research streams, is ultimately your interest to take

macroecon into Virtual Worlds?

JOHN DUFFY: Well, I do think that what Steve and Thomas are doing is very useful. It will

pave the way for doing larger-scale experimentation if there is a consensus that there is

very little difference in the behavior of Second Life participants in two-player games and

those of participants in Real World laboratory studies. So I do feel that it’s a necessary step

and that it’s good that they’re finding this. But I do think that the capabilities of Second Life

have not even begun to be explored. Or other Virtual Worlds as well. And, yes, I am

interested in studying asset pricing. Voting is an issue where you can’t study issues of

turnout in electorate sizes that approximate those of real elections in the laboratory.

But let me give you another instance where I think Second Life could be very useful, and

that is in experiments where subjects are asked to choose various effort levels. And the way

we typically do this in the laboratory is, you click a button according to effort level one

through seven or something, and there’s a cost associated with each. But essentially it’s

effortless effort.

ROBERT BLOOMFIELD: So instead of actually seeing how hard it is, how much effort it

takes to do something, we say, “Are you willing to pay a dollar?”

JOHN DUFFY: That’s right. Yeah. You could actually design experiments where effort

involved some task that had to be performed, and more effort meant just that the task was

more onerous or took more time. You could really think about real time too. In the

laboratory, we’re always concerned about how long we could keep subjects. That’s an issue

on Second Life as well, but, if you structure the experiment correctly, you could induce a

time or an onerous burden associated with higher effort levels, and that’s something where

you could really take advantage of the environment better.

ROBERT BLOOMFIELD: When you mention effort to me, and here we are in Virtual

Worlds, it makes me think of what is often called the grind in game worlds, like World of

Warcraft where, I think, everyone who plays these games knows that there’s a tremendous

amount of effort that you have to go through in order to level up.

Now, Steve, you mentioned that you had a false start. You were going to run these

experiments in World of Warcraft, which actually seems like you would have lots of

opportunities to conduct studies along the lines of what we’re describing. Why didn’t that

work out?

STEVE ATLAS: Well, one of the reasons why I had initially gone into that was because

basically we were going to exploit the black market premium, and the hope was that we

could get even more--essentially we could buy a ton of World of Warcraft gold for a

relatively low cost and then compensate participants in what felt like an awful lot of money.

However, in doing so, in exchanging that money through black market channels, since there

is no official currency exchange mechanism between currency in World of Warcraft and U.S.

dollars, in order to do that we placed all of our participants at risk of having their accounts

terminated. Consequently, that did not go over well with the IRB at Tufts and had to change

venues, in order to do it in a space that would not place participants at risk.

ROBERT BLOOMFIELD: Actually I’m glad you brought up the IRB. So for those who don’t

conduct research that is either taking place at universities in the U.S. or funded by federal

bodies you may not know, the IRB is the Internal Review Board, an organization we have to

make sure that research is being conducted in line with standards of safety and disclosure.

And so actually let me ask you, Steve, did you have any unusual IRB issues in getting

approval for this research at Tufts?

STEVE ATLAS: Other than having to switch venues, that was most significant one. I think

their concern about that was actually quite valid. I don’t think that that means that it would

have been totally impossible to use that space. We just would have to do it in cahoots with

the game developers [AUDIO GLITCH]

ROBERT BLOOMFIELD: I see Allister Beerbaum is asking whether you approached

Blizzard, the maker of World of Warcraft, directly, to see if they would help you out. Did you

try that?

STEVE ATLAS: Yes. I wrote a half dozen emails to them, didn’t get any responses.

ROBERT BLOOMFIELD: Thomas, I’m less familiar with how you need to conduct research

and get approvals in the UK. Can you just tell us a little bit about any unusual challenges

you faced doing that with a Virtual Worlds study?

THOMAS CHESNEY: Certainly. We didn’t have any problems at all. The game theory

research project didn’t have to go through an ethics committee at all. We did another

project, and there was a psychologist on the team there, and he had to put the project

through their ethics committee, but, no, we don’t have an equivalent to your review board.

ROBERT BLOOMFIELD: Okay. Well, actually, I read in your paper, Thomas, you did

mention there was just a little throwaway line regarding Linden Lab’s guidelines for

researchers.

THOMAS CHESNEY: Yes, that’s right.

ROBERT BLOOMFIELD: So can you talk a little about those?

THOMAS CHESNEY: Certainly. In actual fact, Linden don’t require this anymore, but it

used to be, when we started the project, Linden required all researchers to submit proposals

to them and get approval from them before we could do anything. So yes, you’re right, we

did do that, but you don’t need to now. If you were starting the project tomorrow, you

wouldn’t have to do that.

ROBERT BLOOMFIELD: Okay. And another question. And let’s see, this was from

Joia Sands: Were participants participating blind in this study? Or did they know what the

study was about, that it was an economics study? And I guess, why don’t we start with you,

Tom?

THOMAS CHESNEY: With me? Sorry. Did you say?

ROBERT BLOOMFIELD: Yes.

THOMAS CHESNEY: We told them that it was a university project, and it was looking at

behavior. So we invited people to come along for an experiment, but we didn’t mention

economics, and we didn’t tell them what would be involved other than it was a

decision-making task and that they could make some money, and we’d give them the min

and the max amount that they could make.

ROBERT BLOOMFIELD: And, Steve, [AUDIO GLITCH] you?

STEVE ATLAS: [NO RESPONSE]

ROBERT BLOOMFIELD: I think we may have lost Steve for a little bit there. Let’s see. I see

that--actually we got a late start, and we are actually getting to the end of our time. An hour

goes quickly in these sessions. Let’s see. I guess I would like to encourage right now if there

are any more questions from the audience, and there have been quite a few. I’m just

scrolling back to see how many of these I may have missed. I’ve probably missed a lot. So

let’s see. And also, just from our guests, I’m wondering what you might like to mention at the

end of our show that you haven’t had a chance to. John, shall we start with you and any

observations that you’d like to make in closing?

JOHN DUFFY: Just to reiterate what I mentioned before. I think that emphasizing that

Second Life is a cheap place to do experiments is not the way to go. I think what needs to

be said or discovered is to use Second Life to do a very different kind of experimentation

than is done in the laboratory. And obviously, you need to build a bridge between laboratory

and Virtual World experience [AUDIO GLITCH] at least to what other researchers are doing.

But I think the real payoff will come when someone figures out how to make the best use of

Virtual World environments for experimentation. I suggested a couple of ways, and I’m

thinking about it. I’m very interested in issues of monetary policy, for instance, and how you

would even go about doing this, I don’t know yet, but that’s something where you would use

the entire scope of the environment. You would have to play around with the laws by which

money is injected into the system, an epic kind of experimentation you certainly can’t do in a

laboratory, not with much empirical validity. And you can’t do it in the Real World. So that’s

what I’m looking forward to.

ROBERT BLOOMFIELD: Steve?

STEVE ATLAS: I would say that there’s one other scope of research that is very strongly

related to conducting economic experiments, and that would be working with the game

developers themselves possible to exogenously change certain macroeconomic variables in

a controlled way. And, in doing that and then measuring results of that, we might be able to

come up with evidence to either support or refute existing theories about how entire

economies operate. And, in doing so, we could do it in a way that is much less costly to

actual people, by inconveniencing the residents of Virtual Worlds with a small amount, it

would be preferable in terms of welfare impacts than to say try other Third World countries

and changing the economic conditions there. So that’s one other area that is related to what

we’re talking about here where the types of tools that economists use could be particularly

valuable to do on Virtual Worlds themselves rather than in the Real World.

ROBERT BLOOMFIELD: Okay. And, Thomas, anything you’d like to say?

THOMAS CHESNEY: Certainly. I think it’s fantastic that this discussion is going on and that

we are considering these issues. And the future for this does actually look quite exciting. I

mean, for instance, where we’re going with our experiments is, we are trying to combine two

things that have been mutually exclusives before Virtual Worlds came along, which is the

realism of a field experiment with the control of a lab experiment. So we are trying to use the

environment as Professor Duffy suggests, and, yeah, I think things are exciting.

ROBERT BLOOMFIELD: I have been studying this space for, I guess, over a year now,

and I think it’s very exciting. I think it’s clearly more challenging than I thought it was going to

be. But let me just echo some of the things that you guys have said in closing.

Starting with John’s comment that the way to think about it is not that this is going to be

easier or cheaper, and it’s sort of standard advice to doctoral students as they start

presenting their research to faculty and as they respond to peer reviews, that whenever

they’re asked why they made a choice, they shouldn’t be saying, “Oh, well, that was easier,”

because, frankly, no one cares how hard doctoral students have to work. In fact, the answer

needs to be, “Because it was the better way of doing it.”

And so the question that we need to be addressing is really: What is it that Virtual Worlds

are the best thing for examining? And I think that it really does come down to when I think of

the advantages of Virtual Worlds, what we have are lots and lots of people who are looking

for things to do. So Steve is right that we don’t need to be paying Real World wages to

people who are in a place like Second Life, looking for entertaining content. And as much as

experimental economists will deny it, participating in economic experiments is a lot of fun.

There’s a reason that they’re called games. It’s not just a technical theoretical term. They

really do give people challenging decisions to make. And so--yes, and I agree, I see Venus

is saying, “but you have provide a reward.” Absolutely. But experimental economists are

also having to provide a lot of money just to get butts in the seats in their labs. And the butts

are already here in Second Life looking for a good seat to sit down in and do something

interesting.

So the other thing that we’ve got in Virtual Worlds is very complicated institutions. And

certainly in business, in macroeconomics, in financial economics, really these experiments

are about how psychology and economic forces meet complicated institutions.

Very quickly, let me describe a study that I am talking about doing with a doctoral student

here at Cornell. The idea is that we’re going to create what we’re calling synthetic firms, little

firms that people are working together to run. They’re going to be Cornell students who will

work on those here at Cornell, actually running the firms. And, like so many businesses in

Second Life, they will actually be issuing stock on a stock exchange in Second Life. Unlike

the virtual businesses, they will be audited, and everything will be above board. Everything

will be trustworthy. We hope to get thousands of people in Second Life to be trading the

shares of these synthetic firms, and our hope is to manipulate the financial reporting

mechanisms that people can [AUDIO GLITCH]. So the people running the firms will be

using different methods of financial reporting to explain to the investors how they’re

performing, what they’re doing, what the risks are, and we’ll be able to see, in a much better

way than we can in a laboratory, exactly how it is that financial reporting standards affect

firms’ ability to communicate with the stock market and raise capital.

So sorry I went on at such length in that, but I just wanted to get across the idea that I think

this is one way to use the complex rich environment that Second Life allows, in order to look

at issues that you just couldn’t do with a small number of people in a small lab for a short

amount of time.

And, in answer to CeAire Decosta’s question, “Didn’t Linden Lab already shut that type of

thing down?” our plan is to make sure that this is--and these I think are challenges everyone

is going to have to worry about, as they do economic studies in Virtual Worlds--they’re going

to have to make sure it’s not gambling. They’re going to have to make sure that it is not

inappropriately unregulated financial services. So we may be going to the SEC, for example,

looking for a letter of no action, as the Iowa electronic market did in doing very similar

research.

So there are ways to do this. I think it’s going to be a very exciting and very challenging

decade for experimental economists as they figure out how to use Virtual Worlds to do the

type of research that I think really represents the next generation of experimental

economics.

So I think I will answer Pandora Bailey’s question of whether share trading is gambling,

which it is not. I will answer that after the session. But, for now, let me just thank everyone

for coming to Metanomics. I’d like to thank our guests Thomas Chesney--

THOMAS CHESNEY: Thank you. Thank you very much.

ROBERT BLOOMFIELD: --Steve Atlas and John Duffy. I hope we will all have a chance to

talk and figure out how to move this field forward. So bye bye, everyone, and see you all

next Monday.

JOHN DUFFY: Thanks.

STEVE ATLAS: Thanks a lot.

Document: cor1015.doc

Transcribed by: http://www.hiredhand.com Second Life Avatar: Transcriptionist Writer