Responsible Crowdsourcing -...

1 Crowdsourcing: From Theory to Practice and Long-Term Perspectives, Dagstuhl, September 1-4, 2013

Responsible Crowdsourcing A short set of slides towards operationalizing ethical practices in online microtask marketplaces

Martha Larson Delft University of Technology


What is ethics?

“The field of ethics (or moral philosophy) involves •  systematizing, •  defending, and •  recommending

concepts of right and wrong behavior.”

“Normative ethics takes on a more practical task, which is to arrive at moral standards that regulate right and wrong conduct. This may involve articulating the good habits that we should acquire, the duties that we should follow, or the consequences of our behavior on others.”

According to the Internet Encyclopedia of Philosophy


Normative Principles in Applied Ethics

• Personal benefit: acknowledge the extent to which an action produces beneficial consequences for the individual in question.

• Social benefit: acknowledge the extent to which an action produces beneficial consequences for society.

• Principle of benevolence: help those in need. • Principle of paternalism: assist others in pursuing their

best interests when they cannot do so themselves. • Principle of harm: do not harm others.



Normative Principles (continued)

• Principle of honesty: do not deceive others. • Principle of lawfulness: do not violate the law. • Principle of autonomy: acknowledge a person’s freedom

over his/her actions or physical body. • Principle of justice: acknowledge a person’s right to due

process, fair compensation for harm done, and fair distribution of benefits.

• Rights: acknowledge a person’s rights to life, information, privacy, free expression, and safety.



Defining Crowdsourcing

• A general definition: Crowdsourcing is the process of producing value or services in a manner that takes advantage of both human intelligence and human multiplicity. What needs to be done and who does it come into connection outside of the locus of a conventional workplace, where specific tasks are delegated to specific agents at specific locations.

• A specific definition: Here, we focus on crowdsourcing platforms, i.e., online microtask markets where crowdworkers can carry out microtasks for taskaskers in exchange for micropayments.


Why should we care about ethics?

• As taskaskers, we ourselves are actors in crowdsourcing eco-systems and stand to directly benefit from an ethical environment. (Benefits could be: sustainability, reliability, greater sense of satisfaction with our work.)

•  If crowdsourcing can remain mutually beneficial for taskaskers and crowdworkers it will co-evolve with positive public perception and enabling (rather than restrictive) legislation.

• The work of Robert Axelrod (The Evolution of Cooperation 1984) gives cause to question assumption that greed pays in the long run.1

1 http://en.wikipedia.org/wiki/Prisoner%27s_dilemma


When should we care about ethics? (Always)

•  In cases where crowdwork income is not essential for the crowdworkers, we can ask the questions: •  What are people not doing when they are crowdsourcing? •  Does crowdsourcing have an addictive side?

•  In cases where people are subsisting on crowdwork income, we can ask the questions: •  Is the crowdwork paying “only” money, without building a longer

term future? •  Is the money enough? Is the timing of the payment ok?


Suggestions for taskaskers striving to be ethical individuals

• Understand all roles in the community. Try some crowdwork. • Do not reveal identity of the crowdworkers. • Address crowdworkers politely (and apologize if you goof up). • Don’t engage in the “race to the bottom”, offering less and less

reward for your tasks. • Review work promptly. • Create well designed microtasks that are engaging and ergonomic. • Realize that asking crowdworkers to carry out tasks on a microtask

platform requires an investment of your time not only money. • Respond to inquiries (even the ones with horrible grammar). • Use contracts to clearly define your relationships (LbyL1) • Be open and honest (LbyL1)

1 Stephen M. Wolfson and Matthew Lease. Look Before You Leap: Legal Pitfalls of Crowdsourcing. ASIST 2011.


Suggestions for taskaskers striving to be ethical as a group

• When you think or talk about crowdworkers behavior, realize that there is more than one system involved in determining what is ethical and non-ethical behavior on the platform. (Reflect that there is widespread consensus that it is ethical to steal a loaf of bread if your child would otherwise starve.)

• Give people the information that they need to make informed decisions about the use of their work and their information. (Don’t ask crowdworkers to carry out tasks that are used for questionable ends.)

• Understand the past. •  Protect health-state, competence, and personality and any other

personal information of crowdworkers like it was priceless art.


More challenging ways towards responsible crowdsourcing

• Understand the past (see slide 11). • Think beyond instant gratification (see slide 12). •  Engage in ethics by design (see slide 13). •  Spend lots of time following guidelines for human subject

research. •  Spend lots of time doing “non-science” activities, like joint task

forces to improve and innovate crowdsourcing practices. • Be close to workers, where close can be 1st, 2nd, 3rd degree

relationships. • Mirror the ethics mechanism of the real world on the

crowdsourcing platform.


Learning from past lessons

•  Employers generally have more social and economic power than the people they employ. This asymmetry opens opportunities for abuse.

• Assembly-lines offered a breakthrough in production at the end of the 19th Century, but were also associated with sociological problems. Human-being-as-machine is historically a problematic metaphor.

•  Societies have fought hard in the past, and are still fighting to eliminate child labor. Taskaskers should be alert for evidence that their tasks are being carried out by children.

http://en.wikipedia.org/wiki/File:Mill_Children_in_Macon_2.jpg http://en.wikipedia.org/wiki/File:Ford_assembly_line_-_1913.jpg


Beyond instant gratification

•  “No one is complaining” doesn’t necessarily mean we have achieved an ethical environment: What about five to ten years from now?

•  In the moment a cigarette is nice and we are oblivious to consequences. (Think about crowdworkers developing repetitive stress disorder or becoming stuck in a self- reinforcing hand-to-mouth lifestyle.) It took years for legislation to start protecting consumers from lack of awareness of the long term consequences of smoking.

Flickr: Patrin

Ethics with a very broad horizon


Ethics by design

• The value of a fair-trade product is not only its inherent worth, but also how it is produced. In order to support fair-trade crowdwork, each microtask unit should be associated with certification information.

Architectures where the right information flows

Flickr: jetalone

• Crowdworkers can feedback information and self organize if they can communicate among themselves. Every party on a crowdsourcing platform should be able to reach all other parties. Enforcing isolation is not the answer to ensuring that crowdworkers work independently on tasks where independent opinions are critical.


References

•  J. Zittrain “Minds for Sales” http://www.youtube.com/watch?v=Dw3h-rae3uo

• Ross, J. Et al. Who are the crowdworkers? CHI 2010 •  Felsteiner, Alek. Sweatshop or Paper Route?: Child Labor

Laws and In-Game Work. CrowdConf 2010. •  Stephen M. Wolfson and Matthew Lease. Look Before You

Leap: Legal Pitfalls of Crowdsourcing. ASIST 2011. • Kittur, Aniket et al. The future of crowdwork. CSCW 2013. •  See also:

M. Lease “Ethics and Crowd Computing” http://www.slideshare.net/mattlease/crowdsourcing-ethics-a-few

Responsible Crowdsourcing -...

Documents

Transcript of Responsible Crowdsourcing -...