Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies...
Transcript of Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies...
![Page 1: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/1.jpg)
Making fair decisions withalgorithms
Sam Corbett-Davies
with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel
![Page 2: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/2.jpg)
How do we identify bias in algorithmic decisions?
• Idea: consider how researchers have identified bias in human decisions
• Are algorithmic decisions meaning fully different from human decisions?
![Page 3: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/3.jpg)
Case study: Pre-trial decision making
• The decision—release or set bail?
• Judges’ mandate—ensure public safety and appearance in court while releasing as many defendants as possible
• Do judges exhibit racial bias in these decisions?
![Page 4: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/4.jpg)
Evidence from Broward County
![Page 5: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/5.jpg)
The benchmark test
• Compare the rates of detention between comparable groups
• Controlling for age and number of prior convictions, black defendants are 3pp more likely to be detained than white defendants
![Page 6: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/6.jpg)
Problems with the benchmark test
• Omitted variables• Decision makers almost always have more information than what is available
to the researcher (especially in decisions made face-to-face)
• What if disparities in detention can be justified by differences in risk?• Judges are trying to minimize the risk to the community while releasing as
many defendants as possible
![Page 7: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/7.jpg)
The outcome test
• Compare reoffending rates for released defendants only
• If one group reoffends less often, it suggests that judges are holding that group to a higher standard, discriminating against them
![Page 8: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/8.jpg)
The outcome test in Broward County
47% vs. 33%Released black defendants were more likely to
reoffend than released white defendants
Outcome test finds discrimination against whites!
![Page 9: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/9.jpg)
• Distribution of P(Y|X), where X is all information used by the judge
Risk distributions
Likelihood of reoffending if released0 1
![Page 10: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/10.jpg)
Risk distributions
Likelihood of reoffending if released0 1
![Page 11: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/11.jpg)
Likelihood of reoffending if released
Risk distributions
0 1
![Page 12: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/12.jpg)
Likelihood of reoffending if released
The problem with the outcome test
0 1
Reoffence rate for released defendants is the mean of the distribution conditional on being below the threshold
![Page 13: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/13.jpg)
Likelihood of reoffending if released
The problem with the outcome test
0 1
Outcome test suggests discrimination against blue!
Why? Because blue has many extremely low risk individuals
![Page 14: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/14.jpg)
The problem of infra-marginality
• People who are very low risk will be released whether or not judges are biased
• Infra-marginal statistics—such as reoffence rate—are affected by the relative number of very low risk defendants in different groups
• Discrimination occurs on the margin, when someone who wouldn’t otherwise be detained is detained because of their race.
![Page 15: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/15.jpg)
Identifying bias in human decisions
• Prejudice manifests itself as different thresholds:• The reoffending rate P(Y|X) differs between marginal individuals from
different groups
• In human decisions, we usually can’t measure the thresholds directly• We don’t observe all the variables in X, so we don’t know who’s on the
margin
• The marginal reoffending rate isn’t the same as the average reoffending rate among released defendants
![Page 16: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/16.jpg)
Making decisions with algorithms
• Used in many jurisdictions across the US
• Machine-learned mapping from defendant features to a risk score
• Attempts to predict whether a defendant will commit a new violent crime
![Page 17: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/17.jpg)
The promise of risk assessments
Kleinberg et al., Quarterly Journal of Economics
![Page 18: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/18.jpg)
Evidence from Broward County
![Page 19: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/19.jpg)
Potential fairness concerns
• Miscalibration
• Redlining
• Sample bias
• Label bias
• Subgroup validity
• Use of protected characteristics
• Disparate impact
• Disparate error rates
Systematic overestimation of a group’s risk
Discriminating by ignoring relevant information
Using unrepresentative training data
Measurement error in the training labels
Using features that are differentially predictive
Including race/gender in the model
When groups are detained at different rates
When false positive rates differ between groups
![Page 20: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/20.jpg)
Potential fairness concerns
• Miscalibration
• Redlining
• Sample bias
• Label bias
• Subgroup validity
• Use of protected characteristics
• Disparate impact
• Disparate error rates
Systematic overestimation of a group’s risk
Discriminating by ignoring relevant information
Using unrepresentative training data
Measurement error in the training labels
Using features that are differentially predictive
Including race/gender in the model
When groups are detained at different rates
When false positive rates differ between groups
Can be fixed without the need to sacrifice efficiency
![Page 21: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/21.jpg)
Potential fairness concerns
• Miscalibration
• Redlining
• Sample bias
• Label bias
• Subgroup validity
• Use of protected characteristics
• Disparate impact
• Disparate error rates
Systematic overestimation of a group’s risk
Discriminating by ignoring relevant information
Using unrepresentative training data
Measurement error in the training labels
Using features that are differentially predictive
Including race/gender in the model
When groups are detained at different rates
When false positive rates differ between groups
Present a fairness/efficiency trade-off
![Page 22: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/22.jpg)
Potential fairness concerns
• Miscalibration
• Redlining
• Sample bias
• Label bias
• Subgroup validity
• Use of protected characteristics
• Disparate impact
• Disparate error rates
Systematic overestimation of a group’s risk
Discriminating by ignoring relevant information
Using unrepresentative training data
Measurement error in the training labels
Using features that are differentially predictive
Including race/gender in the model
When groups are detained at different rates
When false positive rates differ between groups
Present a fairness/fairness trade-off
![Page 23: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/23.jpg)
Calibration
• Conditional on risk score, groups should reoffend at equal rates
• Principal way that criminologists assess the fairness of a tool
![Page 24: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/24.jpg)
Miscalibration in Broward County
• Women receive systematically higher risk scores than merited by their recidivism rate
![Page 25: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/25.jpg)
Redlining
• Calibration is often considered sufficient for fairness
• There are many risk scores that are calibrated, yet aren’t the best estimate of risk
• A biased decision maker could choose the calibrated risk score that most advantages their favored group
![Page 26: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/26.jpg)
Redlining
• So-named for the analogy with lending discrimination
• Banks ignored the individual creditworthiness of borrowers in minority neighborhoods
• Such a policy could still be calibrated, since minority neighborhoods might have posed a greater average default risk
![Page 27: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/27.jpg)
RedliningWhite Black
0 previous convictions 5% 5%
1-2 previous convictions 20% 20%
3+ previous convictions 40% 40%
Average recidivism rate 20% 20%
![Page 28: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/28.jpg)
Redlining
• Risk score: all whites receive the same risk, only use previous convictions for black defendants
• Risk score is calibrated, blacks and whites with a risk score of 2 recidivate at equal rates.
White Black
0 previous convictions 5% 5%
1-2 previous convictions 20% 20%
3+ previous convictions 40% 40%
Average recidivism rate 20% 20%
1
2
3
2
![Page 29: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/29.jpg)
Redlining
• Risk score: all whites receive the same risk, only use previous convictions for black defendants
• Applying a 25% threshold, only black defendants are detained!
White Black
0 previous convictions 5% 5%
1-2 previous convictions 20% 20%
3+ previous convictions 40% 40%
Average recidivism rate 20% 20%
1
2
3
2
![Page 30: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/30.jpg)
Redlining
• A general strategy to discriminate: add noise to risk scores of the favored group
![Page 31: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/31.jpg)
Why is calibration insufficient?
• Calibration is necessary but not sufficient for fairness
• Corollary: we cannot determine the fairness of an algorithm without inspecting the features used• Otherwise, the designer could have purposefully ignored features in order to
discriminate
• Using all available features avoids this problem, and produces the most accurate algorithm
![Page 32: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/32.jpg)
Sample bias
• The data set was sampled from the population in a biased way
• Over policing in minority neighborhoods results in predictive policing algorithms that focus on those neighborhoods
• Removing the bias is in the decision maker’s interests
![Page 33: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/33.jpg)
Label bias
• Overpolicing in minority areas means that if a white and black person both commit an offence, the black person is more likely to be arrested and appear as a “positive” in our data
• This is a problem: we care about crime, not arrest
• Criminologists find no evidence to suggest that, for violent crimes, P(arrest | crime) is greater for one race or another [Walsh 2009]• So we “arrested for a violent crime” is the least biased outcome
![Page 34: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/34.jpg)
Subgroup validity
• What about “bias” in the input features?• Drug arrests are biased, so we’d expect drug arrests to be less predictive of
future criminal behavior for minority defendants
• Will result in miscalibration
• If the labels are unbiased, we can easily fix with the appropriate interactions
![Page 35: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/35.jpg)
Subgroup validity
![Page 36: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/36.jpg)
Potential fairness concerns
• Miscalibration
• Redlining
• Sample bias
• Label bias
• Subgroup validity
• Use of protected characteristics
• Disparate impact
• Disparate error rates
Present a fairness/fairness trade-off
![Page 37: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/37.jpg)
Use of protected characteristics
• Legally questionable under current law
“…there may be cases where allowing an algorithm to consider protected class status can actually make outcomes fairer. This may
require a doctrinal shift, as, in many cases, consideration of protected status in a decision is presumptively a legal harm.“
Kroll et al., University of Pennsylvania Law Review
![Page 38: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/38.jpg)
Disparate impact
• When one group is detained more often than another
• “Statistical parity” – the requirement that all groups be detained at equal rates
• The decision should be independent of protected characteristics
![Page 39: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/39.jpg)
Disparate impact in Broward County
57% vs. 33%More black defendants than white defendantswere deemed high risk of committing a crime.
![Page 40: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/40.jpg)
Statistical parity as a measure of fairness
• Statistical parity is analogous to the benchmark test for measuring bias in human decisions, and has the same problems
• Differences in detention rates might reflect differences in risk
• Can partially mitigate with controls (conditional statistical parity), but what should we control for?
![Page 41: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/41.jpg)
Disparate error rates
• False positive rates should be equal for all groups
# Detained defendants who wouldn’t have gone on to reoffend
# Defendants who wouldn’t have gone on to reoffend
• ProPublica’s evidence of bias in COMPAS
![Page 42: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/42.jpg)
Disparate error rates in Broward County
42% vs. 22%Among white and black defendants who did not reoffend, more blacks were deemed high risk.
![Page 43: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/43.jpg)
Where do these disparities come from?
• The risk distributions for black and white defendants have different shapes
![Page 44: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/44.jpg)
The optimal rule is a single threshold
• Even if average recidivism rates are equal, a single threshold could produce:
• Disparate detention rates
• Disparate false positive rates
![Page 45: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/45.jpg)
The fairness/fairness trade-off
• The optimal policy that equalizes detention rates or false positive rates uses race-specific thresholds
• Thus, there is a tension between individual fairness (equal thresholds) and group fairness
![Page 46: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/46.jpg)
The fairness/fairness trade-off
![Page 47: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/47.jpg)
The fairness/fairness trade-off
Fairness ConstraintPercent of detainees that
are low risk Increase in violent crime
Equalize detention rates 17% 9%
Equalize false positive rates 14% 7%
• Low risk defendants were only detained because of their race, would bring 14th amendment case
![Page 48: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/48.jpg)
The fairness/fairness trade-off
Fairness ConstraintPercent of detainees that
are low risk Increase in violent crime
Equalize detention rates 17% 9%
Equalize false positive rates 14% 7%
• Increase in violent crime concentrated in minority neighborhoods, victims would also bring 14th amendment case
![Page 49: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/49.jpg)
Analogies to tests for discrimination
• As measures of fairness, disparate impact and disparate error rates face the same problems as their related tests for discrimination in human decisions
• Disparate impact = benchmark test• Both compare rates of detention
• Disparate error rates ≈ outcome test• Both compare infra-marginal statistics
![Page 50: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/50.jpg)
The problem with false positive rates
0.1 0.1 0.1 0.2 0.2 0.3 0.4 0.4 0.5 0.5 0.5 0.7 0.7
0.2 0.2 0.3 0.4 0.4 0.5 0.5 0.5 0.7 0.7 0.8 0.9 0.9
FPR
25%
42%
![Page 51: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/51.jpg)
The problem with false positive rates
0.1 0.1 0.1 0.2 0.2 0.3 0.4 0.4 0.5 0.5 0.5 0.7 0.7
0.2 0.2 0.3 0.4 0.4 0.5 0.5 0.5 0.7 0.7 0.8 0.9 0.9
FPR
25%
26%
0.1 0.1 0.1 0.10.1 0.1
![Page 52: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/52.jpg)
The problem with false positive rates
• The risk assessment algorithm hasn’t changed
• No defendant’s decisions has changed
• Yet the false positive rates tell totally different stories
![Page 53: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/53.jpg)
The problem with false positive rates
• The problem of infra-marginality returns!
• FPR is confounded as a measure of discrimination by the distribution of obviously high- and low-risk defendants
• Fortunately, in algorithmic decisions we can directly observe (and change) the threshold
![Page 54: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/54.jpg)
Making fair decisions with algorithms
• Ensure the following concerns are addressed(transparency is important)
• Miscalibration
• Redlining
• Sample bias
• Label bias
• Subgroup validity
• Then choose a single threshold
![Page 55: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/55.jpg)
Limitations
• Some decisions (eg college admissions) apply to groups rather than individuals
• Our analysis does not consider the long-term equilibrium
• Nor the possibility that some defendants could be more costly to detain
![Page 56: Making fair decisions with algorithms · Making fair decisions with algorithms Sam Corbett-Davies with Emma Pierson, Avi Feller, Aziz Huq, and Sharad Goel. ... •Removing the bias](https://reader034.fdocuments.net/reader034/viewer/2022050506/5f9771fa1c32a515fd44cc5f/html5/thumbnails/56.jpg)