Factful: Engaging Taxpayers in the Public Discussion of a Government Budget
-
Upload
juho-kim -
Category
News & Politics
-
view
103 -
download
1
Transcript of Factful: Engaging Taxpayers in the Public Discussion of a Government Budget
![Page 1: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/1.jpg)
Factful: Engaging Taxpayers in the Public Discussion of a Government Budget
Juho Kim (MIT)Eun-Young Ko (KAIST)Jonghyuk Jung (KAIST)Chang Won Lee (KAIST)Nam Wook Kim (Harvard)Jihee Kim (KAIST)
![Page 2: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/2.jpg)
Deliberative Democracycitizens’ active participation in decision making and related discussions
![Page 3: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/3.jpg)
Deliberative Democracy Online
Open Government Data UI for Discussion Support
Balancer [Munson et al., 2011]
ConsiderIt[Kriplean et al., 2012]data.gov.uk, data.gov, data.seoul.go.kr
Reflect [Kriplean et al., 2012]
![Page 4: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/4.jpg)
Government Budgetplan for government to best allocate resources
But…• hard to comprehend• extremely complex• low interest & awareness
![Page 5: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/5.jpg)
Improve awareness & understanding of budgetary issues
Build interactive systems for
civic engagement
Leverage open government data
![Page 6: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/6.jpg)
Understanding Taxpayers’ Challenges
• Survey– 182 respondents in Korea– Perception of the government budget + estimation quiz
• Semi-structured interviews– 5 taxpayers– 3 experts
![Page 7: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/7.jpg)
1. Low awareness and interest in budgetary issues
knowledgeable about budget info
search for budget info
1
not really
7
very much
3.1
3.2
![Page 8: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/8.jpg)
1. Low awareness and interest in budgetary issues
interest in the budget
budget info is useful
1
not really
7
very much
4.6
4.5
![Page 9: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/9.jpg)
1. Low awareness and interest in budgetary issues
• Federal police budget estimation[Mean: $800B, Median: $11B, Stdev: 2114]
“I feel distant from all the big numbers that don’t really mean anything to me.”
$8.8B
![Page 10: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/10.jpg)
2. Contextual information matters in opinion formation.
“(Taxpayers) sometimes only see their own interests and fail to realize that
compromises need to be made.”
![Page 11: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/11.jpg)
2. Contextual information matters in opinion formation.
![Page 12: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/12.jpg)
2. Contextual information matters in opinion formation.
Seeing other programs in the category sometimes affected respondents’ opinion.
![Page 13: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/13.jpg)
3. News Outlets: Primary source for learning about budgetary issues
• 74% regularly read articles online– U.S. [Purcell et al., 2012]: 50% via news sites, 10% social networks
• News articles: more comprehensible, engaging
• NONE had attempted to read govt. reports
![Page 14: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/14.jpg)
3. News Outlets: Primary source for learning about budgetary issues
• Concerned about media biases
• Others’ comments help recognize the potential subjectivity, bias, or error in an article
• U.S. survey [Purcell et al., 2012]
• 37%: commenting important feature to have• 25%: commenting experience
![Page 15: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/15.jpg)
![Page 16: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/16.jpg)
Factful: Fact-Oriented Budgetary Discussions Online
enhanced news reader application
contextual budgetary facts
reader-initiatedfact-checking
![Page 17: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/17.jpg)
![Page 18: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/18.jpg)
LAYER #1:CONTEXTUAL BUDGET FACTS
![Page 19: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/19.jpg)
Embedding Contextual Budget Facts
![Page 20: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/20.jpg)
Automatically Inserted Overview
5-year trend
category info
category breakdown
category detection: fit score based on word hit count
![Page 21: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/21.jpg)
Most Relevant Budget Programs
link to budget program webpage
budget name, category, amount
program suggestion: for each word in the article, compute TF-IDF score against each program
![Page 22: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/22.jpg)
Automatic Annotation
programs of similar size
monetary value annotation with rule-based detector
![Page 23: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/23.jpg)
LAYER #2: READER-INITIATED FACT-CHECKING
![Page 24: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/24.jpg)
add a comment
do fact-checking
request fact-checking
Reader Activities
![Page 25: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/25.jpg)
Annotative Threaded Discussion
![Page 26: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/26.jpg)
do fact-checking
request fact-checking
Fact-Checking Embedded in the Article
read w/ fact-checked result
![Page 27: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/27.jpg)
Open Government Data Article Text Analysis
Budget Data Processing Pipeline
• opengov.seoul.go.kr• 76% of all internal documents
publicly accessible• 2014: $24B,
13 1st level categories, 4629 individual programs
• Article text parsing• Category detection• Program suggestion• Monetary value detection
![Page 28: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/28.jpg)
Evaluation: with Factful…• H1. Readers will discuss with
more fact-based statements.
• H2. Readers will discuss with more evidence, and more kinds of evidence.
• H3. Readers will become more critical about the article.
![Page 29: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/29.jpg)
Evaluation Setup
Between-subjects, 38 participants
Commenting Commenting
Fact-checking
Commenting
Fact-checking
Contextual Info
Baseline Fact-checking only
Factful
![Page 30: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/30.jpg)
Tasks and Procedures
Three articles about Seoul’s budget & policies
Pre-Q Read Discuss Post-Q
![Page 31: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/31.jpg)
Discourse Analysis• 404 comments
• Discussion coding: each comment is coded with one of the 25 categories[Unweighted Cohen’s 𝜅: 0.613]
![Page 32: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/32.jpg)
Discussion Quality Assessment
• 5 external raters
• 10 questions about discussion quality (score between 1-10)
• Overall quality• Criteria derived from deliberation lit. [Fishkin & Luskin, 2005]
informed, balanced, conscientious, substantive, comprehensive
![Page 33: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/33.jpg)
With Factful, overall discussion quality was higher.
score
6.6
5.67
4.93
1 2 3 4 5 6 7 8 9 10
Baseline
Fact-Checking
Factful
p < 0.05p < 0.05
![Page 34: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/34.jpg)
H1. With Factful, discussions contained more relevant, accurate information.
strongly agree
Discussants participated in the discussion with more relevant, accurate information.
strongly disagree
6.93
6.27
6.27
1 2 3 4 5 6 7 8 9 10
Baseline
Fact-Checking
Factful
p < 0.05p < 0.05
![Page 35: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/35.jpg)
H1. With Factful, participants added more fact-oriented comments...?
2.461.77
1.67
0 1 2 3
Baseline
Fact-Checking
Factful
2.541.771.75
0 1 2 3
Baseline
Fact-Checking
Factful
# comments with objective supporting arguments
# comments that asked for objective information
# comments / person
![Page 36: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/36.jpg)
H2. Factful discussions contained more diverse perspectives and supporting evidence.
6.93
6.00
5.27
1 2 3 4 5 6 7 8 9 10
Baseline
Fact-Checking
Factful
strongly agreestrongly disagree
Discussants participated in the discussion with more diverse perspectives and supporting evidence.
p < 0.05p < 0.05
![Page 37: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/37.jpg)
H3. With Factful, participants became more critical about the article.
4.03
4.77
5.08
1 2 3 4 5 6 7
Baseline
Fact-Checking
Factful
I trust the content of the article.
strongly agreestrongly disagree
p < 0.01p < 0.05
![Page 38: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/38.jpg)
H3. With Factful, participants held a more critical view on the article.
# comments criticizing the article• 0.8 / person in Factful vs 0 in Baseline
# comments criticizing other participants
0.460.31
1.25
0 1 2
Baseline
Fact-‐Checking
Factful
# comments / person
![Page 39: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/39.jpg)
Role of Contextual Information
Automated annotations w/ similar sized programs
Category overview
“Without such information, it would be hard to
determine if the given government spending is
worth or not.”
“It made reading through the article easier, because
the budget terms and numbers in the article
felt less obscure.”
![Page 40: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/40.jpg)
Future Work
• Live deployment• Crowdsourced fact-checking methods• Generalization– different countries– other datasets
![Page 41: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/41.jpg)
Improve awareness & understanding of budgetary issues
Build interactive systems for
civic engagement
Leverage open government data
Data-driven, social, crowdsourced mechanisms
![Page 42: Factful: Engaging Taxpayers in the Public Discussion of a Government Budget](https://reader034.fdocuments.net/reader034/viewer/2022051516/55a5bed61a28abbe058b45a1/html5/thumbnails/42.jpg)
BudgetMap: Issue-Driven Navigation for a Government Budget. Nam Wook Kim, Chang Won Lee, Jonghyuk Jung, Eun-Young Ko, Juho Kim, Jihee Kim.
CHI 2015 Extended Abstracts.
BudgetMap: Issue-Driven Budget Navigation