bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot...

29
bots installing, configuring and running pywikibot Merlijn van Deen, Maarten Dammers user:valhallasw / user:multichill http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013

Transcript of bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot...

Page 1: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

bots

installing, configuring and running pywikibot

Merlijn van Deen, Maarten Dammers

user:valhallasw / user:multichill

http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013

Page 2: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

1. Download packages (might take a while)

2. Bot background

3. Creating a bot user account

4. Install, configure

5. Make some edits on wikidata!

http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013

Page 3: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013

Page 4: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

Bukk, PD, http://commons.wikimedia.org/wiki/File:Hebeb%C3%BChne.JPG

http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013

Page 5: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

Leon Hart, CC-BY 2.0 http://commons.wikimedia.org/wiki/File:Polski_Fiat_Driving_School.jpg

http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013

Page 6: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

bot workflow

1. get a list of pages

2. do something with each page on your list

Page 7: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

1. Download packages (might take a while)

2. Bot background

3. Creating a bot user account

4. Install, configure

5. Make some edits on wikidata!

http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013

Page 8: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013

Page 9: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

1. Download packages (might take a while)

2. Bot background

3. Creating a bot user account

4. Install, configure

5. Make some edits on wikidata!

http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013

Page 10: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!
Page 11: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!
Page 12: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

• cd <rightclick> enter

Page 13: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

• pwb.py login -all • on linux: python pwb.py login -all

Page 14: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

magic key combination

• K <enter>

• n <enter>

• y <enter>

• <enter>

• (your language code) <enter>

• (your bot user name) <enter>

• S <enter>

• Y <enter>

• 22 <enter>

• wikidata <enter>

• (your bot user name) <enter>

• N <enter>

• N <enter>

• (enter password) <enter>

Page 15: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

add claims on wikidata: country

• choose a category

– e.g. en:Category:Bridges_in_France

hsivonen, CC-BY 2.0 http://commons.wikimedia.org/wiki/File:Pont_d%27Avignon.jpg

Page 16: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

Determining Q & P

• Browse the category, click an article

• Click on 'Edit links'

Page 17: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

1

2

3

4

Page 18: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

Q142 = France

P17 = country

Page 19: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

pwb.py claimit

-cat:Bridges_in_France

P17 Q142

adds the claim P17=Q142 (country=France) to all pages in Category:Bridges_in_France

Run for a few edits, then abort with ctrl-c

Page 20: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!
Page 21: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!
Page 22: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

Thue, PD, commons.wikimedia.org/wiki/File:Car_crash_1.jpg

Page 23: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

wikidata: convert infoboxes

• import biological classification from pages

– e.g. from en:Template:Taxobox

OpenCage, CC-BY-SA-2.5, commons.wikipedia.org/wiki/ File:Catopuma_temminckii.jpg

Page 24: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

template-claim mapping

{{Taxobox | name = Asian golden cat ... | regnum = [[Animal]]ia P75 | phylum = [[Chordate|Chordata]] P76 | classis = [[Mammal]]ia P77 | ordo = [[Carnivora]] P70 | familia = [[Felidae]] P71 ... }}

Page 25: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

pwb.py harvest_template

-cat:Felines

-template:"Taxobox"

-namespace:0

regnum P75

phylum P76

sets the claims P75 and P76 based on the Taxobox template for all pages in Category:Felines

Page 26: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines -template:"Taxobox" -namespace:0 regnum P75 phylum P76

Processing [[en:Felinae]]

Adding p75 --> q729

Sleeping for 8.9 seconds, 2013-05-25 13:49:37

Adding p76 --> q10915

Sleeping for 6.4 seconds, 2013-05-25 13:49:50

Page 27: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

P75

P76

Page 28: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

MADe, CC-BY-SA 3.0, commons.wikimedia.org/wiki/File:Belgisch_rijbewijs.JPG Multiple Authors, CC-BY-SA 3.0, www.mediawiki.org/wiki/File:Pwb_icon.svg

Page 29: bots - Wikimedia · 1. Download packages (might take a while) 2. Bot background 3. Creating a bot user account 4. Install, configure 5. Make some edits on wikidata!

Other widely-used scripts:

• replace.py (search-and-replace on pages)

• category.py (add/change/remove categories)

• redirect.py (fix double redirects)

data_ingestion.py (import images to commons)

http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013