All the knowledge of mankind in the palm of a hand
57
All the knowledge of mankind in the palm of a hand
-
Upload
marcio-ramos -
Category
Engineering
-
view
254 -
download
7
Transcript of All the knowledge of mankind in the palm of a hand
All the knowledge of mankind
in the palm of a hand
Deconstructing
All the knowledge of mankind
Imagine yourself, calmly
walking in the street
when suddenly...
You are assailed by the
cruel doubt:
What is the correct
orientation for a
toilet paper?
Or, how did that
theorem of infinite
monkeys go again?
Or seriously, what is
the story behind the
space shuttle?
But, ALL the
knowledge?
Number of pages: 33,088,699 Number of articles: 4,530,955 Number of files: 831,598 Number of edits: 721,601,195 Number of users: 21,533,669 Number of admins: 1,410
As of the 7th of June, 2014:
This is Encyclopædia Britannica:
This is Wikipedia:
2,655 million words across 4.5 million articles (as of April 2014), implying an
average of 590 words per article.
And inside a
computer?
pages-articles.xml.bz2 – Current revisions only, no talk or user pages. (This is probably the one you want.
The size of the 13 February 2014
dump is approximately 9.85 GB
compressed, 44 GB uncompressed).
From http://en.wikipedia.org/wiki/Wikipedia:Database_download
Deconstructing – pt.2
In the palm of hand
Specifications:
Summary of B:
US$35 700 MHz ARM11
512 MB RAM
SD storage (up to 128 GB)
40 grams
So, how about,
We place “The” Wikipedia...
In a Raspberry pi?
(courage, there is a long way ahead)
Buy a RPI (US$37) http://www.amazon.com/RASPBERRY-MODEL-756-8308-Raspberry-Pi/dp/B009SQQF9C
Buy a SD Option 1: SD > 45GB
Option 2: SD < 45GB
Option 1: SD > 45GB
All Wikipedia can be inside, but, bigger SD’s are:
Less compatible, More expensive, More instable,
Slower.
Option 2: SD < 45GB
Place only the OS in the SD, and the dump in a pen
drive / external HD (occupies one USB port =/ )
Install OS in the card http://elinux.org/RPi_Easy_SD_Card_Setup
Choose a reader Option 1: Evopedia
Option 2: Serve MediaWiki
Option 3: Kiwix
Option 1: Evopedia
You can use the official dump, just 9 GB on torrent!
But the reader is less mature, and it is difficult to
place an external dump.
Option 2: MediaWiki (server)
Is what Wikipedia uses to show pages.
A lot more work, a lot more time processing, a lot less
performance on rpi.
Option 3: Kiwix
Very good in viewing wikipedia pages and
making a simple server. Downside: needs to
download own dump, 44 GB...
Installation Option 1: Installing only the
server.
Opção 2: Installing
everything.
Option 1: only the server
It is already compiled for ARM!
http://sourceforge.net/projects/kiwix/files/0.9_rc2/kiwix-server-0.9-rc2-linux-armv5tejl.tar.bz2/download
kiwix-folder$ sudo make install
Option 2: everything
It is not compiled for ARM...
http://www.kiwix.org/wiki/Compilation kiwix-folder$ ./autogen.sh
Solve all the (damn) dependencies with sudo apt-get install protip: read through the ./configure file to see the dependencies
Install xulrunner10-dev, not xulrunner-dev
kiwix-folder$ ./configure kiwix-folder$ make
A LOT of hours later:
kiwix-folder$ make install
Finalizing:
$ kiwix Open dump at SD/pen drive
Fix the index like this: http://sourceforge.net/p/kiwix/bugs/275/
Profit!
Interesting additionals:
Make rpi conect automatically on a wifi and if it can’t make it create an
adhoc server http://lcdev.dk/2012/11/18/raspberry-pi-tutorial-connect-to-wifi-or-create-an-encrypted-
dhcp-enabled-ad-hoc-network-as-fallback/ Or with a secure connection (what? WEP? no way!):
http://www.novitiate.co.uk/?p=183
Interesting additionals2:
Configure VNC server so we can see the screen remotely.
http://elinux.org/RPi_VNC_Server
Thank you
slideshare.net/ marrciovr
