The Brave New World of Free, Open Data and Open Access
128
14/05/22 www.rba.co.uk 1 Karen Blakeman [email protected] , www.rba.co.uk Slides available at http://www.rba.co.uk/as / This work is licensed under a Creative Commons Attribution License The Brave New World of Free, Open Data and Open Access Pre-conference workshop, BIALL 2014, Harrogate Wednesday, 11 th June 2014 #BIALL2014
-
Upload
karen-blakeman -
Category
Internet
-
view
1.285 -
download
5
description
Pre-conference seminar given at BIALL 2014, 11th June 2014, Harrogate
Transcript of The Brave New World of Free, Open Data and Open Access
07/04/23 www.rba.co.uk 1
Karen [email protected], www.rba.co.uk Slides available at http://www.rba.co.uk/as/
This work is licensed under a Creative Commons Attribution License
The Brave New World of Free, Open Data and Open Access
Pre-conference workshop, BIALL 2014, HarrogateWednesday, 11th June 2014
#BIALL2014
All change!
Search engines– new algorithms, ranking and display, quality of results– EU ruling on “right to be forgotten”, how much is being
censored/removed?
Free government and legal resources– how reliable are they?– data and information being moved to Gov.uk (is it?) or web archives
(really?)
Open Access vs open access vs “open access”– how accessible is it?– authority, version– predatory publishers
Open data– how “open” is it really– findability, ease of use, quality
Changes to copyright – data/text mining
07/04/23 www.rba.co.uk 2
Impact of Social Sciences – The right to read is the rightto mine: Text and data mining copyright exceptions introduced in the UK. http://blogs.lse.ac.uk/impactofsocialsciences/2014/06/04/the-right-to-read-is-the-right-to-mine-tdm/
07/04/23 www.rba.co.uk 3
Search engines
Google in particular is undergoing major change
Don’t forget alternative search engines (they may still be subject to recent EU ruling on search results)
Need to understand how search tools work, how the different country versions work and how they present results
Google and Bing (to a lesser degree) personalise results
How to assess quality of results
07/04/23 www.rba.co.uk 4
Five things you need to know about Google search
1. Google personalises your search
Personalises search based on– location– device that you are using– past search history– past browsing activity– activity in other areas of Google e.g. YouTube, blogs, images– content from contacts in your personal networks may be given
priority (possibly)
07/04/23 www.rba.co.uk 5
Private browsing - quickest way “un-personalise”search
Chrome - New Incognito window Ctrl+Shift+N
FireFox Ctrl+Shift+P
Internet Explorer Ctrl+Shift+P
Opera Ctrl+Shift+N
Will not remove country personalisation
Five things you need to know about Google search
2. Google automatically looks for variations on your search terms and sometimes drops terms from your search
– Google may or may not tell you that it has ignored some of your terms
– “..” around terms, phrases, names, titles of documents does not always work
– To force an exact match and inclusion of a term in a search prefix it with ‘intext:’ public transport intext:algal biofuels
– Use Verbatim for an exact match search
Google Verbatim
Google now showing missing search terms?
Not always shown – possibly still a live experiment?
07/04/23 www.rba.co.uk 9
Five things you need to know about Google search
3. Google web search does not search everything it has in its database
– two indexes: main, default index and the supplemental index
– supplemental index may contain less popular, unusual, specialist material
– supplemental index comes into play when Google thinks your search has returned too few results
– Verbatim and some advanced search commands seems to trigger a search in the supplemental index
Five things you need to know about Google search
4. Google changes its algorithms several hundred times a year How Google makes improvements to its search algorithm
- YouTube http://youtu.be/J5RZOU6vK4Q
Five things you need to know about Google search
5. We are all Google’s lab ratsJust Testing: Google Users May See Up To A Dozen Experiments
http://searchengineland.com/just-testing-google-searchers-may-see-up-to-a-dozen-experiments-141570
Mostly minor effects on search but sometimes totally bizarre results
What I see on my screen will not be what you see on your screen, will not be what your colleagues see on theirs, will not be
what your users see.
07/04/23 www.rba.co.uk 13
Hummingbird
Not just an update but a completely new algorithm
Tries to make “sense” of your query and put it into context, natural language queries
Not just search history but also your location, device being used
Now difficult to predict how Google will handle your search and how results will be displayed
Layout of results and menu options depend on type of search
07/04/23 www.rba.co.uk 14
So called “right to be forgotten” ruling
07/04/23 www.rba.co.uk 15
Edition of Monday, January 19, 1998, page 23 - Newspaper - Lavanguardia.es http://hemeroteca.lavanguardia.com/preview/1998/01/19/pagina-23/33842001/pdf.html
Information is NOT removed from the web
Subject can apply to have links in search results that point to specific information removed from the results
Only applies to searches conducted in the EU + Norway, Switzerland, Iceland and Lichtenstein
Not automatic – subject has to apply and request will be assessed to see if the information is “inadequate, irrelevant or no longer relevant, or excessive in relation to the purposes for which they were processed.”
Request form available at https://support.google.com/legal/contact/lr_eudpa?product=websearch# 07/04/23 www.rba.co.uk 16
Could have a serious impact on business and legal research
How far will it go?
People already trying to get links to their company directorships, bankruptcy, IVAs etc removed
Land Registry records
Crime reports in the newspapers
Unfavourable reviews of services
07/04/23 www.rba.co.uk 17
Four things we’ve learned from the EU Google judgment | ICO Blog http://iconewsblog.wordpress.com/2014/05/20/four-things-weve-learned-from-the-eu-google-judgment/
iconewsblog.files.wordpress.com/2014/05/key-points-of-cjeu-case.pdf http://iconewsblog.files.wordpress.com/2014/05/key-points-of-cjeu-case.pdf
BBC News - More Google 'forget' requests emerge after EU ruling http://www.bbc.co.uk/news/technology-27439194
EU Law Analysis: The CJEU's Google Spain judgment: failing to balance privacy and freedom of expression http://eulawanalysis.blogspot.no/2014/05/the-cjeus-google-spain-judgment-failing.html
Google Offers Webform To Comply With Europe’s ‘Right To Be Forgotten’ Ruling | TechCrunch http://techcrunch.com/2014/05/30/right-to-be-forgotten-webform/
The Myths & Realities Of How Of The EU’s New “Right To Be Forgotten” In Google Works http://searchengineland.com/eu-right-forgotten-191604
07/04/23 www.rba.co.uk 18
Last Week Tonight with John Oliver (HBO)Right To Be Forgotten - YouTube https://www.youtube.com/watch?v=r-ERajkMXw0&feature=youtu.be
07/04/23 www.rba.co.uk 19
How to get around it?
Until we see how it works in practice, difficult to give advice
Google says it will indicate on the results page if information has been excluded
Will we be able to use Google.com or Google.ca, for example, or will Google recognise we are in the EU from our IP address and still block results?
Use alternative search engine with no business footprint in Europe?
New search engines starting up without a European presence?
Anonymous proxy servers?
07/04/23 www.rba.co.uk 20
Search commands (Google, Bing)
Think file format– PDF for research documents, government reports, industry papers,
company reports– ppt or pptx for presentations, tracking down an expert on a topic– xls or xlsx for spreadsheets containing data
Use the advanced search screen or the filetype: command Land registry tax evasion data mining filetype:pdf tax evasion data mining filetype:ppt tax evasion data mining filetype:pptx tax evasion UK filetype:xls tax evasion UK filetype:xlsx
07/04/23 www.rba.co.uk 21
Search commands (Google, Bing)
Filetype may not always work
– The filetype: command may not always work even when a site offers data, for example, in spreadsheet format
– Sometimes this is because the data is held in a database and the files are created from a subset of the data when you request the file.
– Instead of filetype:xlsx or filetype:xls simply include the word Excel, xls or csv in your search.
07/04/23 www.rba.co.uk 22
Search commands – site: (most search engines)
Site searchFor searching large websites, or groups of sites by type for example government, NHS, academic
Can exclude sites using -site:
agricultural occupational asthma UK site:nhs.uk
agricultural occupational asthma UK site:ac.uk
agricultural occupational asthma UK site:gov.uk
agricultural occupational asthma UK site:gov.uk –site:hse.gov.uk
07/04/23 www.rba.co.uk 23
Searching in multiple languages
A significant amount of information is in the local language
Google has removed the extremely useful “Translated foreign pages” search option
This is how it can be done now
1.Use Google Translate (http://translate.google.com/) to translate your search into the required language.
2.Copy the translated search and paste it into Google search.
3.Google Chrome will offer to translate page If using another browser click on the ‘Translate this page’ link next to a result to view a translation of just that page.
07/04/23 www.rba.co.uk 24
07/04/23 www.rba.co.uk 25
Evaluating resources
Date of publication, 'last updated'
Check text for clues of publication date
Stated date for a web page or document may be automatically generated when it is put onto the web site
After a web site redesign pages are re-uploaded and are assigned a new time-stamp
Some pages are generated "on the fly" so will always have today's date
Type of web site for example:– gov.uk, ac.uk, .gov, .edu
Who is really behind the site? – use a domain name register such as http://whois.domaintools.com
07/04/23 www.rba.co.uk 26
Who owns a website?
http://www.domaintools.com/ good starting point for identifying who owns a domain name but....
– may be hiding behind an agent– may be using a service such as Privacy Protect (
http://www.privacyprotect.org) – if in the UK and an individual with a personal page then
contact details other than name are not publicly available (Data Protection)
Run a Google or Bing search on the ‘registrant’ or agent and see if you can find out anything more about them
07/04/23 www.rba.co.uk 27
Domaintools check – good news
07/04/23 www.rba.co.uk 28
Domaintools check – what you do not want to see
07/04/23 www.rba.co.uk 29
Quality of statistics
Read the definitions
– Definitions and scope may have changed over the years and may be different for each country
Political manipulation e.g. unemployment statistics
Some data may not exist for some countries e.g. minimum wage
Sudden jumps or flatlining in graphs should alert you to oddities in the data (see later)
Official data not immune from errors
May need an industry expert to help interpret and analyse sector specific data e.g. energy reserves vs resources, “liquids”/oil/petroleum
07/04/23 www.rba.co.uk 30
Some web sites......
07/04/23 www.rba.co.uk 31
Gov.uk
07/04/23 www.rba.co.uk 32
UK Government Web Archive | The National Archives http://www.nationalarchives.gov.uk/webarchive/
Browse by category or choose your organisation from an A-Z listChoose the date of the archived version of the website you want to view
07/04/23 www.rba.co.uk 33
UK Government Web Archive | The National Archives http://www.nationalarchives.gov.uk/webarchive/
07/04/23 www.rba.co.uk 34
http://www.legislation.gov.uk/
Launched in 2010Not all legislation has been updatedPending updates are flagged
07/04/23 www.rba.co.uk 36
Monitoring progress of legislationhttp://services.parliament.uk/bills/2013-14/pensions.html
07/04/23 www.rba.co.uk 38
data.parliament.uk http://www.data.parliament.uk/
07/04/23 www.rba.co.uk 39
The Gazette | Official Public Record https://www.thegazette.co.uk/
07/04/23 www.rba.co.uk 40
FLARE - Foreign Law Research Group Website http://ials.sas.ac.uk/flare/flare.htm
07/04/23 www.rba.co.uk 41
FLARE - Union List of Official Gazettes: Europe http://ials.sas.ac.uk/flare/flare_fog_unionlist_europe.htm
07/04/23 www.rba.co.uk 42
They work for you http://www.theyworkforyou.com/
07/04/23 www.rba.co.uk 43
WhatDoTheyKnow - Freedom of Information (FOI) requests https://www.whatdotheyknow.com/
07/04/23 www.rba.co.uk 44
Official company information
Companies House http://www.companieshouse.gov.uk/
Lists of official company registers– Official Company Registers
http://www.rba.co.uk/sources/registers.htm – Company registration around the world
http://www.commercial-register.sg.ch/home/worldwide.html – Companies House Links
http://www.companieshouse.gov.uk/links/introduction.shtml#reg
US – listed companies– SEC Edgar IDEA – Interactive Date Electronic Applications
http://www.sec.gov/edgar.shtml
Canada - listed companies– SEDAR http://www.sedar.com/
07/04/23 www.rba.co.uk 45
Company Check
http://companycheck.co.uk/UK & Ireland companies and director searchSome info free, some free with login, some free with “pro” account, some pay as you goProvides 5 years of figures and graphs for Cash at Bank, Net Worth, Total Current Liabilities and Total Current AssetsDownload up to 5 yrs of accountsCompanies house documents (£2 or 99p depending on subscription)Credit risk, charges, CCJs, Monitor company for financial changes and when new accounts are filedLists directors of a company - see what other directorships they have
07/04/23 www.rba.co.uk 46
Company Check
07/04/23 www.rba.co.uk 47
Company Check
07/04/23
www.rba.co.uk 48
Company Check Dashboard
07/04/23 www.rba.co.uk 49
OpenCorporates and Google Refine
07/04/23 www.rba.co.uk 54
How to use OpenCorporates to match companies in Google Refine on Vimeo http://vimeo.com/17924204
Mapping Corporate Networks With OpenCorporates http://www.slideshare.net/psychemedia/mapping-corporate-networks-with-opencorporates
07/04/23 www.rba.co.uk 57
ICIJ Offshore Leaks Database http://offshoreleaks.icij.org/
07/04/23 www.rba.co.uk 58
ICIJ Releases Offshore Leaks Database Revealing Names Behind Secret Companies, Trusts | International Consortium of Investigative Journalists http://www.icij.org/offshore/icij-releases-offshore-leaks-database-revealing-names-behind-secret-companies-trusts
https://www.youtube.com/watch?v=GHmiomMNfLQ&feature=youtu.be
Open Access Research Literature
07/04/23 www.rba.co.uk 59
Open Access vs open access vs open access
Open AccessPublicly funded research made available free of charge to the user
Open access journalsPublications that are free to the reader, may or may not be a charge for publishing in the journal
“Open access” predatory publishersPublications of dubious quality that have been set up purely as a means of generating money
07/04/23 www.rba.co.uk 60
Mandated Open Access
USAll research publications resulting from work funded by the US National Institutes of Health are expected to be deposited in PubMed Central (http://www.ncbi.nlm.nih.gov/pmc/)
– some material embargoed for up to 12 or 24 months (http://www.ncbi.nlm.nih.gov/pmc/journals/)
– Europe PubMed Central (http://europepmc.org/) part of PMC network of international repositories
UK1st of April 2013 - researchers at UK Research Institutions are expected to publish as open access any peer‐reviewed research papers and conference proceedings that acknowledge Research Council UK funding
07/04/23 www.rba.co.uk 61
UK Gold versus Green OA
Gold OA– researchers publish their articles in journals that offer open access
publishing (can be established “conventional” publishers)– articles can be made available free of charge to readers
immediately – author or institution/department pays article processing fee– CC-BY
Green OA – researchers deposit copies of articles in an institutional or subject-
based repository, subject to copyright/license permissions – repository makes copies available to the public either immediately or
embargoed (more common) – period of embargo varies (for example
http://cdn.elsevier.com/assets/pdf_file/0018/121293/external-embargo-list.pdf)
– CC-BY-NC
07/04/23 www.rba.co.uk 62
Jeffrey Beall
List of Predatory Publishers 2014 | Scholarly Open Access http://scholarlyoa.com/2014/01/02/list-of-predatory-publishers-2014/
07/04/23 www.rba.co.uk 63
Fragmentation of open access
Where are the open access publications?– Individual OA articles within existing subscription
journals– Separate OA journals, publishers website– Author’s website– Institutional repositories– Aggregators e.g. Scopus, Web of Science, Google
Scholar– Mendeley, ResearchGate?
07/04/23 www.rba.co.uk 64
ResearchGate
http://www.researchgate.net/ Increasingly used to request copies of articles from authors
07/04/23 www.rba.co.uk 65
Google Scholar
Does not cover all key journals in all subjects – no source listTop publications for subjects and languages under Metrics link on home page or http://scholar.google.co.uk/citations?view_op=top_venues&hl=en
Scholar indexes the full text but you may have to pay to view the whole article
Groups different versions of an article together
07/04/23 www.rba.co.uk 67
Google Scholar
07/04/23 www.rba.co.uk 68
Google Scholar
Includes open access material, pre-prints, institutional repositories (but not necessarily author self archived papers on personal websites)
Includes material that is NOT peer reviewed but is structured and looks like an academic article (title in large font, authors, affiliations, abstract, keywords, citations)
Pre-prints and IR copies may differ from final published version – charts and images may be redacted because of copyright restrictions
07/04/23 www.rba.co.uk 69
Google Scholar
07/04/23 www.rba.co.uk 70
Does NOT use the publishers’ metadata
Sometimes gets the author wrong
Beware the advanced search screen and commands
– Date and author search looks in the area of the document where those elements are usually found
– Page numbers, part of an address, data item may be mistaken for publication year
Institutional repositories and open access
BASE - Bielefeld Academic Search Engine http://www.base-search.net/
CORE (COnnecting Repositories) http://core.kmi.open.ac.uk/search
DART-Europe E-theses Portal http://www.dart-europe.eu/basic-search.php
DOAJ: Directory of Open Access Journals http://www.doaj.org/doaj
Institutional Repository Search (IRS) http://irs.mimas.ac.uk/
Open DOAR http://opendoar.org/
RIAN - Pathways to Irish Research http://rian.ie
ROAR - Registry of Open Access Repositories http://roar.eprints.org/
OpenAIRE http://www.openaire.eu/
07/04/23 www.rba.co.uk 71
Specialist search tools for research information
A selection can be found at http://www.rba.co.uk/search/links.shtml#research
ArXiv http://arxiv.org/
BioMed Central http://www.biomedcentral.com/
Chemistry Central http://www.chemistrycentral.com/
ChemSpider http://www.chemspider.com/
Deep Web TechnologiesMednar http://mednar.com/Science.gov http://www.science.gov/Science Research http://scienceresearch.com/WorldWideScience http://worldwidescience.org/
07/04/23 www.rba.co.uk 72
Specialist search tools for research information
Europe PubMed Central http://europepmc.org/
Mendeley http://www.mendeley.com/
Open Biology http://rsob.royalsocietypublishing.org/
PhilPapers: Online Research in Philosophy http://philpapers.org/
PubMed Central http://www.ncbi.nlm.nih.gov/pmc/
TechXtra http://www.techxtra.ac.uk/
07/04/23 www.rba.co.uk 73
Elsevier’s take down notices
Elsevier clamps down on academics posting their own papers online (Wired UK) http://www.wired.co.uk/news/archive/2013-12/17/elsevier-versus-open-access
"Why do we send take down notices? One key reason is to ensure that the final published version of an article is readily discoverable and citable via the journal itself in order to maximise the usage metrics and credit for our authors, and to protect the quality and integrity of the scientific record. The formal publications on our platforms also give researchers better tools and links, for example to data"
07/04/23 www.rba.co.uk 74
Grey literature
Literature that has been “peer reviewed” or assessed/approved in some way by colleagues or subject experts but is not easy to find or access
Print run may have been small, possibly never published electronically Published on the web but page or site is no longer available
Research and technical papers, government reports, pre-prints, market surveys, press releases, committee working papers, conference papers and presentations
Use advanced Google commands and web archives to find documents
May or may not be open access
GreyNet International, Grey Literature Network Service – http://www.greynet.org/ – http://www.opengrey.eu/
07/04/23 www.rba.co.uk 75
Article may be OA but public access outside of the institution may be not allowed or difficult
07/04/23 www.rba.co.uk 77
WATER report published | SCONUL http://www.sconul.ac.uk/news/water-report-published Walk-in Access To E-Resources
BBC News - Public libraries get online access to research journals http://www.bbc.co.uk/news/education-25981183
07/04/23 www.rba.co.uk 78
For personal research, non-commercial use.
Public Library Initiative by PLS and ProQuest | Access to Research http://www.accesstoresearch.org.uk/List of participating libraries and publishers
Public Library Initiative by PLS and ProQuest | Access To Research http://freetoviewjournals.pls.org.uk/Search tool for the journals and articles covered by the agreement.List of journals covered by the agreement
Not just Open Access articles but subscription services as well
Gold Open Access articles can be viewed anywhere. Other articles can only be viewed on library premises
07/04/23 www.rba.co.uk 79
Free patent information
Patent Searching 101: A Patent Search Tutorial IPWatchdog.com Patents & Patent Law http://www.ipwatchdog.com/2012/03/17/patent-searching-101-a-patent-search-tutorial/id=22851/
Compares Google patent search with other services
“Holes in the database” – may not be able to find patents you know exist
Does go back further than some e.g. to US patent no.1 but difficult to focus search
See also Patents Searching with Esp@cenet, Google Patents and USPTO - University of Bradford http://www.brad.ac.uk/library/resources/patents-searching-with-espacenet-google-patents-and-uspto/ 07/04/23 www.rba.co.uk 80
Google Patents
https://www.google.com/patents
Coverage:– US– Canada– European Patent Office (EPO)– Germany– China– World Intellectual Property Organisation (WIPO)
Patents available in original language and English (Google Translate)
07/04/23 www.rba.co.uk 81
Google Patents
07/04/23 www.rba.co.uk 82
Google Patents advanced search
07/04/23 www.rba.co.uk 83
https://www.google.co.uk/advanced_patent_search
Statistics
More open data but....
Raw data – no pretty layouts or visualisations, you have to do that
Data may need weeding and cleaning before it is usable
May have to do a LOT of work on the data to get anything sensible out of it
To see what you could be letting yourself in for look at Tony Hirst’s blog postings on open data at http://blog.ouseful.info/, for example Reshaping Horse Import/Export Data to Fit a Sankey Diagram http://blog.ouseful.info/2013/02/18/reshaping-horse-importexport-data-to-fit-a-sankey-diagram/
07/04/23 www.rba.co.uk 84
Official statistics
OFFSTATS http://www.offstats.auckland.ac.nz/
UK National Statistics Publication Hub http://www.statistics.gov.uk/ Office for National Statistics http://www.ons.gov.uk/
Welsh Government | Statistics http://wales.gov.uk/topics/statistics/
Welsh Assembly Government StatsWales http://statswales.wales.gov.uk/
data.gov.uk http://data.gov.uk/
Eurostat http://ec.europa.eu/eurostat/
European Union Open Data Portal http://open-dat.europa.eu/en/
07/04/23 www.rba.co.uk 85
http://www.offstats.auckland.ac.nz/
07/04/23 www.rba.co.uk 86
UK National Statistics & ONS
http://www.statistics.gov.uk/, http://www.ons.gov.uk/
07/04/23 www.rba.co.uk 87
Publication Hub (http://www.statistics.gov.uk/) is an “index” to what is available and links through to other sites
ONS (http://www.ons.gov.uk/) only shows reports since 2008 even if there are earlier editions. Use the Publication Hub to search for the report title/series. Once you have found an edition of the title click on “Current and past editions” to see the list of editions available. Then click on the relevant report.
07/04/23 www.rba.co.uk 88
Publication Hub search (statistics.gov.uk)
07/04/23 www.rba.co.uk 89
http://statswales.wales.gov.uk/
07/04/23 www.rba.co.uk 90
data.gov.uk http://data.gov.uk/
07/04/23 www.rba.co.uk 91
data.gov.uk http://data.gov.uk/
Not all of the data on this site is open data – may be restrictions on useDownload links sometimes take you to the wrong datasetDownload links sometimes completely brokenIt’s all or nothing! May have to filter the datasets for the information you want and produce your own graphs and chartsVariety of formats
07/04/23 www.rba.co.uk 92
Eurostat http://ec.europa.eu/eurostat
07/04/23 www.rba.co.uk 93
European Union - Open Data Portal http://open-data.europa.eu/open-data/
07/04/23 www.rba.co.uk 94
European Union - Open Data Portal http://open-data.europa.eu/open-data/
07/04/23 www.rba.co.uk 95
Google Public Data Explorer
http://www.google.com/publicdata/One of Google's best kept secrets!
Public data sets made available by Eurostat, World Bank, IMF, CSO Ireland, OECD, ITU, some national statistics offices (but not ONS), and many more.
Source and date updated given.
Charts and charting options can highlight oddities and missing data
Look at the charts to see if there is a sudden change in the trends.
07/04/23 www.rba.co.uk 96
Google Public Data Explorer Minimum Wage
07/04/23 www.rba.co.uk 97
Google Public Data Explorer Minimum Wage
07/04/23 www.rba.co.uk 98
Eurostat - Minimum Wage
07/04/23 www.rba.co.uk 99
07/04/23 www.rba.co.uk 100
Google Public Data Explorer – DPT immunizations
07/04/23 www.rba.co.uk 101
World Bank DPT
07/04/23 www.rba.co.uk 102
Datamarket http://datamarket.com/
Open portal to datasets worldwide and market researchCreates visualisations of the data
07/04/23 www.rba.co.uk 103
Statista http://www.statista.com/
“The Statistics Portal for Market Data, Market Research and Market Studies”
– 60,000 topics from over 18,000 sources– http://www.statista.com/topics/– some information free, registration (free) required – Chart of the day
07/04/23 www.rba.co.uk 104
Statista
07/04/23 www.rba.co.uk 105
Zanran http://www.zanran.com/
Searches graphs, charts, tables, PDFs, spreadsheets
Can limit by location of server, date, and filetype
Title in results list is usually the title or caption to the table and not title of the document
Hover over the thumbnail to see a preview of the table or page
Click on the URL button next to the result to view the original URL of the document
– clicking on it may take you to “page not found” 404
Click on the title of the result to see Zanran’s own copy (free registration usually required
Useful if document no longer available at it’s original location and can’t be found in any of the web archives
07/04/23 www.rba.co.uk 106
Zanran
http://www.zanran.com/ Zanran – great for data in tables, charts and graphs http://www.rba.co.uk/wordpress/2013/01/25/zanran-great-for-data-in-tables-charts-and-graphs/
07/04/23 www.rba.co.uk 107
Guardian Data Store http://www.guardian.co.uk/data
07/04/23 www.rba.co.uk 108
Data and analysis on topics that are in the newsSome data sets created from information obtained via FoILinks to the original datasets are provided
Migrants crossing the Mediterranean: key numbers http://www.theguardian.com/news/datablog/2014/jun/10/migrants-crossing-the-mediterranean-key-numbers-libya-european#start-of-comments
07/04/23 www.rba.co.uk 109
Public Data Grouphttps://www.gov.uk/government/groups/public-data-group
07/04/23 www.rba.co.uk 110
Land Registry http://www.landregistry.gov.uk/
07/04/23 www.rba.co.uk 111
Land Registry price paid data
07/04/23 www.rba.co.uk 112
http://www.rightmove.co.uk/
07/04/23 www.rba.co.uk 113
6 months ago the sold price listed was £185,000 and for 2012.
Land Registry summary of the postcode
07/04/23 www.rba.co.uk 114
Title document
07/04/23 www.rba.co.uk 115
Missing data
07/04/23 www.rba.co.uk 116
Error report filed with the Land Registry three weeks ago
Still waiting for a response
Why might a property/price paid not appear in the data?
Seems not that uncommon according to discussion boards – usually data entry error (but the above example was in the public data until the last update)
Absence of price – gift of property or purchase of a share
Impractical to calculate price e.g. bulk purchase of properties
Commercial transactions
http://www.landregistry.gov.uk/market-trend-data/house-price-data-exclusions
Price paid data report builder
07/04/23 www.rba.co.uk 117
House price indices
How the Land Registry undercooks house prices by £90k [Daily Mail alert!] http://www.thisismoney.co.uk/money/mortgageshome/article-2270763/How-Land-Registry-undercooks-house-prices-90k.html
Which house price index can you trust? http://www.telegraph.co.uk/finance/personalfinance/houseprices/10403980/Which-house-price-index-can-you-trust.html
House Prices - Who Should You Believe? http://www.money.co.uk/article/1006113-house-prices-who-should-you-believe.htm
Why do average house prices differ? - http://moneyfacts.co.uk/news/mortgages/why-do-average-house-prices-differ/
07/04/23 www.rb a.co.uk 118
Crime statistics http://www.police.uk/
07/04/23 www.rba.co.uk 119
Crime statistics http://www.police.uk/
07/04/23 www.rba.co.uk 120
Crime statistics http://www.police.uk/
07/04/23 www.rba.co.uk 121
Electricity micro-generation
07/04/23 www.rba.co.uk 122
Variable Pitch http://www.variablepitch.co.uk/
Electricity micro-generation
07/04/23 www.rba.co.uk 123
Variable Pitch http://www.variablepitch.co.uk/stations/1310/
Electricity micro-generation
07/04/23 www.rba.co.uk 124
Variable Pitch http://www.variablepitch.co.uk/stations/2580/
Virginia Station is the hydroelectric installation that feeds Windsor Castle
FoI request generation data for Virginia Station
07/04/23 www.rba.co.uk 125
https://www.whatdotheyknow.com/request/request_electricity_output_of_ro
Chart and image gallery: 30+ free tools for data visualization and analysis - Computerworld http://www.computerworld.com/s/article/9214755/Chart_and_image_gallery_30_free_tools_for_data_visualization_and_analysis
07/04/23 www.rba.co.uk 126
Google Fusion Tables
07/04/23 www.rba.co.uk 127
And finally....
Per capita consumption of mozzarella cheese (US) correlates with Civil engineering doctorates awarded (US) http://tylervigen.com/view_correlation?id=3890
07/04/23 www.rba.co.uk 128
