Data François Bancilhon twitter.com/ fbancilhon
description
Transcript of Data François Bancilhon twitter.com/ fbancilhon
DataFrançois Bancilhon
twitter.com/fbancilhon
Public Data Editors
Private Data Editors
Data Producers
Data Analytics Dataviz
Data consumers
Platform providers
Tool providers
BI toolsproviders
Web
Datasets Editor
Internal Datasets
Open Data
Socizl Networks
CrawlScrapingETL, etc.
Text MiningStructurationAugmentatio
n…
DeliveryVisualization
Custo
mer
spcs
Data Publicaspecx
Custom Off the shelf
Data developer
Data Publica : expertiseCrawlScrapingETLData cleansingText & Web miningDatabasesOpen Data
WebSocial networkPrivate dataCrowdsourcing
DataPresentation
Transformation
Generic datavizSpecific dataviz
Content
Is Data Global?
• Software vendors operate in a global market• Data is country dependent• The shape of the data is country dependent• The access to the data is country dependent
• You cannot build a local software business• You can build a local data business
Internet start ups
Fortune 500
1 48 04 07 019 125
533 735 93248.8742° N, 2.3470° E
Dataset Examples
1. Public Tender announcements2. Public organisations 3. Economic activity (by vertical and by geography)4. City dashboard5. Employment dashboard6. Real Estate dashboard7. Enrichment of company data8. Weak signal detection in company data9. Directory of a specific category of stores10. Elected officials
Public tender announcements
BOAMP OJEU CUSTOMERPLATFORM
DEMATPLATFORM
CLASSIFICATION(CPV) ANALYSIS DEDUP
DEMATPLATFORM
DEMATPLATFORM
DELIVERY
Public organizations in France
• 175 000 instances• 5 000 types• 6 sources, all of them public• Data: name, code, address, phone, fax, email,
web site• Hard part: deduplication within and between
sources• LOD with public tenders (65% up to now)
Territory dashboard
EconomyDémographyHealthTourismEnvironmentetc.
CountryStateDisctrictCityNeighborhoodetc.
Dashboard
Theme
Geographic granularity
Manual data extraction
Automated data extraction
Manual report
productionCustom report
Vizualisation
Generic dataviz
Territory dashboards
Manual process
Automated data update
Your report
Other SourcesSources Industrial
process
Data set enrichment
• Given a list of companies and some information about them
• Find more information– With the company information, find the web site– Analyse the web site to find more information– Eventually look outside of the web site
Company data enrichment
Total 10 000 companies10 000 companies
(%)
5 779 companies with website
(%)Entreprises with Web site 5 779 58% 100%
Confident web site >0.5 2 084 21% 36%
# of shares information 468 5% 8%
Contact information 1 071 11% 19%
Professional e-mail 1 893 19% 32%
Phone 3 618 37% 62%
Generic email 1 212 12% 21%
Web site analysis
• Analyse the web site do diagnose the company Internet presence
Weak signal detection
• Given a set of company description• Detect some weak signal– Export– Change in management
Web site detection
Web site analysis
Signal detection
Hotel Room Prices
• In a given geographic zone• For all the hotels• For all the rooms• For every reserved day of the year• For every reservation day of the year• Provide the price of the room
French Elected Officials
Scope• Government• Mayors• Senators• « députés »
(congressmen)• « conseillers généraux »• « conseillers
départementaux »
Information• First Name, Las Name• Mandate• Location• Date• Political affiliation
François Bancilhontwitter.com/fbancilhonwww.data-publica.com