Effective Data Science in Aerospace Applications
-
Upload
geoffrey-clark -
Category
Technology
-
view
29 -
download
0
Transcript of Effective Data Science in Aerospace Applications
Effec%veDataScienceinAerospaceApplica%ons
GeoffreyClark|2016-06-25|GR-81-RHOcopyrightLucidata2016
LucidataInforma%cs?
• GeoffreyClark,PrincipalandCEO– DataModeler:logicalandra%onalDW– Solu%onArchitect:analy%cs– Strategy:risks,what-if,sims,wargames
• AssociatesatLucidata– SeniorDataModeler,requirementsexpert– AdvancedAnaly%csexperts– SeniorGISanalystsandFOSS4Gdevelopment
copyrightLucidata2015
DualChallengesofAnaly%cs
Market
TMS
CRM
Analytics Solution StandardBIFormal,Trusted,Shared,PublicDataExplora0onAd-hoc,AgileExperimental
individualsPowerUsers
Clean,Load&JoinData
FormNewBIQues%ons
AnalyzeData
FindNewData
Analytics User Segments
FIN
YourData
Knowledge,Defini%ons,Hierarchies,Rela%onships
Ad-HocDataPlayground
Standard,IndustryDataIndustry
Reference
Execu%ves,Managers
Analysts,KnowledgeWorkers
TechnicalChallenge,DataIntegra%on CulturalChallenge,UserIntegra%on
HowUniqueisyourdata?*
*isthatasourceofadvantage,orconfusion?
En%ty-Rela%onship(ER)Modeling
ThisisanexampleEn.ty-Rela.onshipDiagram(ERD),whichexplainshowtoreadthenota.on.Itisalsoanexampleofahighlyabstractmodel,whichis"datadriven",meaningthatnewThingTypes,newThingsandnewThingRela.onshipsmaybeeasilyaddedtoadatabasebasedonthisdesignwithoutneedingtochangethedatabasestructures.Thisprac.cewascommoninearlydatabases,builtforon-linetransac.onprocessing(OLTP).Thisstandsincontrasttotheconcretebusinessseman.csimplementedaspartofDimensionalDataModelingefforts,suppor.ngon-lineanaly.calprocessing(OLAP).
copyrightLucidata2016
Simulate
Op,mize
Forecast
Derive(datamining)
Summarize&Describe(sta%s%cs)
Visualize
Join&Filter(datawarehousing)
Measure&Store(sourcesystems)
En,,es&Rela,onships(datamodeling)
BI
EA
STAT
SOR
Analy,
cs
QualityAMATEUR PROFESSIONAL
TheProgressionofAnaly%cs
copyrightLucidata2015 5
...restuponthisfounda%on
Thesetypesofmodels...
BigDataHistory,viaGoogleTrends
Source:heps://www.google.com/trends/
Whatresearchdidwedo?
"p1sk"-project#1,surrogatekeys.airport_id=2369
"p2nk"-project#2,naturalkeys.
airport_iata_cd='SEA’airport_from_dt='2011-07-01’
"p3uu"-project#3,universallyuniqueiden%fier(UUID),
airport_uuid=7cbcc311-18c9-4497-99c9-62c42fd1ef2b
"p4hk"-project#4,hashkey.airport_key=0ed805d25fc96166a5895857a252de4b
WhathastheBigDatainnova.oncycletaughtusaboutdatadesign?
OriginalT100“GreenBook”
DataSource:T100“GreenBook”
ImportanceofReferenceData
copyrightLucidata2015
Ifyouhaveafloodof.mestamps,beNerknowwhat.mezonetheyrepresent.And,don’tbeliketheMarsClimateOrbiter,getyourunitsofmeasureright!
Nbr Full Number ISO 31 Description Comments 10^0 1 C62 one (or unit) "EA" for each from ANSI 10^1 10 ten 10^2 100 CEN one hundred 10^3 1000 MIL one thousand 10^4 10,000 ten thousand 10^5 100,000 one hundred thousand 10^6 1,000,000 MIO one million Somewhat confusing 10^7 10,000,000 ten million 10^8 100,000,000 one hundred million 10^9 1,000,000,000 MLD one milliard in EU one billion (US) Horribly confusing! 10^12 1,000,000,000,000 BIL one billion in EU (one trillion in US) Horribly confusing! 10^18 1,000,000,000,000,000,000 TRL one trillion (EUR) Horribly confusing!
ISO31-UnitsofMeasure
Moreworkremainstobeeercoordinatethemeasurementac%vi%esofhumankind,thefuturewillappreciateit!
project#1,surrogatekey project#2,naturalkeysproject#3,universallyuniqueidproject#4,hashkey
Addi%onalFactorstoConsider
Source:DanLinstedtonlearndatavault.com.
Addi%onalResearchPlans
• Howdoesdatadesignchangewhenusingdistributeddatabasetechnology?aliases-massiveparallelprocessing(MPP),“Sharded”
• Howdoesdatadesignchangewhenusingcolumnardatabasetechnology?
• Howdoesdatadesignchangewhenusinggraphdatabasetechnology?aliases–“RDF”,“triplestore”.
• Howdoesperformancechangewithdifferentdiskop%ons–HDD,SSD,SSDRAIDS,etc.
Informa%onDensity CharlesJosephMinard’sCarteFigura.vefrom1869,depic%ngNapolean’s1812invasionofRussia,andaqermathinsevendimensions(la%tude,longitude,%me,temperature,armygroup,andmilitaryphase).“Thebeststa.s.caldrawingevermade”--EdwardTuqe
Source:heps://en.wikipedia.org/wiki/File:Minard.png