Modelling Data On The Web - University of...
Transcript of Modelling Data On The Web - University of...
![Page 1: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/1.jpg)
COMP60411ModellingDataOnTheWeb
UliSattler&BijanParsia
Week1Introduction,DataModels,Tables,andSQL
![Page 2: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/2.jpg)
TopicOverviewWhatisafundamentaldatamodel?Somekeydatamodels
Flat:flatfilesTablebased:relationalTreebased:XMLandabitofJSONGraphbased:RDF
Tradeoffs(esp.representational)betweenthemLookingforthepainpointsandsweetspots
![Page 3: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/3.jpg)
CourseGoals:
Knowledge&UnderstandingThis aimstogiveyoua
goodunderstandingofcoreconceptsofdatamodellingsomefamiliaritywithformalisms,APIs,andlanguages
formodellingdataonthewebdesign/representationissuesthatarise
courseunit
![Page 4: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/4.jpg)
CourseGoals:SkillsThis aimstogiveyoutheability/skillto
comparedifferentdatamodellingformalisms,designoranalyseadatamanagementsystem,
doesitmakegooduseoftheformalism'sfeatures?doesitfititspurpose?
courseunit
![Page 5: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/5.jpg)
CourseStructureLectures
ActivelearningLab
Makesureyouunderstandthecoursework!Readings
AllreadingsavailableonlineCore:the"Learning"eBookseries
(or )(or )
LearningSQL hereLearningXML hereLearningSPARQL
![Page 6: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/6.jpg)
AssessmentCoursework(50%,≈200marks)
Eachweek,amixture1. MCQquizzes(≈10marks)2. Shortessays(≈5marks)3. Amodellingassignment(≈10marks)4. Aprogrammingassignment(≈15marks)Precisemarkbreakdownvaries
Exam(50%)TakenonlineVerylike1&2
![Page 7: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/7.jpg)
Materials&BlackboardAllcoursematerialsareavailableonlineonthematerialspageWeuseBlackboardfor
CourseworkOnlineforums
Usethese!Exam
![Page 8: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/8.jpg)
VariantCircumstancesDisability(EqualityAct):
anyconditionwhichhasasignificant,adverseandlong-termeffectonaperson’sabilitytocarryoutnormalday-to-dayactivities.
Exam&Studysupport&moreGreat,helpfulpeople
and process
DisabilityAdvisoryandSupportService
CounsellingserviceSSO MitigatingCircumstances
...feelfreetoaskus:we'rehappytoadvise!
![Page 9: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/9.jpg)
Assistance&HelpEarlyinterventionismoreeffective
Ifyouarehavingchallengesofanysortthesoonertheyareidentifiedandcommunicatedtousthemorelikelywecanfindagoodresolution
ThisisverytrueformitigatingcircumstancesIfsomethingisinterfering,documentit!FillouttheformwhenthingsarehappeningThereisa"toolate"here!
...whenindoubt,askusandSSOforMitCircs
![Page 10: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/10.jpg)
ExpectedConductWeexpectofyou(andourselves)to
befairmindedtreateachotherwell&withrespectavoidacademicmalpracticetakeresponsibilityforcoursedutiesbeengaged,curious,andactive
Ifyouhaveaproblemorissuepleaseraiseitwithusifthatdoesn'thelp,contactyourprogrammedirector
![Page 11: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/11.jpg)
Preliminaries
Weallhavetostartsomewhere
![Page 12: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/12.jpg)
DataManagement(1)Almosteveryprogrammustdosomedatamanagement
Ifonlyconfigfiles!Manyareinformationheavy
AndmustdealwiththatinformationovertimeDatabaseManagementSystems(DBMSs)
Separate(orseparable)componentSpecialisedforvariablespurposed
Secondarystorage,scaling,complexity,etc.
![Page 13: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/13.jpg)
DataManagement:LifetimeSomedatais(typically)transientorephemeral
PositionofthecursoronthescreenSomedatais(typically)persistent
Bankrecords,addresses,healthdata,libraryentriesCursorpositioncanbe!
(Ifyouarerecordingthescreen...)
We'refocusedondatathatleanstowardpersistent
![Page 14: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/14.jpg)
DataManagement:StructureSomedatais(moreorless)informationallyopaque
E.G.,images,video,text,audioTheinformationcontentisn't(immediately)available
YoutypicallymustdosomeextractionSuchiscalledunstructureddata
SomedataisinformationallytransparentTheinformationcontentisprogrammticallyexplicitSuchiscalledstructureddata
WewilllaterdistinguishStructuredSemi-structured
![Page 15: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/15.jpg)
OutOfScopeThereislotsofDMthat'soutsideourscope1. Performance&Scaling:see2. Concurrency
Thustransactions(YoushouldreaduponACIDity)
3. Tuning,indeedmostphysicallevelstuff4. Cleansing5. Integration
Exceptforatinybit,aroundmerging
COMP62421
Theseconsiderationsdoaffectmodelling!
![Page 16: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/16.jpg)
DataAndTheWebTheWebisacollaborativeinformationstructure
LargelydecentralisedImmenseGrowingrapidlyChangingrapidly
TheWebproducesnewdatachallengesScaleofdataKindofdataShapeofdataUseofdata
![Page 17: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/17.jpg)
DataOn,From,BehindTheWebOntheWeb
data.gov,data.gov.uk,...FromtheWeb
LogfilesBehindtheWeb
Data(base)backedWebsitesThefilesystemisakindofdatabase
ContentManagementSystemsWordpress
SitesasDatabaseFrontEndsSeeAmazon
![Page 18: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/18.jpg)
![Page 19: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/19.jpg)
WhatIsADataModel?ThreeKeyAspects1. UnderlyingDataStructure,"CoreDM"2. DataIntegrity3. DataManipulation4. (Plusafourth!)DataSharing
MoreimportantontheWeb*
![Page 20: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/20.jpg)
"DataModel"IsAmbiguousDatamodelisusedtoreferto...
1. acompletedatarepresentationandmanipulationapproach(wedothis!)
2. justthecoredatamodel
3. aparticulardatarepresentationforadomainorapplication,alsocalledthedomainmodel
"Doesyourcalendardatamodelincludeleapyears?"
Generally,youcantellfromcontext,(2)israre.
![Page 21: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/21.jpg)
KindsOfDataDatacanlenditselftodifferentshapes
Array-likeTree-likeGraph-likeDocument-like
DatacanhavedifferentvolumesSmallto"big"data
DatacanhavedifferentvelocitiesStatic/offlinetostreaming
DatacanhavedifferentusepatternsManyreaders/fewwritersorthereverseorother!
![Page 22: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/22.jpg)
PolyglotPersistence...wearegearingupforashifttopolyglotpersistence—whereany decent sized enterprise will have a variety ofdifferent data storage technologies for different kindsof data. There will still be large amounts of it managed inrelational stores, but increasingly we'll be first asking howwewanttomanipulatethedataandonlythenfiguringoutwhattechnologyisthebestbetforit.
—MartinFowler
![Page 23: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/23.jpg)
PolyglotPersistence(2)This polyglot [e]ffect will be apparent even within a singleapplication. A complex enterprise application uses differentkindsofdata,andalreadyusually integrates information fromdifferentsources.Increasinglywe'llseesuchapplicationsmanage their own data using different technologiesdependingonhowthedataisused.—MartinFowler
![Page 24: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/24.jpg)
Poly-Glot/-SystemPersistenceEvenwithasinglecoredatamodel
MultiplesystemswithdifferentcharacteristicsMultiple,overlapping,domainmodelsMultiple,overlappingowners,versions,variants
ThisisparticularlytrueinontheWeb!
![Page 25: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/25.jpg)
"FlatFiles"--ASimpleModel
![Page 26: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/26.jpg)
ASampleDomainWestartwithaclassicexample:TheAddressBook
PeopleandinformationaboutthemNamesandcontactinformation
Wecandoafirstcutasadiagram
![Page 27: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/27.jpg)
ForExampleBijan!
Name:BijanParsiaCompany:UniversityofManchesterEmail:[email protected]...
Uli!Name:UliSattlerCompany:UniversityofManchesterEmail:[email protected]
![Page 28: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/28.jpg)
Storing!
![Page 29: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/29.jpg)
SlidesarenotagoodstorageplacefordataWehaveanarraylikestructureso...
Howaboutaspreadsheet!1entity/record/personperrowEachfield/attributeisacolumn
Wehavesoftwarethatworkswellwiththis!
![Page 30: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/30.jpg)
InteractingWithTheData
Tothedemo!
![Page 31: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/31.jpg)
PainPointsAround"name"
SortingSortingisoncolumns
Can'tsortbylastnameFiltering
CanfilterbynamesbeginningwithZCannotbysurname'sbeginningwithZ
Around"address"Can'tsortorfilterbypostcodeCan'tsortorfilterbycityCan'tsortorfilterbycounty
Theseareproblemswithourmodel
![Page 32: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/32.jpg)
![Page 33: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/33.jpg)
FixingTheDomainModel
![Page 34: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/34.jpg)
Interacting!
Demoencore!
![Page 35: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/35.jpg)
NewPainPointsVariablenumbersofthe"same"attribute
PhonenumberEmailaddressWebpageInsertingcolumnsispainful
LotsofpartialcolumnsSheernumbersucks
Companieshaveaddresses!Morethanone!Andphonenumbers,etc.
Moreproblemswithourmodel
![Page 36: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/36.jpg)
![Page 37: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/37.jpg)
BadModelBad
![Page 38: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/38.jpg)
FixingTheModel2Wewantaddinga(similar)columntobeeasy!
Easyasaddingarow!MakeanewtablejustforphonenumbersIndexnumberswithpersonrows
![Page 39: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/39.jpg)
FixingTheModelAgain
![Page 40: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/40.jpg)
PainPointsSortingdestroystherelationship
WeusedrownumberstoconnectSortingchangestherownumber!
HardtoseetherecordNolongerasimpleflatfile
CSVformatmakesassumptions
Theseare(mostly)implementationproblems!
![Page 41: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/41.jpg)
WhenADomainModelFailsFailuremustbeanalysed!
Didwegetthedomainwrong?fititwrongintoourcoreDM?pickthewrongcoreCMtomodelitin?
Isitunworkable?workablebutrequiresalotofapplicationcode?reasonablewithsomeworkarounds?
Howmuchtechnicaldebtarewepilingup?
What'sthecostofswitching?
![Page 42: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/42.jpg)
![Page 43: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/43.jpg)
BrokenCoreDataModelIfyouare
always"fighting"thesystemuselotsofapplicationcodetohackthingsliveinanerrorrichenvironmenthaveincreasingamountsofworkaroundsupportinyourdata
Yourdatamodelmightnotbeagoodfitforyourdomainandapplication!
![Page 44: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/44.jpg)
TheRestOfTheDBMSEvenifyourcoredatamodelisn'tagoodfit
Youmightbestuckwiththesystem
YoupaidgoodmoneyforthatOracledatabase!needfeaturesoftheimplementation
isthereanXMLdatabasewithtransactions?what'sthesupportcontract?
bestuckwiththemodelcriticallegacyapps
Justbecausethemodelisbrokendoesn'tmeanthatthesystemis
Orisbrokenenoughtojustifyaswitch
![Page 45: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/45.jpg)
![Page 46: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/46.jpg)
FlatFileProgramming
![Page 47: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/47.jpg)
SharingOurDatabasesSpreadsheets?
Propriatoryish(Excel,GoogleDoc,OpenOffice)Linguafranca:CSV
Comma(orTab)DelimitedValuesExactlythe(pure)flatfilemodelFormat:
Textfile1recordperlineFirstlinecanbespecial(columnnames)Eachcolumnseparatedbya","
Wemayneedtoquotecells(withcommas)
![Page 48: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/48.jpg)
CSVExample
![Page 49: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/49.jpg)
ProgrammaticManipulationIfwestoreourdatabasesasCSV
WecanloadandparsethemintostructuresManipulateourdatafromprograms
Ourprograms,insteadofExcelE.g.,usingtheApacheCommonsCSV
Reader in = new FileReader("path/to/file.csv");Iterable<CSVRecord> records = CSVFormat.EXCEL.parse(in);for (CSVRecord record : records) { String surname = record.get("surname"); String firstName = record.get("first_name"); ...}
![Page 50: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/50.jpg)
SolvingProblemsThissolvessomeproblems!
Inserting/removingcolumnsa"smallmatterofprogramming"Orwecouldusemultiplearrayswithpointers
Wecansplit/combinefieldsatwillWell,withabitofprogramming
WecancontrolsortingwellenoughUsepointerstoconnect
Lotsofwork!
![Page 51: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/51.jpg)
AgainstBespokeProgrammingThisisallatthewronglevel
Flatfilesandflatfile++areubiquitousWeshouldn'tbecodingcomplexfunctions
Overandoveragain!Evenifwecanprogramourwayaroundproblems
Doesn'teliminatetheproblemsSomesolutions(pointers)effectivelychangethemodel!
![Page 53: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/53.jpg)
TablesTable(orrelation)isthecoredatastructure
AtableisasetoftuplesAtupleis
ann-arysequenceasetofkey-valuepairs
FlatfilehadonetableWeallowmany!NamedtablesAkarelations
![Page 54: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/54.jpg)
Relations!(Weusetableandrelationinterchangeably)RelationsarelikeFirstOrderLogic(FOL)predicates
Relationname==PredicatenameNumberofcolumns==Arityofpredicate
Person(bijan,u_o_manchester,...)Predicateistrue(orfalse!)ofitsarguments
Relationis"true"oftupleswhichoccurinitPredicatescanhave
definitions(intensional!)facts(extensional!)
![Page 55: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/55.jpg)
OrderAndIdentityRecords/Rows/Entitiesneedidentity
InExcel,wehadtherowlabelTheorderorpositionofarecordwassignificant
Inourmodel,weneeddistinguishingattributesWepushidentityintothedata:akeyEitheranaturallyuniquesetofattributes
i.e.,adefinitedescriptionoramadeupone:anID
Orderisalwaysapropertyofthedatavaluesimplementation
![Page 56: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/56.jpg)
MultipleTablesActionsonmultipletables:
Splittingatdesigntime:trytonormalizeyourDBruntime:droppingbits
CombiningTaketwotablesandproduceanewtable
ThekeytorelationaldomainmodellingDecomposeyourprobleminto"base"tablesDerivenewtablesforspecificneeds
![Page 58: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/58.jpg)
WhatIsAFormalism?Aformalsystem(orformalism):
syntax:whatcanwewrite?semantics:whatdoesourwritingmean?withprecise(mathematical)definitionsdesignedtocaptureacoherentsetofoperations("syntax"isloose,e.g.,wemightjusthaveacollectionofoperators)
![Page 59: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/59.jpg)
KeyGoalsOfAFormalism1. tobeclearaboutwhatwemean
Inourspreadsheetis"1"anumber,astring,either,both,somethingelse?
2. toallowthedeterminationofkeypropertiese.g.,complexityofqueryanswering
3. toabstractawayfromparticularimplementionse.g.,allowustodeterminewhenwildlydifferentimplementationsarecorrectthuscaninteroperate
![Page 60: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/60.jpg)
FormalismVs.LanguageFormalismsareoftenabstract
Thiscanbeanadvantage!CanbehardtouseifonlyabstractConcreteinstancestypicallyinvolvecompromise
WefocusonconcretelanguagesFormalismsarethetheoryLanguagesarethepractice
Well,itmaybeallrightinpractice,butitwillneverworkintheory.Intheory,thereisnodifferencebetweentheoryandpractice.But,inpractice,thereis.
OtherQuotesOnTheoryvsPractice
![Page 61: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/61.jpg)
![Page 62: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/62.jpg)
SQL:ALanguageForTablesSchema
CREATE TABLEtable_nameUpdate
INSERT INTOtable_nameDELETE FROMtable_nameUPDATEtable_name...
QuerySELECT ... FROMtable_name
SQLoperations(largely)areclosedovertables
![Page 63: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/63.jpg)
AnInfelicityThereisalotoflingowithslightdifferentmeanings.Conceptsgetdividedup
inslightlydifferentways.
Ourtalk Common LearningSQLp.10
CoreDataModelDataIntegrity DataDefinition SQLschemastatements"CREATE"
DataManipulation Query/UpdateLanguage
SQLDatastatements
![Page 64: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/64.jpg)
ASampleSQLProgramCREATE TABLE People ( name varchar(255), company varchar(255), address varchar(255), phone varchar(255), email varchar(255), home_page varchar(255));
INSERT INTO People VALUES ('Aleshia Tomkiewicz', 'Alan D Rosenburg Cpa Pc', '14 Taylor St, St. Stephens Ward, Kent CT2 7PP', '01835-703597','[email protected]', 'http://www.alandrosenburgcpapc.co.uk');SELECT name FROM People
YoumustDefinebeforeUpdatebeforeQueryI.e.,CREATEbeforeINSERTbeforeSELECT
![Page 65: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/65.jpg)
ModellingWithSQLSQLletsusexpressmodelsatthelogicalto(someofthe)physicallevel
SpecifyingindicesisabitphysicalKnowledgeaboutimplementationmayinformmodellingchoices
SQLhasnomechanismsforconceptuallevel
![Page 66: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/66.jpg)
DomainModel1InSQL
![Page 67: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/67.jpg)
DomainModel1InSQL CREATE TABLE People ( name varchar(255), company varchar(255), address varchar(255), phone varchar(255), email varchar(255), home_page varchar(255));
INSERT INTO People VALUES ('Aleshia Tomkiewicz', 'Alan D Rosenburg Cpa Pc', '14 Taylor St, St. Stephens Ward, Kent CT2 7PP', '01835-703597','[email protected]', 'http://www.alandrosenburgcpapc.co.uk');...
Canwedoallthatwedidinthespreadsheet?
![Page 68: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/68.jpg)
SQLManipulationOfDM1CountrecordsinyourPeopletable:
Searchforitems:
Sortthetable!
SELECT COUNT(*) FROM People
SELECT * FROM PeopleWHERE name like 'Aleshia%'
SELECT * FROM PeopleWHERE name like '%Tomkiewicz'
SELECT * FROM PeopleORDER BY name asc
![Page 69: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/69.jpg)
DomainModel2InSQL
![Page 70: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/70.jpg)
DomainModel2InSQL CREATE TABLE People ( first_name varchar(255), surname varchar(255), company varchar(255), street_address varchar(255), city varchar(255), county varchar(255), post_code varchar(255), phone varchar(255), email varchar(255), home_page varchar(255));
INSERT INTO People VALUES ('Aleshia', 'Tomkiewicz', 'Alan D Rosenburg Cpa Pc', '14 Taylor St', 'St. Stephens Ward', 'Kent', 'CT2 7PP', '01835-703597','[email protected]', 'http://www.alandrosenburgcpapc.co.uk');...
![Page 71: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/71.jpg)
SQLManipulationOfDM2Theoldquerieswork,butwecanimprovethem
Searchforitems:
WecanrecreateDM1!
SELECT * FROM PeopleWHERE first_name = 'Aleshia'
SELECT * FROM PeopleWHERE surname = 'Tomkiewicz'
SELECT first_name || " " ||surname as name, street_address || ", " ||city ||", "|| county ||" " || post_code as address,phone,email,home_pageFROM People
![Page 72: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/72.jpg)
![Page 73: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/73.jpg)
DomainModel3InSQL
![Page 74: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/74.jpg)
DomainModel3InSQL CREATE TABLE People ( person_id SMALLINT UNSIGNED, first_name varchar(255), surname varchar(255), company varchar(255), street_address varchar(255), city varchar(255), county varchar(255), post_code varchar(255), email varchar(255), home_page varchar(255), CONSTRAINT pk_person PRIMARY KEY (person_id));
CREATE TABLE Phone ( person_id varchar(255), number varchar (255), CONSTRAINT pk_phone_number PRIMARY KEY (number));
INSERT INTO People VALUES ('1','Aleshia', 'Tomkiewicz', 'Alan D Rosenburg Cpa Pc', '14 Taylor St', 'St. Stephens Ward', 'Kent', 'CT2 7PP', '[email protected]', 'http://www.alandrosenburgcpapc.co.uk');INSERT INTO Phone Values ('1', '01835-703597')INSERT INTO Phone Values ('1', '01944-369967')
![Page 75: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/75.jpg)
SQLManipulationOfDM3RecreateDM1andDM2:easyFindeveryonewithsamephonenumberCanwehaveunassignednumbers?
![Page 76: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/76.jpg)
How'dDMDo?CoreDM/Datastructure
TablesseemtoworkSQLandRelationalModel
Wecandoeverything!AllqueriesinallmodelsModel3has2tables/requiresjoins
DomainModel3Neaterinsertinganddeleting
Canhaveasmanyphonesasyouwant!Everyotherdomainmodelcanbederived
Justwritethequery:defineasaview!
![Page 77: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/77.jpg)
![Page 78: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/78.jpg)
ExpressivePowerSQLisexpressive
ThecoredatamodelisrichComposingandfilteringtablesdoesalot!Operatorsandfunctionshelpful
Withoutconcat,there'dbetrouble!Thelanguageispowerful
ReasonablycomposableLotsoffeaturesExtendedandextensibleinmanyimplementations
Interopproblems!
![Page 79: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/79.jpg)
QueryingWithSQL
![Page 80: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/80.jpg)
SchemasVs.QueriesCREATEstatements
"create"emptytablesoutofnothingatallwithcertainconstraintswithsomeexpectationofpermanence
SELECTstatements"generate"newtables(possiblywithdata)outofexistingtablesaccordingtosomeconstraintswithnoexpectationofpermanence
![Page 81: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/81.jpg)
ClosedOverTablesSQLis(mostly)closedovertables
MostSQLconstructstaketablesandproducetablesClearexception:Functions!
ManipulationismanipulationoftablesNotrows,columns,orcellsdirectlyRows,columns,andcellsare"degeneratetables"...
![Page 82: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/82.jpg)
FilteringVKeyoperationSELECT:ignoringsomeparts
Basically"find"CanfilterrowsorcolumnsorbothRequires"testing"functionsonvalues
![Page 83: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/83.jpg)
FilteringColumns"Projection"SpecifiedintheSELECTclause
Keepallcolumns:
Justasinglecolumn:
Multiplecolumns:
Renamecolumns:
SELECT * FROM People
SELECT county FROM People
SELECT name, county FROM People
SELECT street_address AS address FROM People
![Page 84: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/84.jpg)
![Page 85: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/85.jpg)
FilteringRowsJustcalled"filter"or"selecting"SpecifiedintheWHEREclauseofyourquery:
Equality:
Range:
Compoundcriteria:
SELECT * FROM People WHERE surname = "Smith"
SELECT * FROM PeopleWHERE heartrate > 95
SELECT * FROM PeopleWHERE heartrate > 95 AND county="Kent"
![Page 86: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/86.jpg)
BuildingTablesWithCrossJoinThefundamentaloperationisCartesianproductPeoplexPhoneThismakesanewrowoutofeverypairofrowsbetweenthetwotable
What'sthesizeoftheresult?Notreallyauser-orientedfeature
"Incidentally"crossjoinsaredangerous!
![Page 87: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/87.jpg)
BuildingTablesWithInnerJoinAninnerjoinisajoinfilteredoncommoncolumns
Usefulforourphonerecords!
(Specialcasecalleda"natural"join.)
SELECT * FROM People, PhoneINNER JOIN ON People.person_id = Phone.person_id
![Page 88: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/88.jpg)
BuildingTablesWithOuterJoinAnouterjoinislikeaninnerjoinbutitreturnsalsorowsthatdon'thaveamatchintheothertable
leftouterdifferentfromrightouterSELECT * FROM People, PhoneRIGHT OUTER JOIN ON People.person_id = Phone.person_id
willreturnalsopeoplewhohavenophone!
![Page 89: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/89.jpg)
BuildingAndFilteringOncewe'vebuiltatablewecanfilterthingsweneed:
SELECT * FROM People, PhoneRIGHT OUTER JOIN ON People.person_id = Phone.person_idWHERE People.surname = "Smith"
...youknewthatalready!?
![Page 90: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/90.jpg)
TheCostAkeyissuewithjoins
WorsecaseisaCROSSEvenifyoudon'tgeneratetheCROSS
Youmighthavetoconsiderallthepairs(Ifyouaren'tcareful)
GoodoptimisersavoidbothConsideringlotsofmatches(thinkindexes)Generatinglargeintermediatetables
![Page 92: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/92.jpg)
MultiplePhoneColumnsSomepeoplehavenoneoroneOrnoemailorwebpage
![Page 93: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/93.jpg)
NoSurnameEvenifwenormalisedthataway
Somepeopledon'thaveasurname!
![Page 94: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/94.jpg)
Nullnullisadistinguishedvaluewhichcanmean:
"Valuenotyetknown""Notapplicabletothisentity""Valueundefined"checkout
Keyproperty:Unequaltoeverythingnull = nullisnevertrueMatchonnotnull,ratherthannull
LSQL
Strangevalue!
![Page 95: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/95.jpg)
OuterJoinsIfyouhavenonullsinyourbasetables
youcan'tgetthemintablesderivedbyinnerjoinHowever,the2phonecolumntableisderivable
WeusetheouterjoinOuterjoinstakeatableT
foreachrowinTextenditwiththe(projected)columnsfromanothertableIfthere'samatch,addthematchedvalues*else,addnulls
SeeLearningSQL forsomeworkedexamplesChapter10
![Page 96: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/96.jpg)
![Page 97: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/97.jpg)
NullProliferationnullnevermatches
SoiteratedouterjoinsproliferatenullsAsyougetwider,yougetsparser
Ifyouarematchingonasparseattributenullsposechallengeforrelationaltheory
AndsomewhatforpracticeStartsmovingfromthesweetspot
![Page 99: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/99.jpg)
SQLDrivenWebsitesManywebsitesarebackedbyadatabase
PHPmakesiteasyConsiderWordPressandotherCMSs
LotsofunstructuredcontentStuffinblobsandtextfields
KeypropertiesScalingACID:Atomicity,Consistency,Isolation,Durability
TransactionsConcurrentaccess
Thereisa thatisstillgoodreading,espchps -
keyhistoricaltext11 12
![Page 100: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/100.jpg)
![Page 101: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/101.jpg)
CSV&SQLProgramsOnTheWeb
Othergovernmentrepositories:data.govdata.gov.uk
Scientificsitescinicaltrials.govuniprot.org...
UNDatarepository
![Page 102: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/102.jpg)
GoogleQueryVizLanguageASQLlikelanguage
UsedinGoogleDocsSpreadsheetQUERYfunctiontakesqueriesasargument
![Page 103: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/103.jpg)
WebSQLTheWhatWGandW3CtriedtostandardizeWebSQL
Thisspecification introducesasetofAPIs tomanipulateclient-sidedatabasesusingSQL.
Localdatabasebackedwebapps
ForofflineuseJustincreasedcapabilities
function prepareDatabase(ready, error) { return openDatabase('documents', '1.0', 'Offline document storage', 5*1024*1024, function (db) db.changeVersion('', '1.0', function (t) { t.executeSql('CREATE TABLE docids (id, name)'); }, error); });}
![Page 104: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/104.jpg)
![Page 106: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/106.jpg)
ReadingThereisa thatisstillgoodreading,
espchps -keyhistoricaltext
11 12
![Page 107: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/107.jpg)
AnyQuestionsSoFar?
![Page 108: Modelling Data On The Web - University of Manchesterstudentnet.cs.manchester.ac.uk/pgt/COMP60411/slides/… · · 2017-09-29design or analyse a data management system, ... You might](https://reader031.fdocuments.net/reader031/viewer/2022030416/5aa28e547f8b9a84398d3646/html5/thumbnails/108.jpg)
Labs&CourseworkNext,wegototheLabsYoulookinBBatWeek1coursework:
QuizQ1ShortEssaySE1SmallModellingexerciseM1SomequeryingCW1
Read,think,askus!