Open Testbeds deliverable-final - RITICSritics.org/wp-content/uploads/2018/07/Open... · PETRAS The...
Transcript of Open Testbeds deliverable-final - RITICSritics.org/wp-content/uploads/2018/07/Open... · PETRAS The...
OpenTestbedsforCNI
AuthorsChrisHankin(editor),ImperialCollegeLondon
DeephChana,ImperialCollegeLondonBenGreen,LancasterUniversityRafiullahKhan,Queen’sUniversityBelfastPeterM3,NationalCyberSecurityCentrePeterPopov,CityUniversityofLondonAwaisRashid,LancasterUniversityandUniversityofBristolSakirSezer,Queen’sUniversityBelfast
1Introduction:Rationale/JustificationIndustrialControlSystems(ICS),composedofcombinationsofhardware,softwareandICTnetworks,orchestratethemyriadoffunctionsneededtoexecutecomplextaskssuchasthedeliveryofutilityservicesandtheoperationofintricateanddisparatemanufacturingprocesses.ICSareexamplesofcyber-physicalsystems–digitalsystemsthataffectandareaffectedby,physicalprocesses–whoseuseisgrowingthroughdevelopmentsinsmart-citytechnologiesandtherapidemergenceoftheInternetofThings.Suchsystemsareincreasinginimportanceastechno-socialcomponentsoftheCriticalNationalInfrastructure(CNI)ofthefutureandastheyextendtheirscope,becomingubiquitous,accessibleandtransformativetowidersocietyandtheeconomy,theneedtounderstandtheirsecuritycharacteristicsalsoincreases.TodatetheResearchInstituteinTrustworthyIndustrialControlSystems(RITICS)activityhasfocussedonidentifyingexistingtechnicalandpracticalproblemsthatsurroundthedevelopmentofsecureandtrustworthyICS.InordertodeveloprealisablesolutionstotheseproblemsRITICShasconductedaresearchprogrammethatincludesworkin:
• Theoryandanalysis• Simulationandexperimentation• Testingandimplementation
Toeffectivelyexecutethismission,theneedforasimulation/labspacewherecomponentsandinstancesofinvestigateddigitalsystemsmaybephysicallyconfiguredfor'closetoreal-world'fidelityisvital.RITICSpartnershavedevelopedsmall-scaletestbedfacilities.Thiswhitepapersurveysthecurrentrangeoffacilities,summarisesthelessonslearnt,presentstheissueswithlinkingthesefacilitiesandconcludeswithaforwardlook.Ourambitionistointerconnecttheexistingsystemstogetherinordertoachievethescaleofreal-worldsystemsandtousethecapabilitiestoaccelerateandincreaseefficiency/effectivenessoftheUKinvestment.Thiswillenableusto
• betterunderstandtheinterdependenciesbetweendifferentsectors• betterunderstandthesimilaritiesanddifferencesbetweenInformationTechnology(IT)and
OperationalTechnology(OT)• testandpreparefortargetedanduntargetedattacks• providetrainingtoclosetheskillsgap• validatevarioustheoriesabouthowtodealwithnewandunknownthreats• extendunderstandingofsystem-userrelationshipsacrossanarrayofsectors
TherewillalsobetheneedtodevelopabusinessmodelforhowtheOpenTestbedsmightoperate.
2.TheUKLandscape
TheLancasterICSTestbedThemostextensivetestbedfacilitiesdevelopedwithinRITICShavebeendevelopedbytheMUMBAprojectattheUniversityofLancaster.Inadditiontotheirlab-basedtestbedwhichcanbeconfiguredinanumberofways,theyhavealsodevelopedatable-topwatertreatmentdemonstrator.
Ahigh-levelviewofLancaster’sICStestbedisshowninFig.1[1].ThearchitectureisbasedonthePurdueReferenceArchitecture.CurrentlysplitacrosssixManufacturingZones,anICSDemilitarisedZone,andanEnterpriseZone(withitsownseparateDemilitarisedZone),allequipmentinthetestbedisphysical(unlessotherwisenotedasVirtualisationPlatforminFig.1).ItisimportanttonotethatLancaster’stestbedhasfocusedonthedevelopmentofsystemsanddevicesacrossLevels0,1,2,3,DMZand4ofthePurduemodel.
Figure1:NetworkDiagramofSecurityLancaster’sICSTestbed
ITRC’sDAFNIprojectTheInfrastructureTransitionsResearchConsortium(ITRC)isaconsortiumof7universities(Cambridge,Cardiff,Leeds,Newcastle,Oxford,SouthamptonandSussex),investigatingwaystoimprovetheperformanceofinfrastructuresystemsintheUKandaroundtheworld.Theirresearchishelpingbusinessesandpolicymakerstoexploretheriskofinfrastructurefailureandthelong-termbenefitsofinvestmentsandpoliciestoimproveinfrastructuresystems.
TheDataandAnalyticsforNationalInfrastructure(DAFNI)projectwillcreateanationalinfrastructuredatabaseforvisualisationandanalysis.Itwillbeashared,securesystemforacademicresearchandaresourceforbusinesses,innovatorsandpolicy-makers.
DAFNIwelcomesideasabouthowitwillworkwithdifferentpartners,andonissuessuchas:security,access,suitablebusinessmodels;andnextsteps.
AkeyfeaturewillbeDAFNI’ssimulationandvisualisationfacilitiestoallowuseofmodelsinamoreflexibleway,enablingthesystemsofsystemsanalysisandincorporatingobservedandsimulateddatasets.DAFNIwillbenefitfromtheexperienceoftheITRC)whichhasbeendevelopingaone-stopdatabaseforUKinfrastructure(NationalInfrastructureSystemsMODel–NISMOD).It’smuchmorethanacurationofdata,andallowsrepresentationofinterdependenciestoinformplanningdecisions,includingviaavisualisationdashboard.AlthoughNISMODcontainsover400datalayers(representingmultiplesectors,demographics,economics),theinfrastructuresectorneedsgreaterdetail,torepresentindividualbuildingsandtodevelopplausibleconnectivitynetworks,whichDAFNIcandeliver.ITRC-Mistralisdevelopingameta-databasetogiveuserstheexperienceofasingleinterface,althoughitbringstogethermanydatabases,andthisisthemodelthatwillbeappliedtoDAFNI.
OnechallengewillbehowtomakeDAFNIsuccessfuloperationally.DAFNI’svisionistobuildanenvironmentwherepeoplecantrydifferentsolutions,whichmeansbeingresponsivetoallusers.ExistingmodelsmightincludeJASMINthatusesthedesktopasaservicetoolusingastandardtoolkit,withnorestrictionsonusers.
5GTestbedsTheDepartmentofDigital,Culture,MediaandSport(DCMS)areinvestingina5GtechnologytestnetworkaimingtoputBritainattheforefrontofthenextwaveofmobiletechnology.
5GresearchinstitutionsatKing’sCollegeLondonandtheUniversitiesofSurreyandBristol,havebeenawarded£16mtodevelopthecutting-edge5Gtestnetworkwhichwillbringacademiaandcommercialcompaniestogethertotrialthetechnologyandmakesurepeopleandbusinessescanrealisethebenefitssooner.
Thistestnetworkwilltrialanddemonstratethenextgenerationofmobiletechnologyandisthefirstpartofafour-yearprogrammeofinvestmentandcollaborationintheGovernment’snew5GTestbedsandTrialsprogramme.
Theuniversitieswillworktogethertocreatethreesmall-scalemobilenetworkswhichtogetherwillformthetestnetwork.Eachnetworkwillhaveanumberoftheelementsexpectedinacommercial5Gnetwork-includingmobilesignalreceiversandtransmittersandthetechnologytohandle5Gsignals-tosupporttrialsofitsmanypotentialuses.
Otheracademicinstitutions,industryandlocalauthoritieswillalsobeabletobidforfurtherfundingtobepartofthisprogrammefrom2018/19onwards.Furtherdetailsonopportunitiesandthefundingavailablearepublishedintheprospectus.
UKCRICTheUKCollaboratoriumforResearchinInfrastructure&Cities(UKCRIC)willprovideleadershipandsupportforthedevelopmentandgrowthofacoordinatedandcoherent,worldclass,UK-basednationalinfrastructureresearchcommunity,spanningatleast14universities.Itwillengagegovernment,cityandcommercialpolicymakers,investors,citizensandacademiainajointventurethatdrivesinnovationandvaluecreationintheexploitationofservicesprovidedbynationalinfrastructure.Throughcentralcoordination,providingafocalpointforknowledgetransfer,UKCRICwillsupportastep-changeinthenation’sapproachtoinfrastructureinvestment.Itwillalsodevelopacommercialresourcethathasconsiderableexportpotentialforaninternationalmarketthatisvaluedat$57trillionintheperiodupto2030.
UKCRICwillunderstandhowtomakethesystemofsystemsthatconstitutesthenation’sinfrastructuremoreresilienttoextremeeventsandmoreadaptabletochangingcircumstancesandcontexts,andhowitcanprovideservicesthataremoreaffordable,accessibleandusabletothewholepopulation.
PETRASThePETRASHubhasfundingforthecreationofanumberofdemonstrators.Theprojectisstilldebatingwhatformtheseshouldtake.
UniversityofBristolWiththeMumbaprojectteam’smovetotheUniversityofBristol,anewICStestbedisbeingsetupthatwillincludemultiplefieldsitesandindustrialprocessestosupportresearchonsecurityofindustrialcontrolsystems,includingbothlegacyandnon-legacydevicesandIndustrialInternetofThings(IIoT).
3.InternationalFacilitiesHolmetal[3]presentanoverviewofinternationalfacilitiesasattheendof2015.Theyidentify30testbedsthatwereeitherplannedorinoperation,almosthalfofwhichwereintheUS.TheyciteSiaterlisetal[4]whopresentthefollowingcriteriathatcybersecuritytestbedsshouldfulfil:
• Fidelity:tobeasaccuratearepresentationoftherealsystemaspossible• Repeatability:repeatedrunsshouldgiveconsistentresults• Measurementaccuracy:observingrunsshouldnotperturbtheoutcome• Safeexecutionoftests:theeffectofatestshouldbecontainedwithinthetestbed
Thesearereasonablerequirementstoexpectofanytestbedfacility.
The iTrust Water testbeds (Singapore) are small-scale networks within a controlled laboratoryenvironment,composedofasmall-scalewaterdistributionnetwork (WADI)anda treatmentplant(SWaT). The testbeds are used for security analysis for water distribution networks, to assessdetectionmechanismsforcyberandphysicalattacks,aswellastounderstandcascadingeffectstootherconnectedsystems.The[iTrust]InternetofThingsAutomaticSecurityTestbed(Singapore)isasmall-scale laboratory composed of GPS simulator, Wi-Fi localization simulator, time simulator,movementsensor,tosimulatethedifferentenvironmentalconditionsinwhichIoTdevicesoperate.Thetestbedsupportsstandardandcontext-basedsecuritytestingandanalysisforIoTdevicesunderrealconditionsagainstasetofsecurityrequirements.Power-Cyber(USA)isasmartgridtestbedwiththepurposetoperformvulnerabilityassessment(i.einspect weaknesses within the infrastructure), design mitigation methods, and develop cyber-physical metrics (i.e metrics combining cyber-physical properties), cyber forensics tools (exploreways to detect cyber-attacks specific to industry protocols and field devices), and securemodels(explorationofinnovativesecurityapproaches).TheUniversity of Illinois atUrbanaChampaign has developed the Cyber-Physical ExperimentationEnvironmentforRemoteAccessDistributedICS(CEER).Asummaryofthiseffort(includedverbatimhere) has been extracted from: https://iti.illinois.edu/research/energy-systems/cyber-physical-experimentation-environment-radics-ceer
“Thegoalofthisprojectistoprovideatestbedonwhichprospectivetechniquesandtoolscanbedeveloped,refined,andvalidatedinacontextwithunprecedentedsystemfidelity.Weareclosingthegapbetweenneedsandstateoftheartthroughatestbed,CEER,thatisinnovativeinseveralways.
CEERbringstotheICSdomainforthefirsttimeproductionqualitysoftwaretoflexibly(andremotely)defineexperiments,configuretestbedresources,andrunexperiments.Itbringsthefruitsofstate-of-the-artmodelingofgridsystemstoprovidesyntheticbutrealisticdynamicgridstate.Itbringscutting-edgeappliedresearchintemporalcoordinationofrealdevices,deviceemulation,andsimulatorsofdiversekindstoenablecreationofexperimentaltopologiesthataremuchlargerthantheensembleofphysicalICSdevicesinthetestbed.CEERbringsbest-of-breedICSsysteminstrumentationandmonitoringtechnologytoenableuserstocloselytracktheresultsoftesting.Itwillbeabletoaccuratelyrepresentthesmartgridinteractionsfromgeneration,transmission,anddistribution.Itwillalsosupporthigh-fidelityexplorationofassetsineachofthesedomains,including,butnotlimitedto,generationassets,gridcomponentsintransmissionanddistributionsubstations,controlcenteroperation,andadvancedmeteringinfrastructure.”
Anapproachtakenbythecolleaguesbehindthistestbedistousehigh-fidelitysimulatorsofthe“physicalworld”,whichallowsforclose-to-trueimpactofcyber-attackstobeaccountedfor.
TheUSNationalInstituteofStandardsandTechnology(NIST)isdevelopingacybersecuritytestbed(seeFig.2[2]).
Figure2:TheNISTTestbed
Theaimistomeasuretheeffectofprevailingstandardsandguidanceontheperformanceofcontrolsystems.Thetestbedisdesignedasaseriesofenclavesthataddressdifferentindustrialsectors.ThetestbedusessimulationwhereappropriatewithHardware-in-the-Loop(HIL)componentssimulatingtheinterfacesbetweensensors/actuatorsandthecontroller.Thedifferentenclavesallowstudyofcontinuousprocesses(suchaschemicalmanufacture),discreteprocesses(suchasautomotiveassembly)andhybridprocesses(suchaspharmaceuticalmanufacture).Performanceismeasuredusingappropriatetechnicalperformanceindicatorsfortheprocesses.
TheDepartmentforBusiness,EnergyandIndustrialStrategyCivilNuclearTeamisintheplanningphaseregardinganupcomingjointexercisewithEstonianMODofficials.TheEstonianofferisanetworkdefenceexercisescenariomakinguseofafully-equippedcybertestrangeinTallinn,andwillinvolveparticipantsfromacrossthecivilnuclearsector.TheNationalCyberSecurityCentrewillalsohaveinvolvementintheexerciseusingtheEstonianDefenceForcesCyberRange,consistingofasimulatedofficenetworkconsistingoftypicalserversandworkstationsaswellasfacilitiessupportsystemswillbeimplemented.
HitachiaredevelopingaSecurityTrainingArena(SeTA)attheirOmikaWorksinJapan.Theemphasisofthiscentreistotrainoperatorshowtodealwithcyberincidentsinanuclearpowerplant.TheyplantorunjointexerciseswiththeUK(andpossiblyUS)in2018.TheyhavehadpreliminarydiscussionswithImperialCollegeLondonandRoyalHollowayUniversityofLondonaspotentialacademicpartnersinthisprogramme.
ThelargestsecurityclusterinEuropeissituatedinTheHagueSecurityDelta.In2015,theypublishedaproposalforanational,multi-sectortestbed[5].Atthetimeofwriting,theyarestillrecruitingpartnerstoassistintheconstructionofthefacility;theinitiativeissupportedbyTNO,KPNNVandtheMunicipalityofTheHague.TheminimumrequirementsfortheDutchnationaltestbedareasfollows:
• Theplatformshouldhosttestlabsformultiple,differentcriticalinfrastructuresectors• Theplatformshouldgenerateknowledgethatcanbeusedtocreatesolutionsforcritical
infrastructureequipment• Itshouldbeavailablefortrainingofinformationsecuritystaffonthreatsandexploits• Thetestbedshouldfacilitatethecreationofanetworkofhighlyqualifiedinformation
securitystaff• Thetestbedfacilityshouldperiodicallyproduceconfidentialreportsaboutnewlydiscovered
threatsandvulnerabilities• Thetestbedfacilityshouldprovideopenandfreelyavailablesecurityreportswiththe
securitysolution• Thetestbedfacilityshouldturnsecurityrequirementsintonewindustrystandards• Thetestbedfacilityshouldeducatecriticalinfrastructurecompaniesinbestpracticesand
lessonslearnedfromacrossallsectors• Thetestbedfacilityshouldestablishcooperationandinformationsharingamong
participatingpartners
TheserequirementsoverlapsignificantlywiththeambitionthatweoutlinedinSection1above.
4.DesignIssuesandLessonsLearntThekeydesignissuesandlessonslearntfromtheconstructionoftheLancastertestbed[1],whichalsofindechoesintheothercitedpapersare:
1. Theneedtoinclude,eitherphysicallyorvirtually,adiverserangeofdifferentdevices(vendorsandversions)
2. Theneedforscaletoprovidefaithfulrepresentationsofrealsystems3. Appropriatemechanismstomanagethecomplexityoftheinfrastructure
DiversityAneffectivetestbedshouldbeabletomimicavarietyofICSsetups.Keyquestionsinclude:
1. Selectionofdevicesandprotocolsforinclusion;2. Providingdifferentconfigurationsofdevices/manufacturerstypicalinICSsettings;and3. Balancingdeviceandprotocoldiversityagainstotherrequirements,suchasthe
implementationofthephysicalprocessitself.
ExperienceswithintheLancastertestbedhavehighlightedthat[1]:
• Deviceandtechnologyselectionsshouldbemarket-driven;• Fieldsitesinatestbedshouldrepresentdifferentreal-worldscenariossuchashomogeneity
andheterogeneityofvendorsaswellascombinationsoflegacyandnon-legacydevices;• Processdiversitycanhelpmodelstealthattacksthatexploitphysicalaspectsoftheprocess
butthatsuchprocessdiversitymaybetraded-offinfavourofdiversityofdevicesandfieldsites.
ScaleSoftwaredoesnotprovidesimulationsofmanyessentialtypesofdevices,i.e.fromdifferentvendorsorthesamevendorbutdistinctiveversions.Theaccuracyandreliabilityofsuchsimulationsinmimickingreal-lifeoperationsalsoremainanissue.Therefore,whilethecostofphysicalequipmentcanbealimitingfactor,thebenefitsitcanbringinrelationtoexperimentalrigourisanoverridingconstraint.Ontheotherhand,virtualisationandVLANscanprovideeaseofintegrationandscalingofthetestbedinfrastructure[1].
ComplexityAlthoughtheunderlyingarchitecturemaybecomplexandinvolveanumberofnetworkzones,thisshouldbeastransparenttotheuseraspossible.Transparencycanbeachievedbyprovidingasinglepointthroughwhichaccesstoandextractionofdatafromthedifferentzonescanbemanaged.AsecondlessonlearntbytheLancasterteam[1]isthenecessitytocreateandmaintaingooddocumentationofthetestbedasitevolves.
FurtherLessonsTheNSFreportonCybersecurityExperimentationoftheFuture[6]providesadetailedroadmapforthedevelopmentoffutureexperimentationinfrastructureoverthenear-term(3years),mid-term(5years)andlong-term(10years).Thereportalsoreviewstheexperienceof46US-basedexperimentalfacilities.
Thetop5recommendationsfromthereportareasfollows:
• Focussingonmultidisciplinaryexperimentationdrawingonboth“hard”sciencesandsocialsciencewillhavethegreatestimpactinacceleratingcybersecurityexperimentationinthenearterm.
• Theabilitytoaccuratelyrepresentfullyreactionarycomplexhumanandgroupactivityinexperimentswillbeinstrumentalinenvironmentsthatrealisticallyrepresentreal-worldsystems.
• Creatingopenstandardsandinterfacesisamid-termpriority.• Researchanddevelopmentusingthelatestadvancesindatascienceisneededtocreate
reusable,extensible,validatedexperimentdesigns.• Researchinfrastructuremustbeusablebyabroadrangeofresearchersandexperts,notjust
restrictedtocomputerscienceresearchers.
5.LinkingTestbedsMostUKacademicinstitutionsandresearchcentresareprovidedconnectivitybyJanet,ahigh-speed,secureandreliableworld-classnetwork.Janetprovidesatleasta10Gb/sphysicallinkandaClass-BIPaddresspool,enablingalltypesofInternetserviceswithinUKacademiccampuses,includinglow-latencyVoiceoverIP(VoIP).However,experimentallabfacilitiesandresearchtestnetworksaresignificantlyconstrainedintakingadvantageoftheJanetinfrastructure.InordertoavoidsecurityandQualityofService(QoS)relatedthreatstotheJanetnetwork,researchlabfacilitiesare,inmostcases,disconnectedandrelyonexternalmulti-megabitADSLlinesvialocalISPproviders.Theselimitationsnotonlyconstrainresearchcapabilitieswithintheseinstitutions,butalsoimpairnationalandinternationalcollaborationsthatrequirehigh-speedconnectivityamongstcollaboratingpartners.
Anotherkeyfactorthatlimitstheresearchcapabilities,qualityandeffectivenessisthelimitedavailabilityofresourceswithinacademicinstitutionsandresearchcentres.SettingupanexperimentallabfacilityorresearchtestnetworkisextremelyexpensiveespeciallyintheICSandSCADAdomainduetotheneedforexpensivedevices/equipment.Thenon-availabilityofstate-of-the-artexperimentalresourcessignificantlylimitsresearchpotentialofindividualacademicinstitutions.
Theaimforlinkingtestbedsistoenableallpartnerinstitutionswithleadingedgeresearchcapabilities,experimentallabfacilitiesandtestnetworksbysharingresourcesoversecureandreliablehigh-speedJanetinfrastructure.TheteamatQUBhaverecentlyproposedanapproachwhichisuniqueofitskindbyinter-linkinglabfacilitiesofalluniversitiesacrosstheUKasshowninFigure3.ThisproposaldoesnotjustfocusonlinkingICSandSCADAfacilitiesbutproposesamoregeneralnetworkoftestbeds.
AshighlightedontherightinFigure3,thesharingnaturewillenableallparticipatinginstitutionstobenefitfromthesamesearchfacilitiesandhaveaccesstotest/experimentalnetworkswhichtheywerelackingindividually.TheproposedresearchnetworkwillbebuiltuponJanet’snetworkinfrastructureusingconfigurablemulti-gigabitVPNtunnels,providingconnectivityofupto10Gb/samongstthepartners,whilefacilitatingstrictisolationfromeachnode’smaincampusnetwork;asimilararchitectureisalreadyunderevaluationtoallowexternalconnectivitytotheLancastertestbed.Centralisednetworkadministrationandmanagementwillprovideprojectspecificconfigurationofthenetwork(topology,bandwidth)andexternalconnectivitytonationalandinternationalpartners,andtheInternet,viaasecuregatewayusingJanetandthird-partyISPs.The
baselinearchitecturewillbelaidoutassuchthatthenetworkcanbescaledtoexpandbeyondthecurrentpartners,capableofservicingtheUKacademicresearchcommunityformanyyearsahead.
Figure3:AProposedUKNetwork
DirectBenefitstoUKUniversitiesandResearchCentresTheacademicandindustrialpartnerswillbeabletotakeimmediateandfulladvantageoftheproposedconnectivitybyprovidingthem:(i)anextendedresearchnetworkinfrastructureforexperimentalstudies,(b)accesstothecollectivepoolofstate-of-the-artexpensivetechnologies,tools,testbedsanddatasets,whicharecurrentlyonlyavailabletotheowninginstitution,(c)providethenecessaryinfrastructureforgeneratingresearchspecifictrafficsamples,logfilesandotherspecialistdatasets,andthefacilitytosharealargerepertoireofexistingdatasetsamongstthepartners,(d)limitedaccesstospecialistnetworkingskillsanddomainknowledgebytakingadvantageofthenetworkanditsdedicatedstaff.
Thescaleanddiversityoftheproposedresearchnetworkandtestfacilitiesprovidesuniqueresearchopportunitiesforendusers,suchasenterprisesandcorporationsrelyingonlargeITnetworksandITsecurity.Theproposedresearchnetworkwillprovidenumerousbenefitsforpartnersbysharingorprovidingaccesstoexpensiveandrareresources,accesstomorerealisticexperimentalenvironmentandimprovingresearchcollaboration.
QUBhasanextensiveexperimentalnetworkandtestlabinfrastructurecloselycoupledtoasystempenetrationtestandtraininglab.Theinter-linkedresearchnetworkwillextendthattestcapabilityandprovideamorerealisticanddistributedICSsystemtoexperimentwith.Inadditiontothe
Aberdeen
Dundee
EdinburghGlasgow
Belfast
Liverpool Manchester
Leeds
Sheffield
LeicesterAberystwyth
Birmingham
Cambridge
Cardiff London
Southampton
Plymouth
Bristol
Oxford
Canterbury
Lancaster
Layer3VPNTunnelLayer2LightPath
janetQUB
UoE
USW
UE
DMU
UB
Internet
ISPJanet
Gateway
Network AdministrationManagement and SecurityTraffic monitoring and acquisition
RHUL
Lanc.Bris.
benefitsofscalethatwillaccruetoexistingRITICStestbeds,suchastheoneatLancaster,thenetworkwillalsobenefitothernewRITICSpartners.Forexample,theplannedtestbedatBristolwillbelinkedintothenetworkaswellasthoseuniversityfacilitiescurrentlysupportedbyAirbus.ThecollaborativeprojectbetweentheUniversityofSouthWales(USW)andAirbusDefenceandSpacecalledSCADACyberSecurityLifecycle(SCADA-CLS),istargetingthedevelopmentofacyberforensiccapabilityforSCADAprocesscontrolsystems.Theinter-linkedresearchnetworkwillprovideanextendedICSnetworkthatUSWcaneffectivelyutiliseforforensic/incidentmanagementtriageprocessmodelling,andthedevelopmentofSCADAforensictoolsfordataacquisition,incidentmanagementandsituationalawareness,usingSCADAtestfacilitiesatQUBanditsFP7/H2020partners.DeMontfortUniversity’s(DMU)CYRANcyberrangetechnology,whichwillbedirectlyaccessiblebyallpartners,providesaplatformforcyber-attack/defencescenariosforexperimentalresearchandforeducationalgamesthatincludephysicalartefactssuchasPLCcontrolledproductionlinesandfiltrationsystems.DMUwillbeabletoextendtheCYRANcapability,accessingPLCcontrollersatQUB,USWandEuropeanpartners.SCADAandothertypesofICSrelatedlargedatasetscannowbegenerated,takingadvantageoftheadditionalphysicalresourcesfromthepartners’testbeds.ThisdataisdirectlyrelevanttoexistingprojectswithinDMUonSCADAForensics,undertakenwithAirbusGroupInnovationsaswellasresearchonPrivacyMetricsandIncidentResponsemanagement.
Bycombiningmanytestnetworkswithuniqueproperties,moregeneralcybersecurityresearchprojectswillalsobenefit.Logfilesfromnextgenerationfirewalls(ngFW)withintheproposedresearchnetworkwillbeusedtoanalysemalicioustrafficinLANnetworks;workingwithmultiplengFWdatawillenabletheanalysisacrossawideareanetwork.Akeybenefitisthegenerationoflargelogfileswithintheexperimentalnetworkwithoutbeingconstrainedbytheprivacyandethicalchallengesoflivecampusnetworks.Dedicatedmonitoringandinterceptiontechnologywithintheproposedresearchnetworkwillprovideadvancedtrafficvisibilityandpacketprocessingcapabilityformanyprojects.Theproposedresearchnetworkwillallowpartners(a)tofurtheranalyserepetitiveexternalattackstotheirITinfrastructurebyreplayingattackpatterns,(b)usecross-sitetestcapabilitiestoundertakestressandpenetrationtestingonneworexperimentalsecurityandnetworkappliances,and(c)assessnewcybersecurityarchitecturesandthreatmitigationstrategiesoncorporatenetworksandwebsites.
Datasetandtesttrafficgenerationandsharingisoneofthemostimportantandchallengingtopicsinnetworkandcybersecurity.Availabledatasetssuchasinterceptedtrafficareconstrainedandinmostcasesrelevanttoaspecifictypeofthreat.Privacyandethicalconsiderationspreventtheuseofanyintercepteddata,suchasfromaUniversitycampusnetwork.Furtherconstraintsarethatmalware,APTandDDoSrelatedprojectsrequiresfreshdatasetsandtrafficcontainingtargetedthreatsinordertounderstandtrafficpatternsrelatedtothreats,andforoptimisingdetectionalgorithmssuchasmachinelearningclassifiers.Theproposedresearchnetworkbringstogetherhighlydiversetestnetworksatascaleandthetrafficcapacityofalargenetwork,providingauniqueopportunityforgeneratingtailoreddatasetsandsampletraffic.
Transportexperimentallabfacilitiesarequiteexpensivetoestablishandonlyfewuniversitieshaveadvancedtestbeds.Theinterconnectedresearchnetworkwillbeofsignificanceforimprovingcollaborationamongstacademicinstitutionsandeffectivelysharingtheirtransportlabfacilities.BirminghamCentreforRailwayResearchandEducation(BCRRE)oftheUniversityofBirmingham(UoB)hassignificantexperimentallabfacilitiesforresearchinaddressinggrandsystem-wideaswellascomponentlevelchallenges.UoBrailwayresearchcoversvariousaspectsincludingsafety,operationsandmanagement,dataintegrationandcybersecurity.Recently,theUKRailResearch
andInnovationNetwork(UKRRIN)researchcentrehasbeenestablishedforsupportingnewinnovationsinrailtransport.UKRRINaimistobringtogetherexistingfacilitiesatdifferentacademicinstitutionsandaccelerateinnovationandnewproductdevelopmentintherailindustry.AspartofUKRRIN,UoBwillcarryoutresearchindigitalrailsystemscoveringcybersecurity,trafficmanagementandrailwayconditionmonitoringandsensing.UniversityofNewcastle(UoN),LoughboroughUniversity(LU)andUniversityofHuddersfield(UoH)withinUKRRINwillcollaborateonhighvaluerollingstocksystems,assetoptimisationandthrough-lifemanagementandenergymanagement.Whereas,UniversityofSouthampton(UoSA),UniversityofSheffield(UoS),UniversityofNottingham(UN)andHeriot-WattUniversity(HWU)arecarryingoutresearchonrailwayinfrastructurewithinUKRRIN.TheproposedinterconnectedresearchnetworkwillbethemediumenablingallpartnersofUKRRINtocollaborateeffectivelyandshareexperimentalresources.
ProposedSystemArchitectureTheproposedinter-linkedresearchnetworkwouldbedevelopedinmultiplephases,takingadvantageoftheavailableJanetconnectivityandsparebandwidthcapacityoftheacademicinstitutes.
Figure4outlinestheoverallnetworkandtestbedarchitectureamongstthepartners.Aninitialphasewouldtargetthedevelopmentofthebasicoverlayarchitectureontopoftheexistinglayer3Janetconnectionviamulti-gigabitVPNtunnelsandtheestablishmentofthenetworkwiththenecessarynetworkadministrationandmanagementtoolsandsupportresources.Acontrolcentrewithnetworkadministrationandmanagementtoolswouldbeestablished,responsiblefortheadministrationandmanagementofthelinksamongstthepartners,JanetandtheexternalconnectivitytotheInternet.Phase-2developmentwouldprovideadditionalphysicallinkcapacitiesandexternalconnectivitytointernationalandindustrialpartners.
Figure4:TheProposedArchitecture
Janetlayer2LightPathprovidesthelowestcostandhighest-bandwidthconnectivity,fortheproposednetworkitalsointroducessignificantchallengesprovidingthenecessarylayer3networkinfrastructureandsupportingcybersecurity(malware,DDoS,cloud),IoT,andIndustrialcontrol
relatedtestlabs.NumerousSDNandcloudnetworktestbedscantakefulladvantageofthelayer2bandwidthsuchasstreamingterabytesofdatabetweenbig-datalabs.
RequirementsforSecureInter-linkingofDiverseTestbedsTheobjectiveistodevelopaplatform,basedonasecureoverlaynetworkarchitecture,forinterconnectingvariousacademicandindustrialtestbedsintoalargerUKwideresearchnetwork.AsdepictedinFigure4,suchaprivateoverlaynetworkapproachhasthreebasicrequirements:(i)networkconnectivitykit,(ii)centralizedoperationsandmanagementand(iii)highspeedJanetnetwork.
NetworkConnectivityKitThenetworkconnectivitykitenablesremotetestbedsitesfrompartnerstobeconnectedtotheprivateoverlaynetworkasshowninFigure4.Thesolutionisscalableandnewtestbedscanbeeasilyintegratedwithinthenetworkwithoutmajortechnicalsupport.Tobecomepartoftheinterconnectedphysicaltestbedinfrastructure,eachacademicorindustrialpartnershouldbeprovidedwithanetworkconnectivitykitorrack-mountedkit.Thebasicarchitectureoftherack-mountedkitisshowninFigure5andconsistsof:
• AFirewall/Router/VPNwhichwillbemanagedfromtheCSITTestNetworkManagementcentre.
• Adistributionswitchthathasportmirroringcapabilitiestopermittrafficcapture.• Trafficdatastoragecapability.Terabytesofnetworktrafficdatamayneedtobecaptured
andstoredforlateranalytics.
Figure5:ArchitectureoftheNetworkConnectivityKit
ThenetworkconnectivitykithasaVPNclientandappropriatelydimensionedcommunicationandstoragehardware.ItistailoredfortheproposedresearchnetworkcomprisedofappliancesforatraditionalIPnetworkandanSDNnetwork,capableofsupportingadvancedfirewall,andVPNtunnelwithVLANsegregationcapability.
OperationsandManagementTheproposedresearchnetworkhascentralizedmanagementformanagingconnectivitybetweendistributedtestbedsites,networkaccesscontrolanddataacquisition.Italsohasadata-set(sampleddataandtrafficpatterns)repositorywithpostprocessing,indexingandaccesscontrol.Aneffective
managementandadministrationstructureisessentialtoensurethesuccessoftheproposedresearchnetworkanditsefficientutilisationbythepartnersandthewiderresearchcommunity.
ThecentralizedoperationsandmanagementsystemwillbehostedattheCentreforSecureInformationTechnologies(CSIT)datacentre.TheproposedresearchnetworkwillbemanagedbyQueen’sUniversityBelfast,aspartofCSIToperationandmanagementinfrastructure,incollaborationwithallpartners.
HighspeednetworkThehighspeedlinkcapacitieswillbeleasedfromJanetwhichwillactasthebackboneandfabricoftheproposedresearchnetwork.ThemajorityofcostinthedevelopmentofproposedresearchnetworkisassociatedwiththeleasingofcommunicationlinksfromJanetandprovidingthenetworkconnectivitykittoeachpartner.GoingbeyondthecurrentQUBproposal,itshouldalsobeconsideredhow4Gtechnologies,asusedatLancaster,canbeincorporatedintothenetworkinasecure,reliable,andmanagedformat.TherecouldbeafurtherextensiontowardsPSTN/GSMservices,inwhichlegacydial-uptechnologiesmayalsobeapplied.
UseCaseExamplesInter-linkingexperimentalresourcesfromacademicandindustrialpartnersmakestheproposedresearchnetworkquiteheterogeneousconsistingofdiversetestbedsinalldifferentresearchareas.Basedontheresearchtopic,apartnercanrequestresourcesinaspecificdomainfromcontroloperationsandmanagementcentre.TheControlcentrewillcreateasecuresegregatedVLANwithdedicatedexperimentalresourcesbasedontherequest.Theallocatedresourcescanthenbeexploitedbythepartnerstoexperimentanddetermineeffectivenessoftheirdevelopedtechnologiesandresearchtools.Toillustratetheutilityofthenetwork,thissectionpresentstwoICSusecaseswheretheproposedresearchnetworkcanbeutilized.
DistributedIntrusionDetectionandPreventionAnIntrusionDetectionSystem(IDS)monitorsanetwork/systemformaliciousactivitiesorviolationofpoliciesandraisesalerts.Whereas,IntrusionPreventionSystem(IPS)complementsIDSbyalsotakingdefensiveactionswhenamaliciousactivityisdetected.SeveralacademicinstitutesandresearchcentresareactivityinvolvedinIDS/IPSresearchtoimprovedetectionefficiencyandeffectivelyhandleemergingthreats.NISTpublishedrecommendationsthatIDS/IPSsystemsshouldbehybrid,distributedinnature,havedecentralizeddecisionmakingandcentralizedmanagementandrefinementofdetectedevents.ThehybridIDS/IPSsystemsperformbothhost-basedmonitoringaswellasnetwork-basedmonitoringformaliciousactivitiesdetection.Thedistributednaturesuggestsmultiplesensorstobedeployedinsysteminsteadofrelyingonasinglesensorforredundancyandbettermaliciousactivitiesdetection.
SeveralICSsystemsaredistributedinnaturee.g.,powersystems.ToinvestigateIDS/IPStechnologiesforadistributedICSnetwork,apartnercanrequestresourcesfromthecontrolcentreofproposedinterconnectedresearchnetwork.ThepartnerwillbenefitfromnothavingitsownbututilizingsharedICStestbedsfromotherpartnerinstitutes.Thiswillenablethepartnertocontinueresearchinthistopicevenifitislackingequipment.
Figure6:TheDistributedIDSscenario
RealisticExperimentalPowerSystemsPlatformTheproposedresearchnetworkcanbeusedtoconductresearchinamorerealisticdistributedexperimentalplatform.E.g.,powersystemsarehighlydistributednowadaysduetodevelopmentofrenewableandgreenenergysources(e.g.,windfarms,solarpanels,etc).Thistrendisbecomingmoreandmorecommonandgreenelectricitysourcesarepredominantlylocatedatgeographicallyisolatedareas.Severaluniversitiesareconductingresearchondistributedgenerationandtransmission,microgridsandsubstationsincludingQueen'sUniversityBelfast,ManchesterUniversityandStrathclydeUniversity.Distributedgenerationandintegrationintomaingridtakesbenefitfromsynchrophasortechnology.SynchrophasortechnologyincludesacontrolcentrethatreceivesGPStimestampedelectricalmeasurementsfrommicrogrids(ordistributedgenerators)andmaingrid.Controlcentreperformsprocessingtodetermineifamicrogridissynchronizedwiththemaingridandcanbesafelyconnectedtocontributeelectricitytothemaingrid.Normally,microgridscandynamicallyconnectanddisconnectfromthemaingridwhichincreasestheriskforpowersystems(ifconnectedinnon-synchronizedstate).Queen'sUniversityBelfasthasalocaltestbedondistributedgenerationandspecificallyresearchingsolutionstoensuresafety,resilienceandcybersecurity.Sincepowersystemsaredistributedinnature,suchsystemsneedtobestudiedinamorerealisticandgeographicallydistributedexperimentalplatform.AsshowninFigure7,theproposedinterconnectedresearchnetworkcanprovidesuchadistributedexperimentalplatformbycombiningresourcesavailableatotherpartnersaswell.ThiswillenableQueen'sUniversityBelfasttoexperimentwithanynewlydevelopedsafetyandsecuritytechnologiesinamorerealisticdistributedpowersystem.Further,partnersinterestedtoconductresearchinthisareabutlackingresourcescanalsobenefitbyaccessingsharedresourcesfromotherpartners.
Figure7:TheDistributedPowerscenario
6.FutureDirections/ConclusionsThiswhitepaperenvisagesaninter-linkednetworkofopentestbedfacilitiesthatwillsupportthegrowingRITICScommunityto:
• betterunderstandtheinterdependenciesbetweendifferentsectors• betterunderstandthesimilaritiesanddifferencesbetweenInformationTechnology(IT)and
OperationalTechnology(OT)• testandpreparefortargetedanduntargetedattacks• providetrainingtoclosetheskillsgap• validatevarioustheoriesabouthowtodealwithnewandunknownthreats• extendunderstandingofsystem-userrelationshipsacrossanarrayofsectors
TheproposalfromRITICSisambitiousandrequiresconsiderableinvestmenttorealisebutrecentlyannouncedNCSCfundingwillallowthedevelopmentofaprototypeinBelfast.WefeelthatcreatingsuchanationalfacilitywillallowtheUKresearchcommunitytomeetthecriteriaoutlinedaboveandrepeatedbelow:
• Fidelity:tobeasaccuratearepresentationoftherealsystemaspossible• Repeatability:repeatedrunsshouldgiveconsistentresults• Measurementaccuracy:observingrunsshouldnotperturbtheoutcome• Safeexecutionoftests:theeffectofatestshouldbecontainedwithinthetestbed
andplaceusinaleadinginternationalpositionforthiswork.
References[1]B.Green,A.Le,R.Antrobus,U.Roedig,D.HutchisonandA.Rashid:Pains,GainsandPLCs:TenLessonsfromBuildinganIndustrialControlSystemsTestbedforSecurityResearch.CSET@USENIXSecuritySymposium2017.
Layer 3 VPN TunnelLayer 2 Light Path
janet
Control Center
Academic Partner
Strathclyde University
Queen’s University BelfastManchester University
[2]R.Candell,D.M.AnandandK.Stouffer:ACybersecurityTestbedforIndustrialControlSystems.ISAProcessControlandSafetySymposium,2014.
[3]H.Holm,M.Karresand,A.VidstromandE.Westring:ASurveyofIndustrialControlSystemTestbeds.NordSec2015,LectureNotesinComputerScience,9417,SpringerVerlag,2015.
[4]C.Siaterlis,A.GarciaandB.Genge:Ontheuseofemulabtestbedsforscientificallyrigorousexperiments.IEEECommunicationsSurveys&Tutorials15(2),2013.
[5]TheHagueSecurityDelta:SecuringCriticalInfrastructuresintheNetherlands:TowardsaNationalTestbed.https://www.thehaguesecuritydelta.com/images/HSD_rapport_Testbed_EN.pdf
[6]D.Balenson,L.TinnelandT.Benzel:CybersecurityExperimentationoftheFuture(CEF):CatalyzingaNewGenerationofExperimentalCybersecurityResearch.http://cyberexperimentation.org/files/2114/5027/2222/CEF_Final_Report_Bound_20150922.pdf