The Next Generation Internet: Unsafe at any Speed?
description
Transcript of The Next Generation Internet: Unsafe at any Speed?
-
The Next Generation Internet: Unsafe at any Speed?
Ken BirmanDept. of Computer ScienceCornell University
-
Convergent TrendsExisting Internet exhibiting brownouts, security and quality-of-service problemsTalk of a next generation Internet offering 10 to 100-fold performance improvementsA new generation of networked applications includes large numbers of critical ones
-
Typical Critical ApplicationsMedical monitoring and clinical databases. Community health information networks. Remote home care and Remote telesurgery Integrated modular avionics systems. Air traffic control. Free flight, 4-D navigation
-
Medical NetworksContacted a number of technical and business people in this field (HP Careview, EMTEK, Hospital for Sick Children)Asked: What are the trends? How are networks changing healthcare?How are these systems made secure & reliable?Got any good stories for me?
-
An ICU Computer SystemBedsideClinical data server Digital library and online PDRLaboratories PharmacyDoctors office
-
a field in transitionDuring 1980s, hospitals used largely dedicated systemsClient-server architectures now becoming dominant, but trend is a recent oneSystems ran in physical isolation and had limited, mission-specific functionality
-
Important distinctionMedical monitoring equipment, computer controlled devicesThese practice medicineFDA regulated, like a drugSoftware subjected to extreme verification methods, safety certification is costly and hardExtends to the IEEE medical information bus for connecting bedside devices
-
Important distinctionMedical monitoring equipment, computer controlled devicesClinical data systemsBy definition, not considered safety criticalMaintain the legally binding patient recordThink of a database system. Human checks all entries, even data obtained directly from devices or lab reports.
-
Traditional ApproachEach runs as a separate networkDeveloped completely independentlyNo interconnection of any kind
-
Networking technology?Monitoring network is increasingly a dedicated real-time LAN, this permits configuration flexibility, remote telemetry, even adjustment of monitoring devicesClinical database system increasingly connected to laboratories, community health information networks (CHINs), physicians office, insurance and HMOs
-
Platform choices?Overwhelming trend is to introduce standard PCs and workstations, standard Internet technologiesForced migration from dedicated platforms to shared, standard network platformsWeb access now common from PCs that run clinical database software
-
bluring the distinctionIncreasingly, see monitoring network cross-connected to the clinical data networkSome physical isolation: not yet common to see an IV perfusion drip controllable over an internet within a hospitalPerimeter security using passwords, firewalls. But medical security needs are unusual; mismatched to standard solutions.
-
Creep of critical roleTechnically, clinical data system is non-criticalBut increasingly, the system actually is critical: doctors and nurses depend upon theAccuracy and timeliness of reportingCorrect data for lab results, vitals, medicationsFDA is simply late to catch up with trendsMoreover, already seeing Windows 95 and MS Access as basis for such systems
-
Consumer / society pullIntensive and growing cost pressuresDesire for freedom from medical system, home careConsolidation of hospitalsHMOs want to control care plan create trend towards remote telemedicine, even robot telesurgery, CHINs
-
Vision: A Virtual Private NetworkApplication shares the network with untrusted agents but is isolated from them.
-
Reality?Current VPN support approximates this, but configuration potentially awkward, slowMany CHINs wont use VPNsBy running over the Internet, CHINs are exposed to bandwidth fluctuations and denial of service from many causes
-
Good stories?Many cases of security or privacy violations (EMTEK has a good one). HP told me that some hackers accidently disrupted a cardiac monitor in the Boston area a few years ago (trying to track this down)Nutty nursing aide in Britain changed orders, discharged patients, scheduled testsHP Careview, starved for bandwidth, flickers on and offline in some critical care units...
-
Broad picture?Application trends outstripping technologyDecision making is by societal consensus, cost pressures, reflects HMO needs.Hospital executives insisting on standardsHospital network of future: PCs, off-the-shelf Internet software, standard Web stuff. Critical or not, like it or not, its happening!
-
What about aviation?Much use of computer technologiesFlight management system (controls airplane)Flaps, engine controls (critical subsystems)Navigational systemsTCAS (collision avoidance system)Air traffic control system on groundIn-flight, approach, international hand-offAirport ground system (runways, gates, etc)
-
What about aviation?Much use of computer technologiesFlight management system (controls airplane)Flaps, engine controls (critical subsystems)Navigational systemsTCAS (collision avoidance system)Air traffic control system on groundIn-flight, approach, international hand-offAirport ground system (runways, gates, etc)
-
ATC system componentsControllersAir Traffic Database (flight plans, etc)X.500 DirectoryRadarOnboard
-
similar turmoilOn-board systems moving to COTS, integrated modular avionicsBoeing 777 SafeBus a success storyUnlikely it could be replicated with standard O/S and standard ATM or LAN hardwareEmergence of 4-D navigation (free flight) systems: ground network penetrates level-A critical cockpit components.
-
Free flightGround systemOn-board conflict alertand resolution systemTransponder and GPS
-
Future avionics systemsGround systems rely increasingly on automation, have form of a highly available, highly critical network. Built using standard PCs, software toolsGround network becomes critical to flight safetyOn-board avionics are basically a dedicated real-time LAN built with standard PCs but perhaps non-standard O/S. One platform, many apps.Safety validation of components replaces current validation of system. Think plug n play
-
The list goes onDisaster warning and response coordinationPower management (grid control)Banking, stock markets, trading systemsComputer-controlled vehiclesMilitary intelligence, command and controlCritical business applications
-
Commercial Off The ShelfBuild using COTS Standard componentsBuy off the shelf, then harden themIntended to be cheaper, easier to maintainAs a practical matter, there is nothing else on the shelf!Roll-your-own solutions abandon powerful tools that make modern computing great!
-
Technology MountainCOTS
-
Reliable Technology MountainCOTS
-
Next Generation InternetCurrent Internet looks frailOnly government investment can address security, reliability, scalability and performance problems of the InternetExpectation is that well build it quickly, hence that we basically know how today
-
Next Generation InternetConcrete details?Seeks 10 to 100-fold performance improvementOriginally expected to provide IP-v6 interfaceOriginally expected to implementLong IP addressesIPSec, DNSecQuality of Service options over some form of Diffsrv (or RSVP) mechanism
-
Reality checkBoth IPv6 and RSVP now uncertain due to resistance from mainstream IPv4 crowdRSVP resource use on routers grows as O(n2)IPv6 would outmode a huge existing investmentHow likely is it that the NGI will solve the practical problems identified earlier?How does one build a secure, reliable, scalable, high performance network application, anyhow?Do we in fact know how to do this?
-
Glimpse of the IPv4 crowdThey gave us TCP/IP, core internet services, stuff on which we run email, webThey elevate the end-to-end argument to a religion (basically: packets, not circuits)Little experience with critical applications
-
What about QoS?Best scheme: DiffsrvUses an edge-classification of packets; routers look at just a byte or twoBut routers distort flow dynamicsYou send 50 packets per second but within the network, a router might see a burst of 100, then a second of silenceConsequence is that Diffsrv will be at best stochastic (and it also cant handle routing changes)
-
a troubling implicationIt seems unlikely that the NGI will easily support isolation of critical subsystems with the range of properties requiredMore likely: a tool for building virtual circuits (one-one connections) that run at very high speeds Missing connection is the step from the network to the robust application
-
What do we need?Isolation of functionsCritical functionality compartmentalizedComponents only interact through well-defined interfaces with well-defined semanticsDeveloper proves that implementation respects interface definition and semanticsOn the other hand, adequate performance is fundamental to providing robustness
-
Evidence for these claims?This is how modern avionics modules are built (wing flap and engine control, flight management system, inertial navigation)Process is extremely costly and works only for very small pieces of softwareSafeBus on Boeing 777 allows such software to share platform by creating very strong firewalls between components
-
Agenda emergesFind ways to divide and conquerTransform big nasty system into smaller independent modulesRun them in an environment that has strong properties, which the modules exploitResulting system has strong properties tooCan we apply this to familiar distributed computing problems?
-
PhilosophyImagine a network as an abstract data typeAn Overlay Network or ONWe can instantiate it multiple times, condition each copy with desired quality properties:A Virtual Overlay Network or VONHow to introduce properties?Mixture of resource reservation at routers, on a per-ON basis, and management actions at edges
-
A VONLooks like a dedicated Internet, although hosted on a shared infrastructureSupports guarantees of properties such asBandwidthNoise levelSecurity and freedom from denial of serviceTreated as an aggregate, not a set of pt-to-pt connections!
-
Making Vision a Reality1) NGI needs to give us the ON mechanism2) We need to implement VONs using fairly standard protocols over the base ONs3) Must be able to produce specialized solutions for reliability/security needs4) Solutions amenable to selective use of formal tools
-
NGI hooks?Diffsrv and RSVP wont do itCreates an O(n2) resource reservation problemProblem is that both schemes are fundamentally connection oriented, and VON concept is fundamentally multipoint in natureHence these point-to-point QoS mechanisms are not suitable for supporting VONsAny other options?
-
Switches supporting flows already existMCI, Sprint, AT&T already sell each other dedicated bandwidth with isolationThis is on a scale of perhaps 10s of flows and hence classification is easyVONs might mean that a switch would see thousands, but such scaling seems well within technical feasibility
-
Router understands flowsLooks like this
-
Router understands flowsLooks like thisLooks like thisLooks like thisActs like thisFlow 1Flow 2Flow 3Everything else
-
Things to noticeA flow in this sense aggregates all the traffic for one ON the identifier is for the ON not the endpointsClassification task is thus much smaller and resources needed to support this are linear in number of ONs that pass through the switch, not the number of potential connectionsEach ON is like a dedicated network
-
An ON hasA bandwidth guarantee (router sets resources aside on its behalf)Perhaps latency guaranteesCan offer isolation between flowsBut not much else
-
NGI part of the pictureNGI needs to give us raw ONs but also:Robust routing infrastructureNamingAbility to build an ON tolerant of one link or router failureMany building blocks are already in placeBut the core Internet community is balking on all forms of QoS: isolation or other guarantees seen as inconsistent with end-to-end philosophy
-
But suppose we get our wishNext President declares moral equivalent of war after continuing Internet siege shuts down his web site during election:Let there be Overlay Networks!Then what?
-
Our new goal?Create VONs by adding properties to OnsUser sees VON as a set of end-points with minimum guarantees, like isolation, between themWe need a way to strengthen these propertiesE.g. manage security keys, manage RSVP parameters, routing, network name spaceWe may also need ways to reliably communicate (1-1, 1-n patterns)
-
VONs as abstract data types
-
VONs as abstract data typesFocus on the processes and network
-
VONs as abstract data typesThink of the ON interface as an abstract typeONONON
-
VONs as abstract data typesAdd encryption by substituting a module that looks the same but encrypts messagesencryptencryptencrypt
-
Layered MicroprotocolsInterface to Horus is extremely flexibleHorus manages group abstractiongroup semantics (membership, actions,events) defined by stack of modulesencryptfiltersignftolHorus stacksplug-and-playmodules to givedesign flexibilityto developervsync
-
Layered Microprotocols in HorusInterface to Horus is extremely flexibleHorus manages group abstractiongroup semantics (membership, actions,events) defined by stack of modulesencryptfiltersignftolEnsemble stacksplug-and-playmodules to givedesign flexibilityto developervsync
-
Layered Microprotocols in HorusInterface to Horus is extremely flexibleHorus manages group abstractiongroup semantics (membership, actions,events) defined by stack of modulesencryptfiltersignEnsemble stacksplug-and-playmodules to givedesign flexibilityto developervsyncftol
-
Same stack under each endpoint
-
Multiple VONs in single applicationencryptvsyncftolencryptvsyncftolencryptvsyncftolYellow group for video communicationGreen forcontrol andcoordination
-
Examples of reliability modelsVirtual synchrony model: emerged from our work on Isis, now widely acceptedBimodal multicast model: probabilistic and has neat performance properties but weaker logical consistency guaranteesSecure group communicationMultimedia channels
-
Virtual Synchrony ModelcrashG0={p,q} G1={p,q,r,s} G2={q,r,s} G3={q,r,s,t}pqrstr, s request to joinr,s added; state xfert added, state xfert requests to joinp fails
-
Virtual Synchrony ToolsVarious forms of replication:Replicated data, replicate an object, state transfer for starting new replicas...1-many event streams (network news)Load-balanced and fault-tolerant request executionManagement of groups of nodes or machines in a network setting
-
Stock Exchange Problem: Vsync. multicast is too fragileMost members are healthy.
but one is slow
-
Measured Impact of PerturbationThroughput (msgs/sec)Amount Perturbed
_919232030.xls
Chart2
200.0400080016200
192.8640308582200
187.6876876877200
153.1862745098200
55.9565776957200
45.02476362200
24.6353961372200
15.2606519351200
8.5217346843200
Virtual Synchrony Protocol
Pbcast Protocol
amount perturbed
throughput (msgs/sec)
Effect of Perturbation
Sheet1
0.10.10.0049990.005200.04000800162000.1200.0400080016200
0.20.20.0051850.005192.86403085822000.2192.8640308582200
0.30.30.0053280.005187.68768768772000.3187.6876876877200
0.40.40.0065280.005153.18627450982000.4153.1862745098200
0.50.50.0178710.00555.95657769572000.555.9565776957200
0.60.60.022210.00545.024763622000.645.02476362200
0.70.70.0405920.00524.63539613722000.724.6353961372200
0.80.80.0655280.00515.26065193512000.815.2606519351200
0.90.90.1173470.0058.52173468432000.98.5217346843200
&A
Page &P
Sheet2
&A
Page &P
Sheet3
&A
Page &P
Sheet4
&A
Page &P
Sheet5
&A
Page &P
Sheet6
&A
Page &P
Sheet7
&A
Page &P
Sheet8
&A
Page &P
Sheet9
&A
Page &P
Sheet10
&A
Page &P
Sheet11
&A
Page &P
Sheet12
&A
Page &P
Sheet13
&A
Page &P
Sheet14
&A
Page &P
Sheet15
&A
Page &P
Sheet16
&A
Page &P
_919232303.xls
Chart2
200.0400080016200
192.8640308582200
187.6876876877200
153.1862745098200
55.9565776957200
45.02476362200
24.6353961372200
15.2606519351200
8.5217346843200
Virtual Synchrony Protocol
Pbcast Protocol
amount perturbed
throughput (msgs/sec)
Effect of Perturbation
Sheet1
0.10.10.0049990.005200.04000800162000.1200.0400080016200
0.20.20.0051850.005192.86403085822000.2192.8640308582200
0.30.30.0053280.005187.68768768772000.3187.6876876877200
0.40.40.0065280.005153.18627450982000.4153.1862745098200
0.50.50.0178710.00555.95657769572000.555.9565776957200
0.60.60.022210.00545.024763622000.645.02476362200
0.70.70.0405920.00524.63539613722000.724.6353961372200
0.80.80.0655280.00515.26065193512000.815.2606519351200
0.90.90.1173470.0058.52173468432000.98.5217346843200
&A
Page &P
Sheet2
&A
Page &P
Sheet3
&A
Page &P
Sheet4
&A
Page &P
Sheet5
&A
Page &P
Sheet6
&A
Page &P
Sheet7
&A
Page &P
Sheet8
&A
Page &P
Sheet9
&A
Page &P
Sheet10
&A
Page &P
Sheet11
&A
Page &P
Sheet12
&A
Page &P
Sheet13
&A
Page &P
Sheet14
&A
Page &P
Sheet15
&A
Page &P
Sheet16
&A
Page &P
_930556971.xls
Chart2
200.0400080016200
192.8640308582200
187.6876876877200
153.1862745098200
55.9565776957200
45.02476362200
24.6353961372200
15.2606519351200
8.5217346843200
Virtual Synchrony Protocol
Pbcast Protocol
amount perturbed
throughput (msgs/sec)
Effect of Perturbation
Sheet1
0.10.10.0049990.005200.04000800162000.1200.0400080016200
0.20.20.0051850.005192.86403085822000.2192.8640308582200
0.30.30.0053280.005187.68768768772000.3187.6876876877200
0.40.40.0065280.005153.18627450982000.4153.1862745098200
0.50.50.0178710.00555.95657769572000.555.9565776957200
0.60.60.022210.00545.024763622000.645.02476362200
0.70.70.0405920.00524.63539613722000.724.6353961372200
0.80.80.0655280.00515.26065193512000.815.2606519351200
0.90.90.1173470.0058.52173468432000.98.5217346843200
&A
Page &P
Sheet2
&A
Page &P
Sheet3
&A
Page &P
Sheet4
&A
Page &P
Sheet5
&A
Page &P
Sheet6
&A
Page &P
Sheet7
&A
Page &P
Sheet8
&A
Page &P
Sheet9
&A
Page &P
Sheet10
&A
Page &P
Sheet11
&A
Page &P
Sheet12
&A
Page &P
Sheet13
&A
Page &P
Sheet14
&A
Page &P
Sheet15
&A
Page &P
Sheet16
&A
Page &P
_919232070.xls
Chart2
200.0400080016200
192.8640308582200
187.6876876877200
153.1862745098200
55.9565776957200
45.02476362200
24.6353961372200
15.2606519351200
8.5217346843200
Virtual Synchrony Protocol
Pbcast Protocol
amount perturbed
throughput (msgs/sec)
Effect of Perturbation
Sheet1
0.10.10.0049990.005200.04000800162000.1200.0400080016200
0.20.20.0051850.005192.86403085822000.2192.8640308582200
0.30.30.0053280.005187.68768768772000.3187.6876876877200
0.40.40.0065280.005153.18627450982000.4153.1862745098200
0.50.50.0178710.00555.95657769572000.555.9565776957200
0.60.60.022210.00545.024763622000.645.02476362200
0.70.70.0405920.00524.63539613722000.724.6353961372200
0.80.80.0655280.00515.26065193512000.815.2606519351200
0.90.90.1173470.0058.52173468432000.98.5217346843200
&A
Page &P
Sheet2
&A
Page &P
Sheet3
&A
Page &P
Sheet4
&A
Page &P
Sheet5
&A
Page &P
Sheet6
&A
Page &P
Sheet7
&A
Page &P
Sheet8
&A
Page &P
Sheet9
&A
Page &P
Sheet10
&A
Page &P
Sheet11
&A
Page &P
Sheet12
&A
Page &P
Sheet13
&A
Page &P
Sheet14
&A
Page &P
Sheet15
&A
Page &P
Sheet16
&A
Page &P
_917772005.xls
Chart2
200.0400080016200
192.8640308582200
187.6876876877200
153.1862745098200
55.9565776957200
45.02476362200
24.6353961372200
15.2606519351200
8.5217346843200
Virtual Synchrony Protocol
Pbcast Protocol
amount perturbed
throughput (msgs/sec)
Effect of Perturbation
Sheet1
0.10.10.0049990.005200.04000800162000.1200.0400080016200
0.20.20.0051850.005192.86403085822000.2192.8640308582200
0.30.30.0053280.005187.68768768772000.3187.6876876877200
0.40.40.0065280.005153.18627450982000.4153.1862745098200
0.50.50.0178710.00555.95657769572000.555.9565776957200
0.60.60.022210.00545.024763622000.645.02476362200
0.70.70.0405920.00524.63539613722000.724.6353961372200
0.80.80.0655280.00515.26065193512000.815.2606519351200
0.90.90.1173470.0058.52173468432000.98.5217346843200
&A
Page &P
Sheet2
&A
Page &P
Sheet3
&A
Page &P
Sheet4
&A
Page &P
Sheet5
&A
Page &P
Sheet6
&A
Page &P
Sheet7
&A
Page &P
Sheet8
&A
Page &P
Sheet9
&A
Page &P
Sheet10
&A
Page &P
Sheet11
&A
Page &P
Sheet12
&A
Page &P
Sheet13
&A
Page &P
Sheet14
&A
Page &P
Sheet15
&A
Page &P
Sheet16
&A
Page &P
-
The problem gets worse as the system scales up00.10.20.30.40.50.60.70.80.9050100150200250Virtually synchronous Ensemble multicast protocolsperturb rateaverage throughput on nonperturbed membersgroup size: 32group size: 64group size: 96
-
Why does stability matter?Swiss Stock ExchangeExchange is fully electronic [FTCS-27 paper]Uses Isis SDK to distribute all bids/offers and all trades. Every node has the pictureBut this means that entire trading history available to 50 member banks & firms and hundreds or thousands of traders!Unstable node could bring exchange to its knees.Similar issues seen in many other settings
-
Pbcast has a probabilistic reliability modelEither almost all destinations receive the message or almost none do soThis is strong enough to use in applications with critical reliability needs (but not necessary for all their communication purposes -- put side by side with virtual synchrony)
-
Chart1
0
0.0291968046
0.000581897
0.0000102291
0.0000001783
0.0000000033
0.0000000001
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.0000000005
0.0000000364
0.0000035333
0.0006171974
0
number of processes to deliver pbcast
p{#processes=k}
Pbcast bimodal delivery distribution
Sheet1
00.00E+00
206.6298838.34570312.19E-036.43E-01101.78E-081.77E-0612.92E-02
255.9189458.653321.53.02E-031.02E+00152.18E-111.62E-0525.82E-04
305.440437.3408221.47E-039.52E-01203.42E-135.71E-0731.02E-05
355.0712897.5048832.53.92E-045.38E-01257.87E-161.66E-0641.78E-07
404.7636726.77343837.05E-052.07E-01308.89E-184.96E-0853.29E-09
454.531256.8759773.59.48E-066.13E-02352.60E-209.97E-0866.67E-11
504.3193366.33593841.01E-061.51E-02402.17E-223.10E-0971.53E-12
4.59.00E-083.26E-03458.26E-255.38E-0984.05E-14
56.85E-096.40E-04505.80E-271.76E-1091.26E-15
5.54.60E-101.16E-04553.67E-292.82E-10104.63E-17
62.88E-111.98E-05601.65E-311.56E-11112.04E-18
6.51.91E-123.20E-06121.08E-19
71.94E-135.07E-07136.95E-21
7.54.57E-148.72E-08145.45E-22
82.09E-142.03E-08155.23E-23
8.51.35E-147.74E-09166.16E-24
91.07E-144.37E-09178.93E-25
9.59.55E-153.08E-09181.60E-25
109.01E-152.50E-09193.53E-26
209.65E-27
213.27E-27
221.37E-27
237.13E-28
244.59E-28
253.67E-28
263.62E-28
274.43E-28
286.68E-28
291.24E-27
302.85E-27
318.00E-27
322.75E-26
331.16E-25
345.91E-25
353.67E-24
362.76E-23
372.51E-22
382.75E-21
393.62E-20
405.74E-19
411.10E-17
422.53E-16
437.06E-15
442.40E-13
451.01E-11
465.30E-10
473.64E-08
483.53E-06
496.17E-04
500.00E+00
Sheet1
Predicate I for 1E-9 reliability
Predicate II for 1E-12 reliability
#processes in system
fanout
Fanout required for a specified reliability
Sheet2
Predicate I
Predicate II
fanout
P{failure}
Effects of fanout on reliability
Sheet3
Predicate I
Predicate II
#processes in system
P{failure}
Scalability of Pbcast reliability
number of processes to deliver pbcast
p{#processes=k}
Pbcast bimodal delivery distribution
-
Pbcast has stable throughputGets this from a mixture of gossip-style local repair with several innovations to avoid overload when some process failsWe implemented the protocol and experimentally confirmed this
-
Chart4
00
0.97019867550.9735099338
0.02317880790.0198675497
00.0066225166
0.00331125830
0.00331125830
00
00
00
00
00
00
00
00
&A
Page &P
Pbcast with .05 sleep probability
Pbcast with .45 sleep probability
Inter-arrival spacing (ms)
Probability of occurence
Histogram of throughput for pbcast
Histograms
fifo/.05BinFrequencyfifo/.45BinFrequencyPbcast/.05BinFrequencyPbcast/.45BinFrequency
Traditional Protocol with .05 sleep probabilityTraditional Protocol with .45 sleep probabilityPbcast with .05 sleep probabilityPbcast with .45 sleep probability
0.001870.005680.0046280.00530.0059370.00500.0060.0050
0.0031150.012130.0047730.01310.0059480.012930.0060080.01294
0.003410.015170.0049740.015290.0060080.01570.0060210.0156
0.0037610.0200.0051870.02230.0060250.0200.0060470.022
0.004010.02500.005270.02580.0060290.02510.0060660.0250
0.0042520.0300.0054040.0350.006030.0310.0060660.030
0.0042830.03500.0055680.03530.0060370.03500.0060660.0350
0.0045610.0400.0057090.0410.0060370.0400.0060660.040
0.004570.04500.0058120.04520.0060470.04500.0060670.0450
0.0045740.0500.0059050.0510.006050.0500.0060680.050
0.0045790.05500.0059310.05500.0060520.05500.0060690.0550
0.0045980.0610.0059450.0600.0060530.0600.0060740.060
0.0046010.06500.0060270.06520.0060540.06500.0060760.0650
0.0046040.0700.0062470.0700.0060540.0700.0060770.070
0.004611More00.006373More00.006057More00.006079More0
0.0046160.0064670.0060570.006079
0.0046220.0066860.0060580.006083
0.0046370.006760.0060590.006083
0.0046420.0070870.006060.006084
0.0046420.0074380.0060620.006085Traditional Protocol with .05 sleep probabilityTraditional Protocol with .45 sleep probabilityPbcast with .05 sleep probabilityPbcast with .45 sleep probability
0.0046630.0079020.0060640.0060860.00568300
0.0046640.0080030.0060650.0060870.0121331293294
0.0046730.0080320.006070.0060890.015172976
0.0046740.0080760.0060710.0060910.0202302
0.0046740.0080930.0060710.0060920.0250810
0.0046790.0088210.0060740.0060970.030510
0.0046820.0091610.0060740.0060970.0350300
0.0046840.0094390.0060760.0061020.040100
0.0046840.0095980.0060760.0061020.0450200
0.004690.0096980.0060770.0061060.050100
0.0046950.0097020.0060770.0061070.0550000
0.0046990.0098620.0060780.0061070.061000
0.0047180.009930.0060790.0061110.0650200
0.0047240.0099570.0060790.0061110.070000
0.0047340.010050.0060830.006112
0.0047350.0101240.0060830.006113299108302302
0.0047390.0106250.0060840.006113
0.0047630.0106520.0060850.006116
0.0047650.0108220.0060850.006117
0.0047660.0112220.0060850.006117Traditional Protocol with .05 sleep probabilityTraditional Protocol with .45 sleep probabilityPbcast with .05 sleep probabilityPbcast with .45 sleep probability
0.0047690.0112510.0060860.0061180.00568300
0.0047710.0115180.0060870.0061180.0121331293294
0.0047750.0117170.0060870.006120.015172976
0.004780.01190.0060880.006120.0202302
0.0047810.0120050.0060880.0061210.0250810
0.0047970.01210.0060890.0061210.030510
0.0048040.0121780.0060890.0061220.0350300
0.0048210.0123390.0060890.0061240.040100
0.0048240.0124610.006090.0061240.0450200
0.0048270.0126670.006090.0061240.050100
0.0048290.0127050.0060910.0061240.0550000
0.0048380.0127760.0060910.0061250.061000
0.0048470.012880.0060920.0061260.0650200
0.004850.0128810.0060920.0061260.070000
0.0048620.0129590.0060920.006126
0.0048630.013270.0060930.006126299108302302
0.0049160.0135170.0060950.006127
0.0049190.0135390.0060950.006129Traditional Protocol with .05 sleep probabilityTraditional Protocol with .45 sleep probabilityPbcast with .05 sleep probabilityPbcast with .45 sleep probability
0.0049210.013810.0060970.0061310.0050.22742474920.027777777800
0.0049280.0142010.0060980.0061330.010.71237458190.2870370370.97019867550.9735099338
0.0049290.0143540.0060990.0061330.0150.05685618730.26851851850.02317880790.0198675497
0.0049310.0144220.00610.0061330.0200.21296296300.0066225166
0.0049380.0148380.00610.0061340.02500.07407407410.00331125830
0.0049420.0152750.0061010.0061360.0300.04629629630.00331125830
0.0049440.0153410.0061010.0061370.03500.027777777800
0.0049570.0156270.0061020.0061380.0400.009259259300
0.004960.0156410.0061030.006140.04500.018518518500
0.0049610.0157230.0061030.0061410.0500.009259259300
0.0050030.0157680.0061030.0061410.0550000
0.005010.0158320.0061040.0061420.060.0033444816000
0.0050160.0159540.0061040.0061430.06500.018518518500
0.0050320.0162180.0061050.0061430.070000
0.0050390.0162650.0061050.006144
0.0050460.0164180.0061050.006144
0.0050460.0168060.0061050.006144
0.0050590.0168380.0061060.006145
0.0050680.016980.0061060.006145
0.0050830.0174190.0061060.006145
0.0051180.0174540.0061070.006146
0.0051290.0176560.0061070.006146
0.0051310.0182410.0061070.006146
0.0051510.0186210.0061090.006147
0.0051710.0186810.0061090.006148
0.0051720.0189150.006110.006148
0.0051950.0191280.0061110.006149
0.0051960.0199140.0061110.006149
0.0051970.0206180.0061110.00615
0.0052340.0206230.0061110.00615
0.0052530.0209220.0061110.00615
0.005260.0209340.0061120.006151
0.0052820.0211980.0061130.006151
0.0052880.0213060.0061130.006152
0.0052940.0219240.0061140.006153
0.0053230.0222670.0061140.006153
0.0053360.0254720.0061140.006153
0.0053520.0264850.0061140.006154
0.0053610.0266770.0061150.006154
0.0054680.0284970.0061150.006154
0.0055610.0294520.0061150.006155
0.0055940.0316370.0061150.006155
0.0056340.0317270.0061150.006156
0.0056470.0320650.0061160.006157
0.0056550.0369850.0061170.006158
0.0056560.0417480.0061180.006158
0.0056690.0439830.0061180.006158
0.0056810.0450930.0061180.006159
0.0056930.0625090.0061180.006159
0.0057060.0628730.006120.00616
0.005720.006120.006162
0.0057250.006120.006163
0.0057270.0061210.006163
0.0057320.0061220.006164
0.0057410.0061220.006166
0.0057440.0061230.006166
0.0058090.0061230.006166
0.0058320.0061230.006166
0.005840.0061230.006166
0.0058430.0061240.006168
0.0058530.0061240.00617
0.0058550.0061250.00617
0.005870.0061250.00617
0.0058720.0061250.006171
0.0058840.0061250.006172
0.0058940.0061260.006172
0.0059070.0061270.006173
0.0059360.0061280.006173
0.005940.0061280.006173
0.0059580.0061290.006174
0.0059690.0061290.006175
0.0059860.0061290.006176
0.0060.0061290.006176
0.0060080.0061290.006177
0.0060150.0061290.006179
0.0060250.0061290.00618
0.0060340.006130.006181
0.0060440.006130.006181
0.0060510.0061310.006185
0.0060660.0061320.006186
0.0060820.0061320.006187
0.0060910.0061320.006187
0.0060960.0061340.006187
0.0061060.0061340.006188
0.0061140.0061360.006189
0.0061310.0061370.006189
0.0061490.0061380.00619
0.0061590.0061380.006191
0.0061710.0061380.006192
0.0061950.0061390.006193
0.0062010.006140.006193
0.0062160.006140.006193
0.0062190.0061410.006195
0.0062350.0061420.006196
0.0062420.0061420.006199
0.0062540.0061420.006201
0.0062630.0061420.006202
0.0062730.0061430.006202
0.0062790.0061430.006203
0.006280.0061430.006204
0.0062910.0061430.006204
0.0063210.0061450.006205
0.0063320.0061450.006205
0.0063390.0061460.006206
0.0063450.0061470.006206
0.0063470.0061470.006207
0.0063480.0061470.006208
0.006350.0061490.006208
0.0063550.006150.006208
0.0063560.0061510.006211
0.006370.0061510.006211
0.006380.0061520.006214
0.0063870.0061530.006216
0.0063880.0061530.006218
0.0063910.0061540.006221
0.0063960.0061540.006222
0.0064090.0061550.006223
0.0064180.0061550.006223
0.0064210.0061550.006224
0.0064220.0061550.006224
0.0064340.0061560.006224
0.0064450.0061560.006227
0.0064470.0061560.00623
0.0064470.0061570.006231
0.0064510.0061570.006234
0.0064690.0061570.006234
0.0064690.0061580.006235
0.0064740.0061590.006235
0.0064810.0061590.006236
0.0064810.0061610.006237
0.0064860.0061620.006238
0.0064940.0061620.00624
0.0064970.0061630.00624
0.0065090.0061640.006241
0.0065170.0061640.006242
0.006520.0061650.006247
0.0065280.0061650.006247
0.0065630.0061650.00625
0.0065770.0061670.00625
0.0065970.0061680.006252
0.006630.0061690.006252
0.0066380.0061690.006254
0.0066550.0061720.006254
0.0066640.0061720.006254
0.0066650.0061730.006255
0.0066650.0061740.006256
0.0066810.0061740.006258
0.006690.0061750.00626
0.0067040.0061760.006261
0.0067380.0061760.006262
0.006760.0061770.006263
0.0067620.0061770.006263
0.0067660.0061790.006264
0.0067850.006180.006266
0.0067960.006180.006268
0.0068020.0061810.006268
0.0068370.0061810.006268
0.0068770.0061820.006271
0.0068820.0061820.006272
0.0069320.0061820.006274
0.0069440.0061830.006274
0.0069620.0061840.006274
0.006970.0061840.006276
0.0069780.0061850.006276
0.0069840.0061860.006279
0.0070170.0061940.006281
0.0070320.0061950.006283
0.0070570.0061980.006283
0.0070910.0061980.006289
0.0070950.0061990.006289
0.0071230.0062030.006291
0.0071390.0062050.006292
0.0071480.0062060.006293
0.007150.0062070.006295
0.0071780.0062070.006299
0.0071780.0062080.006301
0.0071950.0062080.006302
0.0072520.0062090.006302
0.0072660.0062120.006303
0.0073430.0062120.006303
0.0073430.0062170.006308
0.0073470.0062180.006313
0.0073490.0062180.006318
0.0073570.0062210.006322
0.0074040.0062210.006323
0.0076160.0062250.006328
0.0076390.0062250.006335
0.0077020.0062250.006338
0.0077290.0062250.006342
0.0078370.0062280.006345
0.0079060.0062320.006348
0.0079070.0062330.006356
0.0079720.0062340.006367
0.0080220.0062340.006375
0.0080290.0062360.006381
0.0081680.0062490.006381
0.0081950.0062640.006384
0.0082360.006270.006386
0.0082380.0062770.006387
0.0082420.0062890.006389
0.0082580.0062980.006392
0.0082750.0063080.006435
0.0082990.0063270.00645
0.0083840.0063340.006461
0.0084050.0063370.006463
0.0084890.0063370.006477
0.0085030.0063420.006483
0.0085470.0063670.006513
0.0085970.0063740.006566
0.0085980.0063940.006603
0.0086010.0064260.006642
0.0087170.0064260.006692
0.008720.0064780.006711
0.008890.0064920.006847
0.0089120.006510.00686
0.0089650.0065450.00686
0.009020.0066230.006887
0.009260.006710.006957
0.0093070.0067880.007104
0.009590.0069060.007113
0.009640.0070740.007295
0.0099740.0070970.007297
0.0099940.0071120.007391
0.0100220.0072970.007416
0.010040.0073110.007757
0.0100790.0074690.00821
0.0101070.0074820.008302
0.010240.0075290.008333
0.010530.0079130.008349
0.0109420.007930.0085
0.0113920.007980.00876
0.0115780.0082320.008958
0.0116620.0086610.00903
0.0116820.0087670.009121
0.0118010.0092540.009175
0.0119930.0101640.009281
0.0137070.0104310.010412
0.0138190.0110130.011389
0.0140370.0127690.011573
0.0144880.0129210.012047
0.0566960.0135720.013404
0.0139560.0143
0.021560.018217
0.0280.01829
&A
Page &P
Histograms
00
00
00
00
00
00
00
00
00
00
00
00
00
00
&A
Page &P
Traditional Protocol with .05 sleep probability
Traditional Protocol with .45 sleep probability
Time to receive 100 messages
Probability of occurence
Histogram of throughput for Traditional Protocol
g10.003fifo
00
00
00
00
00
00
00
00
00
00
00
00
00
00
&A
Page &P
Pbcast with .05 sleep probability
Pbcast with .45 sleep probability
Time to receive 100 messages
Probability of occurence
Histogram of throughput for PBCast
1\1\g1\5\g2\5\g
Fifo/highPbcast/highFifo/lowPbcast/lowFIFO/hPbcast/hFIFO/lPbcast/lFIFO/hPbcast/hFIFO/lPbcast/l
Traditional w/1 sleeperPbcast w/1 sleeperTraditional w/1 sleeperPbcast w/1 sleeperTraditional w/5 sleepersPbcast w/5 sleepersTraditional w/5 sleepersPbcast w/5 sleepers
0.05151.262153.82599.999699.9981123.82150.75799.999999.999277.5588265.736161.412199.997
0.15126.037153.98699.999799.998174.0949153.16100.00299.997771.1499264.722136.632199.995
0.25101.96155.231100.00499.640765.026150.35796.038399.99967.6259262.719116.164199.997
0.3577.0642145.21999.999799.999750.6761150.62573.306199.995559.6272267.606106.311199.992
0.4563.9061153.02996.421199.994439.7611153.33158.5499.995752.2691260.94587.0828199.996
0.5550.3154152.36775.08999.998431.1254151.62743.980499.996431.1254151.62743.980499.9964
0.6539.4076153.88853.555599.99821.6599153.26533.497899.996921.6599153.26533.497899.9969
0.7526.2399149.92739.227299.999114.7746153.56323.037596.84914.7746153.56323.037596.849
0.8516.0008153.21722.373599.99799.07249152.90212.206499.99879.07249152.90212.206499.9987
0.955.67649153.628.4509999.99772.9353156.2562.935399.99922.9353156.2562.935399.9992
1/1/b
FIFO/hPbcast/hFIFO/lPbcast/l
1\3\gThroughput for traditional protocol, measured at faulty hostThroughput for Pbcast, measured at faulty host
Traditional w/3 sleepersPbcast w 3/sleepers151.261153.83199.9998100.003
154.991153.64399.9998153.25799.8381100.001
112.362151.774102.277149.679100.00499.2005
80.445151.8778.8386126.51799.999498.7813
64.5427149.9563.8418116.21895.878116.218
50.2844155.49250.027899.75174.765880.7126Throughput for traditional protocol, measured at correct host
41.3491151.82839.078177.962453.128863.896
25.3831153.22625.773953.385638.951343.8917Throughput for PBCast, measured at correct host
19.4144150.58415.333730.327821.954330.3278
9.07342153.4564.879068.210887.879528.21088
3.4324152.63
&A
Page &P
0000
0000
0000
0000
0000
0000
0000
0000
0000
0000
&A
Page &P
Throughput for traditional protocol, measured at correct host
Throughput for PBCast, measured at correct host
Throughput for traditional protocol, measured at faulty host
Throughput for Pbcast, measured at faulty host
Probability of Sleep Event
Average Throughput
High Bandwidth comparison of PBCast performance atfaulty and correct hosts
000000
000000
000000
000000
000000
000000
000000
000000
000000
000000
&A
Page &P
Traditional w/1 sleeper
Pbcast w/1 sleeper
Traditional w/3 sleepers
Pbcast w 3/sleepers
Traditional w/5 sleepers
Pbcast w/5 sleepers
Probability of sleep event
Throughput measured at unperturbed process
High Bandwidth measurements with varying numbers of sleepers
0000
0000
0000
0000
0000
0000
0000
0000
0000
0000
&A
Page &P
Traditional w/1 sleeper
Pbcast w/1 sleeper
Traditional w/5 sleepers
Pbcast w/5 sleepers
Probability of Sleep Event
Average Throughput
Low Bandwidth measurements with varying numbers of sleepers
-
Chart3
151.262153.825154.991153.643123.82150.757
126.037153.986112.362151.77474.0949153.16
101.96155.23180.445151.8765.026150.357
77.0642145.21964.5427149.9550.6761150.625
63.9061153.02950.2844155.49239.7611153.331
50.3154152.36741.3491151.82831.1254151.627
39.4076153.88825.3831153.22621.6599153.265
26.2399149.92719.4144150.58414.7746153.563
16.0008153.2179.07342153.4569.07249152.902
5.67649153.623.4324152.632.9353156.256
Traditional w/1 sleeper
Pbcast w/1 sleeper
Traditional w/3 sleepers
Pbcast w 3/sleepers
Traditional w/5 sleepers
Pbcast w/5 sleepers
Probability of sleep event
Throughput measured at unperturbed process
High Bandwidth measurements with varying numbers of sleepers
Sheet1
0.10.0049990.005200.040008200
0.20.0051850.005192.8640309200
0.30.0053280.005187.6876877200
0.40.0065280.005153.1862745200
0.50.0178710.00555.9565777200
0.60.022210.00545.02476362200
0.70.0405920.00524.63539614200
0.80.0655280.00515.26065194200
0.90.1173470.0058.521734684200
Sheet1
99.999699.998199.999999.9992
99.999799.9981100.00299.9977
100.00499.640796.038399.999
99.999799.999773.306199.9955
96.421199.994458.5499.9957
75.08999.998443.980499.9964
53.555599.99833.497899.9969
39.227299.999123.037596.849
22.373599.997912.206499.9987
8.4509999.99772.935399.9992
Traditional w/1 sleeper
Pbcast w/1 sleeper
Traditional w/5 sleepers
Pbcast w/5 sleepers
Probability of Sleep Event
Average Throughput
Low Bandwidth measurements with varying numbers of sleepers
Sheet2
99.999699.998199.9998100.003
99.999799.998199.8381100.001
100.00499.6407100.00499.2005
99.999799.999799.999498.7813
96.421199.994495.878116.218
75.08999.998474.765880.7126
53.555599.99853.128863.896
39.227299.999138.951343.8917
22.373599.99792130.3278
8.4509999.99777.879528.21088
Traditional w/1 sleeper
Pbcast w/1 sleeper
Throughput for traditional protocol, measured at perturbed host
Throughput for Pbcast, measured at perturbed host
Probability of Sleep Event
Average Throughput
Low Bandwidth comparison of PBCast performance atfaulty and correct hosts
Sheet3
99.999699.998199.999999.9992
99.999799.9981100.00299.9977
100.00499.640796.038399.999
99.999799.999773.306199.9955
96.421199.994458.5499.9957
75.08999.998443.980499.9964
53.555599.99833.497899.9969
39.227299.999123.037596.849
22.373599.997912.206499.9987
8.4509999.99772.935399.9992
Traditional w/1 sleeper
Pbcast w/1 sleeper
Traditional w/5 sleepers
Pbcast w/5 sleepers
Probability of Sleep Event
Average Throughput
Low Bandwidth measurements with varying numbers of sleepers
151.262153.825154.991153.643123.82150.757
126.037153.986112.362151.77474.0949153.16
101.96155.23180.445151.8765.026150.357
77.0642145.21964.5427149.9550.6761150.625
63.9061153.02950.2844155.49239.7611153.331
50.3154152.36741.3491151.82831.1254151.627
39.4076153.88825.3831153.22621.6599153.265
26.2399149.92719.4144150.58414.7746153.563
16.0008153.2179.07342153.4569.07249152.902
5.67649153.623.4324152.632.9353156.256
Traditional w/1 sleeper
Pbcast w/1 sleeper
Traditional w/3 sleepers
Pbcast w 3/sleepers
Traditional w/5 sleepers
Pbcast w/5 sleepers
Probability of sleep event
Throughput measured at unperturbed process
High Bandwidth measurements with varying numbers of sleepers
151.262153.825151.261153.831
126.037153.98699.9998153.257
101.96155.231102.277149.679
77.0642145.21978.8386126.517
63.9061153.02963.8418116.218
50.3154152.36750.027899.751
39.4076153.88839.078177.9624
26.2399149.92725.773953.3856
16.0008153.21715.333730.3278
5.67649153.624.879068.21088
Throughput for traditional protocol, measured at unperturbed host
Throughput for PBCast, measured at unperturbed host
Throughput for traditional protocol, measured at perturbed host
Throughput for Pbcast, measured at perturbed host
Probability of Sleep Event
Average Throughput
High Bandwidth comparison of PBCast performance atfaulty and correct hosts
00
00
00
00
00
00
00
00
00
Virtual Synchrony Protocol
Ideal Behavior
amount perturbed
throughput (msgs/sec)
Effect of Perturbation
-
Now we have several styles...Each style or model yields a VON with different propertiesApplication might not see the multicast stackInstead, the environment in which the application runs could see the stack and use it on behalf of the applicationFor example, a library could use stack to maintain the keys with which it authenticates actions
-
Formal methodsWith so much riding on VON, we need strong guarantees that the stack works!If protocols can be formally proved correct, confidence will be far strongerCan we use formal tools on network protocols built in this compositional manner?
-
Exploiting formal methodsVan Renesse and Hayden: code stack with language having strong semanticsThey used OCaml dialect of MLNow we can bring formal tools to bear on issues of correctness:Using Nuprl system for thisBasically, it automates proofs and program transformations
-
Initial Progress?Presented in 1999 ACM SOSP paperHave formalized the transformations used to optimize stacks for high performanceWe show that from one initial stack, we can produce multiple optimized stacks for common cases. Yields big speedups!
-
StepsTransform Ensemble stack into a single function in a functional styleUse partial evaluation to produce optimized version for common cases Use theorem proving to establish that stacks provide desired propertiesTransform back to imperative styleResulting code is optimized yet retains properties of original stack
-
Optimization ExampleencryptvsyncftolOriginal code is simple but inefficientOptimized code for common case is provably equivalent yet inefficiencies are eliminatedencryptvsyncftol? Common case?
-
Optimization Exampleencryptvsyncftolencryptvsyncftol? Common case?? Common case?We do nearly as well as hand-optimization and can automatically handle much bigger stacks!
-
Wrapping things upBy building better networks, and isolating protocol components and system components and adopting a modular architecture and selectively using formal methods we make it more and more practical to gain both high performance and other desired properties, such as reliability, security, stability, etc.
-
Potential NGI lets critical applications share network with untrusted onesVONs
-
But will it happen?Current political agenda focuses on speed and e-commerce transactionsEnd-to-end community resists giving any guarantees no matter how simpleAnd NGI focus is exclusively on point-to-point QoS, which seems unscalabledenying us the one primitive building block on which the whole concept depends!
-
Conclusions?The world needs better networks!Improve them by improved opportunity for modularity, isolation, guarantees of security and quality of service VONs and layers built over themLacking this, we face very serious problems simply going forward in directions to which society is already committed.
-
More info
http://www.cs.cornell.edu/ken/unsafe.ps