SIXPACK: Securing Internet eXchange Points AgainstCurious onlooKers

Marco ChiesaUniversité catholique de Louvain

Daniel DemmlerTechnische Universität Darmstadt

Marco CaniniUniversité catholique de Louvain

Michael SchapiraHebrew University of Jerusalem

Thomas SchneiderTechnische Universität Darmstadt

ABSTRACTInternet eXchange Points (IXPs) play an ever-growing role inInternet inter-connection. To facilitate the exchange of routesamongst their members, IXPs provide Route Server (RS) ser-vices to dispatch the routes according to each member’s peer-ing policies. Nowadays, to make use of RSes, these policiesmust be disclosed to the IXP. This poses fundamental ques-tions regarding the privacy guarantees of route-computationon confidential business information. Indeed, as evidenced byinteraction with IXP administrators and a survey of networkoperators, this state of affairs raises privacy concerns amongnetwork administrators and even deters some networks fromsubscribing to RS services. We design sixpack, an RS servicethat leverages Secure Multi-Party Computation (SMPC) tokeep peering policies confidential, while extending, the func-tionalities of today’s RSes. As SMPC is notoriously heavyin terms of communication and computation, our designand implementation of sixpack aims at moving computa-tion outside of the SMPC without compromising the privacyguarantees. We assess the effectiveness and scalability ofour system by evaluating a prototype implementation usingtraces of data from one of the largest IXPs in the world. Ourevaluation results indicate that sixpack can scale to supportprivacy-preserving route-computation, even at IXPs withmany hundreds of member networks.

1 INTRODUCTIONWith the rise of Internet eXchange Points (IXPs) as the emerg-ing physical convergence points for Internet traffic, new pri-vacy concerns arise. IXPs offer centralized Route Server (RS)services for ranking, selecting, and dispatching BGP routes totheir (potentially many hundreds of) member networks [74].However, to benefit from these centralized services, IXPmembers need to divulge private information, such as peer-ing relationships and route-export policies to the IXP or,even worse, to other IXP members. Such information can re-flect sensitive commercial and operational information, andis consequently often regarded as private [42, 88]. Indeed,our interaction with IXP administrators, and our survey ofnetwork operators (§2), reveal that such privacy concernsare widespread and that some networks even refrain from

subscribing to RS services for precisely this reason. Beyondprivacy, our survey reveals three additional IXP members’concerns for RS usage: limited routing policy expressiveness,reliability, and insufficient value. This situation hinders RSadoption and makes it hard to provide novel valuable rout-ing services to IXP members, as these can rely on the expo-sure of (even more) sensitive data. Indeed, on the one hand,advanced performance-oriented routing services are funda-mental to improve performance of video and latency-criticalInternet applications [17, 21, 22, 50, 56, 72, 73, 82, 84, 86]. Inthis regard, today’s largest content providers (e.g., Googleand Twitch) resort to active measurement techniques forinferring route performance [38], a difficult task in prac-tice [22]. On the other hand, our survey reveals that a largemajority of network operators (60%) is concerned about shar-ing network performance information such as IXP port uti-lization with external entities. In this regard, the goal ofsupporting advanced Internet routing features while pro-tecting sensitive information is the subject of several recentstudies [8, 42, 43, 54, 55, 60, 67, 89].How should we design RSes? To increase trust in IXPsand motivate further adoption of RS services, we argue thatRSes should meet the following basic requirements: a) Easymanagement, i.e., relieve IXP members from the burden ofconfiguring numerous BGP peering sessions, b) Policy expres-siveness, i.e., provide IXP members with highly-expressiveroute selection at least equivalent to having multiple bilat-eral BGP sessions [34], c) Performance-driven routing, i.e.,dispatching tools that leverage the IXP’s superior visibilityinto data-plane network conditions, d) Efficiency, i.e., today’sRSes are required to compute and dispatch routes in theorder of hundreds per second, with full routing-table trans-fers performed in the order or minutes [2, 25, 77], e) Privacypreservation, i.e., no IXP member nor the IXP itself should beable to learn information about routing policies of the othermembers (except for information that can be deduced fromits own RS-assigned routes), f) Reliability, i.e., guaranteedconnectivity upon failures in the IXP infrastructure.SIXPACK: A privacy-preserving advanced RS. To ac-complish the above, we advocate implementing an RS viasecure multi-party computation (SMPC) [36, 87]. With an

1

CoNEXT’17, December, 2017, Seoul, South Korea M. Chiesa, D. Demmler et al.

SMPC-based realization of an RS, the desired routing out-come can be computed without the IXP or IXP membersgaining visibility into the “inputs” to this computation, i.e.,members’ private routing policies.

However, realizing our vision of a privacy-preserving RSis highly nontrivial. General-purpose SMPC machinery isexcessively heavy in terms of computation and communi-cation overheads and is thus infeasible to employ for thispurpose [42]. Consequently, attaining feasible runtimes andcommunication costs requires devising a suitable highly-optimized, privacy-preserving scheme specifically tailored tothe RS context. We present sixpack, the first IXP route serverservice that satisfies all the aforementioned requirements.1It efficiently ranks, selects, and dispatches BGP routes basedon the members’ expressive routing policies and any IXP-provided performance information without leaking any con-fidential business peering information.Conceptually, in sixpack, the IXP route server service is

jointly performed by two independent and non-colludingcomputational parties, RS1 and RS2, which run an SMPC pro-tocol. Each IXP member encrypts its BGP routes, announcesthem to the RSes, and creates two “shares” of its (private)business peering policy that are sent to the two RSes. Eachof the RSes, in turn, sends to each IXP member, upon com-pletion of the SMPC, a share of its output, that the membercan use to recover its selected routes. sixpack provably guar-antees that as long as RS1 and RS2 do not collude with eachother, neither the RSes nor other IXP members will learnany information regarding an IXP member’s business androuting policies. We envision RS1 and RS2, as being run bythe IXP and a neutral and well-regarded international orga-nization (e.g., NANOG or RIPE), which is already trusted tosupport and operate fundamental Internet services (thoughnot necessarily trusted for privacy confidentiality)2. In addi-tion, one of the two RSes should be executed on a machinethat is located outside the IXP but in very close proximity(i.e., within the same colocation data center) so as to be in aseparate security domain while keeping latency of inter-RScommunication at a minimum (i.e., < 1ms).We observe (§5) that a naïve application of SMPC to RS

computation, in which the entire RS computation is carriedout via SMPC machinery, is infeasible in practice as a resultof unrealistic computation and communication overheads.We thus resort to the following approach. sixpack is care-fully designed to keep complex computation outside theSMPC, to the largest extent possible without compromising

1While the focus in this work is not reliability, even today, at large IXPs,members are contractually requested to set multiple peering sessions withdistinct RSes for redundancy requirements [26]. Also, recently proposed In-ternet drafts [48] further mitigate the impact of failures in the IXP network.2In addition, our survey of network operators [64] reveals RIRs enjoy thetrust of a large fraction (88%) of respondents.

privacy. Specifically, in sixpack, we carefully decompose theRS functionality into two simple, yet crucial, SMPC-basedbuilding blocks for efficient route-dispatch, called Export-All and Select-Best, while performing the most complexcomputation over unencrypted data outside of the SMPCwithout leaking any private information. Using Export-All,all available BGP routes that are exportable to an IXP mem-ber, that is, the routes that other IXP members are willingto advertise to that member, are dispatched to the memberin a fully privacy-preserving manner. Then, the memberranks its available routes according to its local preferencesover routes and feeds the resulting ranking as input intoSelect-Best. Select-Best leverages this information andinformation from the IXP (e.g., port utilization) to select thebest route for thatmemberwithout leaking any information.3Thus, sixpack both goes well beyond the services offered bytoday’s RSes (by delegating route-selection from the RS tothe member and incorporating performance-related infor-mation into the route selection process) and provides strongprivacy guarantees to IXP members.We discuss sixpack’s optimized design, underlying as-

sumptions, threat model, deployment challenges, IXP visibil-ity of data-plane traffic, etc., in detail in the sequel.Paper contributions.• An analysis of the operators’ concerns about peering withRSes at IXPs through a survey with 119 responses and a mea-surement of RS usage at one of the largest IXP worldwide.• The design and implementation of the first IXP route servecapable of keeping the peering policies and routing prefer-ences private, while allowing the IXP members to expressarbitrary BGP routing policies including policies that incorpo-rate confidential performance-related information availableat the IXP (e.g., port utilization).4• Applying an SMPC approach to the important and timelycontext of route dispatch at IXPs and devising efficient andprivacy-preserving SMPC building blocks for RSes.•An evaluation of our prototype. Through experiments witha BGP trace from one of the largest IXPs in the world, ourresults show that sixpack scales to hundreds of IXP mem-bers and achieves BGP processing times below 90ms at the99th-ile. Via microbenchmarks, we assess the online costsof SMPC-based RSes to be well within real-time processingrequirements of large IXPs.

2 ON IXPS AND PRIVACYWe present below preliminaries on Internet exchange pointsand quantify, via measurements, the extent to which routeserver services are used. We also report on our interactionwith IXPs and member network administrators, including3Thanks to our modular design, a member may skip the Select-Best phaseat the cost of revealing to the IXP that it disregards using IXP information.4We plan to release our sixpack prototype as open source.

2

SIXPACK: Securing Internet eXchange Points Against Curious onlooKers CoNEXT’17, December, 2017, Seoul, South Korea

a survey of network operators, indicating that privacy con-cerns are a significant factor hindering widespread RS usageand corroborating assumptions underlying past research re-garding the privacy of routing policies.Background on IXPs. IXPs are high-bandwidth physicalnetworks located within a single metropolitan area. IXPsare typically geographically distributed [5, 23] and hostedwithin colocation centers [49], facilities operated by thirdparty providers that offer high levels of physical security.Heterogeneous economic entities use IXPs to exchange

Internet traffic with each other [1]. To do so, each memberconnects its own network to one or more physical ports atthe IXP network. After physical connectivity is established,each member announces the set of IP prefix destinations forwhich it is willing to receive traffic and starts receiving routeannouncements from the other members of the IXP.

The routes used to reach prefixes are spread and selectedvia the de facto standard inter-domain routing protocol ofthe Internet, i.e., Border Gateway Protocol (BGP). To this end,a full-mesh of BGP sessions among each pair of IXP membersmay be established. At medium to large IXPs, which can haveover 800 members and carry over 5Tbps, such full-meshescan be partially replaced by a Route Server (RS) service to easethe exchange of BGP announcements among members [74].The RS establishes a BGP session with each IXP member, col-lects and distributes their BGP route-announcements. Notethat data-plane traffic does not traverse the RS, which is onlyinvolved in control-plane traffic.Each IXP member has the freedom to specify, for each

destination IP prefix, an export policy, i.e., the set of other IXPmembers that are allowed to receive its route announcements.The RS selects, for each member, a route (per IP prefix) that is“exportable” to that member (according to members’ exportpolicies), and dispatches it to that member.

Today’s IXPs do not allow their members to influence theRS’s route selection with the members’ import policies. Thesepolicies comprise of the traditional BGP local preferences [18]and regular expressions that the RS would use to rank andfilter available routes [34]. For instance, an operator may beinterested in routing its traffic through a certain IXP memberunless the announced route traverses a specific network.Howwidespread isRSusage?Despite the fact that such anRS service eases the management of BGP sessions, facilitatespeering, and lowers hardware requirements on connectedBGP routers, there is anecdotal evidence of its limited usage.To corroborate this, we performed an analysis of RS usage atone of the largest IXPs worldwide. We have been reportedthat similar values hold for at least another of the largestIXPs worldwide. We examined both data traffic and BGPcontrol plane messages so as to quantify the fraction of trafficthat is routed along the routes dispatched by the RS. Fig. 1presents a CDF graph showing the fraction of IXP members

0 0.2 0.4 0.6 0.8 10

0.20.40.60.81

Share of public traffic

Shareof

IXPmem

bers

Figure 1: CDF of RSusage at a large IXP.

rA rBmA 0 0

mB 0 0

mC 1 1

mD 0 1

rA rBmA 0 1

mB 1 0

mC 0 0

mD 1 1

rA rBmA 0 1

mB 1 0

mC 1 1

mD 1 0

(a) (b) (c)

= ⨁

Figure 2: (a) Export pol-icy matrix in plain text.(b) Random shares re-ceived by RS1. (c) Element-wise XOR of (a) and (b)received by RS2.

that have less than a certain fraction of “public traffic”, i.e.,traffic routed along the RS-computed routes. As shown inthe leftmost part of the figure, 40% of members do not routetheir traffic via RS-prescribed routes. About two-thirds ofthe remaining members route less than half of their trafficaccording to the RS and only 20% of members route over50% of their traffic along RS-computed routes. In terms ofabsolute amount of traffic, we discovered that less than 17%of the overall traffic is routed via RS-prescribed routes. WhileAger et al. [1] observed that most of the networks peer withthe RS, we showed that members prefer to route the vastmajority of their traffic through based on the informationexchanged through bilateral BGP sessions.Are privacy concerns hinderingwider RS usage and in-novation? One of the main barriers facing the transitionfrom a full-mesh of BGP peering sessions to a star topol-ogy (via an RS) is that the export policy of each member(and, to support route-selection at the RS, potentially alsothe import policy of each member) must be revealed to theIXP. This information is considered confidential, primarilydue to commercial reasons. Indeed, our interaction with IXPadministrators and network operators reveals such privacyconcerns and, moreover, that some networks do not connectto RSes for precisely this reason. In particular, we circulateda survey among the network operator community [64] withthe aim of exploring their perceptions about privacy at IXPs.We collected 120 responses belonging to a broad range ofdifferent networks: Tier 1 ISPs (8%), Tier 2/3 ISPs (57%) CDNs(6%), content providers (12%), and others (17%), with almostall networks connecting to an IXP and 80% of them using RSservices (at least for that fraction of traffic not belonging toan established bilateral peering). According to our survey,the most critical concerns regarding RSes among respon-dents were: no control over best route selection, i.e., lackof import policy configuration tools (53%), reliability (40%),lack of route visibility (37%), privacy (19%), and legal restric-tions (7%). Since the 1st and 3rd items requires members todisclose their policies to the RS, we further investigated thisprivacy aspect: (i) 40-45% of respondents consider their localpreferences over BGP routes to be private, both with respectto the IXP and with respect to other members, (ii) 60% of

3

CoNEXT’17, December, 2017, Seoul, South Korea M. Chiesa, D. Demmler et al.

respondents expressed concerns about sharing their IXP portutilization with other members, and (iii) 1 of every 4 respon-dents that do not use an RS service marked concerns aboutdisclosing export policies to the IXP as a reason. With com-ments ranging from “Nothing should be considered private”to ’“Everything listed is supposed to be private/proprietaryinformation”, our survey revealed the heterogeneous require-ments of Internet domains. Despite the existence of manynetworks with open peering policies, our survey reveals thatlocal-preferences over routes and port utilizations are stillconsidered as a private information not to be divulged.

We point out that beyond privacy concerns, revealing sen-sitive information also entails the risk of triggering attacks onthe weaker parts of the network [66], e.g., easier bandwidth-exhaustion attacks if port utilization is revealed [85].

3 THREAT MODEL AND ASSUMPTIONSTo provide strong privacy guarantees, sixpack relies on threeassumptions (A), which we list and justify next.Assumption 1: Two non-colluding RSes.We assume thatthe RS service consists of two distinct route servers: one is op-erated by the IXP and one by an independent, non-colludingentity. The latter RS is executed on a machine that is outsideof the IXP domain but connects to the IXP network at the co-location center where the IXP is hosted. Since this instancelives in a separate security domain, this solution minimizesthe possibility of an RS instance being compromised by theIXP, while keeping latency at a minimum [4].We believe that neutral international organizations (e.g.,

RIPE), which are already trusted to support and operate fun-damental Internet services such as DNS and IP allocation,should be assigned the task of running an instance of an RSor, alternatively, supervising those that do. We argue thatrunning an RS instance is a simpler task than operating a dis-tributed DNS system. Our survey of network operators [64]reveals that RIRs enjoy the trust of an overwhelming fraction(88%) of respondents. We point out that, even at today’s largeIXPs, members are requested to set up multiple peering ses-sions with distinct RSes for redundancy requirements [26].In SMPC, redundancy is required for both RS instances.Assumption 2: Honest-but-curious RSes. sixpack pro-tects against so-called “honest-but-curious attackers”, i.e.,RSes that stick to the protocol but try to infer the members’private inputs. We argue that as today some networks re-frain from peering with others via route servers because offear of revealing private information to the RSes, this modelcaptures an important desideratum in the IXP ecosystem.Furthermore, if cheating (e.g., deviation from the protocolby an RS) were detected, this would result in massive lossof trust in the service and, consequently, severe economicconsequences for the IXP. We point out, however, that ourSMPC circuits could be evaluated with a framework secure

against malicious adversaries (e.g., [71]) if desired, thoughat higher computation and communication overheads.Assumption 3: No visibility into data traffic. sixpack isdesigned to hide control-plane information from the RSes.We view data-plane privacy-preservation, i.e., preventingthe IXP from inferring routing policies from observing datatraffic traversing the IXP network, as an orthogonal problemthat requires further exploration. We refer the reader to §8for an explanation of why inferring routing policies fromthe data plane is highly challenging even when informationabout BGP routes is available. We point out, however, thatsixpack actually does make the inference of routing policiesfrom data traffic more challenging for the IXP. Guaranteeingdata-plane-level privacy can also involve other approachessuch as encrypting and decrypting IP headers at IXP ingressand egress. We leave this interesting topic for future research.

4 SECURE MULTIPARTY COMPUTATIONOverviewof SMPC. Securemulti-party computation (SMPC)allows multiple parties, to jointly compute the outcome of afunction f while keeping their inputs to it and the outputsof f private. SMPC was established as a purely theoreticalconstruct more than 30 years ago and was initially seen astoo inefficient to be used in practice due to large communi-cation and computational overhead. However, a recent lineof research has improved SMPC primitives drastically andshowed that practical implementations of SMPC are possible.See, e.g., [14, 44, 45, 59, 62, 69].

One application of SMPC is outsourcing the computationof a function f from the parties holding the private inputs,called “members” henceforth, to several external computa-tional parties. Members’ inputs are distributed to the com-putational parties while remaining secret, and the computa-tional parties run a computation on the distributed inputs,without obtaining visibility into the actual inputs or outputs.

In this work, we employ the well-established SMPC proto-col of Goldreich-Micali-Wigderson (GMW) [36] that operateson a Boolean circuit, consisting of logic gates such as ANDand XOR, which represents the function f to be computed.We rely on the proven security of the GMW protocol [35, 36].

While the GMW protocol, in general, can also be executedby more than two parties, we consider the scenario whereprivate inputs from the members are outsourced to exactlytwo computational parties SMPC1 and SMPC2. This decisionis made mainly for performance reasons. The outsourcingof private member inputs is achieved via XOR-based secretsharing as follows: Each member selects, for every plaintextinput bit b, a random bit b1, and computes bit b2 = b⊕b1. Thebits b1 and b2, called shares of b, are distributed between theSMPC parties such that SMPCi obtains share bi . Importantly,from the perspective of the two computational parties, theshares are indistinguishable from random bits.

4

SIXPACK: Securing Internet eXchange Points Against Curious onlooKers CoNEXT’17, December, 2017, Seoul, South Korea

After the evaluation of all gates in a circuit, the outputshares computed by the two parties are sent to the mem-bers, who are then able to recover their plaintext output bycomputing the XOR of the two output shares.Optimization of GMW. We rely on the GMW protocol,whichwas shown to be beneficial in several practical cases [78],for two main reasons: (1) GMW precomputes all crypto-graphic operations in an input- and even function-independentsetup phase, which can be parallelized and computed at anytime before the private inputs are known; and (2) GMW alsooffers performance benefits in certain scenarios, as it allowsto process multi-input AND and MUX gates very efficientlyand can work on an identical sub-circuit in parallel, cf. [78].We make heavy use of these features of GMW to achievevery low runtimes. We designed our circuits to have a lowdepth, thus achieving good runtimes for evaluation withGMW. Our implementation uses the ABY framework [27]which implements GMW for two parties using state-of-the-art optimizations, such as SIMD (single instruction multipledata) processing and very efficient vector gates. Finally, wealso rely on the proven security of a symmetric cipher forencrypting/decrypting routes, which we instantiate with 128-bit AES in CTR mode. We verified that AES adds negligibleoverhead compared to the SMPC, which holds in generaland especially on machines with the AES-NI instruction set.We emphasize that our circuits could be evaluated also

with other SMPC protocols (e.g., Yao’s garbled circuits [87])or protocols that provide security against stronger adver-saries (e.g., [71]) and/or use more than two computationalparties (of which a fraction can be corrupted). However, theseprotocols have significantly higher communication and/orcomputation complexities.

5 SIXPACK PRIVACY-PRESERVING RSWe introduce sixpack, a privacy-preserving RS service forIXPs. sixpack combines the benefits of a centralized route dis-patch service with the provable guarantee of privacy preser-vation. Through sixpack, IXP members can receive the bestavailable BGP routes according to arbitrary local route pref-erences and auxiliary information of the IXP (e.g., knowledgeof congestion level and other performance metrics [56]).Building on recent advances in SMPC, sixpack employs

two independent and non-colluding computational entitiesto correctly dispatch route announcements without gainingany visibility into the members’ routing policies (i.e., export/import policies) nor leaking any private IXP performance-related information to the members.To illustrate the non-trivial challenges facing sixpack’s

design, we first discuss a fairly naïve approach to applyingSMPC to route-dispatch at IXPs, and why this approachfails. We then describe the key design ideas behind sixpack,

EXPORT-ALL SELECT-BEST

LOCAL-RANKING

members’selectedroutes

BGP routes &export policies

RS

Members

IXP information

exportable routes

Route prefs.

next-hop ranking

Figure 3: The sixpack system architecture.present the routing policy model, and discuss in detail thetwo main components of the system.

5.1 A Naïve ApproachAs general-purpose SMPC is capable of arbitrary compu-tation, it would be tempting to implement sixpack solelywithin a single SMPC, thus providing arbitrary privacy-pre-serving policy expressiveness. This task entails devising afunction that takes as input the set of members’ export poli-cies, the set of arbitrarily complex members’ import policies(e.g., regular expressions on the AS networks traversed by aroute), and the IXP’s performance-related information (e.g.,port utilization), and outputs the resulting “best” BGP routes.

However, even performing a single full regular expressionstring matching operation in SMPC using state-of-the-artimplementations is overly prohibitive in practice [53, §5]as this operation is shown to require runtimes in the orderof minutes. Even by restricting the members to use a singleregular expression in their import policy (a fairly restrictiveassumption), evaluating the import policies in a large IXPwith 500 members would take days! In contrast, sixpackcomputes and dispatches routes in the order of tens of mil-liseconds, improving upon the naïve approach by 5 ordersof magnitude. This is made possible through a combinationof several ingredients, as discussed below.

5.2 SIXPACK DesignTo achieve practical runtimes, sixpack is carefully designedto keep complex computation outside the SMPC, to thelargest extent possible without compromising privacy (seeFig. 3). Specifically, the route dispatch computation is splitinto three operations to be performed sequentially, calledExport-All, Local-Ranking, and Select-Best. Both Ex-port-All and Select-Best are SMPC-based componentsof the system that are executed within the RS by the twonon-colluding entities (depicted as a single green-coloredbox). The Local-Ranking component, in contrast, is locallyexecuted by each member (depicted as a single blue-coloredbox). Next, we describe the sixpack pipeline for processingBGP route announcements. For readability, we assume mem-bers connect with a single BGP routers (see [65] for the casewith multiple routers). We observe that BGP withdrawalmessages can be handled in a similar way (see [65]).Step I: Exporting all permissible routes. sixpack pro-cesses streams of BGP announcements generated by the IXP

5

CoNEXT’17, December, 2017, Seoul, South Korea M. Chiesa, D. Demmler et al.

members. Through Export-All, sixpack takes as input aBGP route destined to a prefix π and its associated exportpolicy and outputs the route to the IXP members authorizedto see that route. This operation is performed in SMPC, andso neither the route nor the export policy is disclosed toany unintended entity. The route and its export policy areboth stored in encrypted form within the RS. We discussExport-All in detail later in this section (§5.4).Step II: Ranking routes based on local preferences. Atthis point, each member that received the new route executesLocal-Ranking to rank all its available routes towards πaccording to its (arbitrarily complex) local import policies.A key idea embedded into sixpack’s design is performingthis computationally-heavy operation outside of the SMPC,i.e., at the member-side. Now, recall that in BGP, each IXPmember only announces at most one single route towards π .Thus, a ranking of the routes destined to π corresponds to aranking of the IXP members, where a member assigns thelowest preference to those IXP members from whom it didnot receive a route to π . We call such ranking the next-hopranking, and this is the output of Step II.Step III: Incorporating IXP information into route se-lection. When multiple routes for a certain prefix are avail-able, sixpack submits the next-hop ranking to the RS service.Upon receiving a next-hop ranking from a member, the RSproceeds to run the SMPC-based Select-Best component fordispatching the best-selected route to that member. The bestroute is computed based on the next-hop ranking and, impor-tantly, the performance-related information available to theIXP (e.g., port utilization). To improve efficiency, we observethat, in practice, it is convenient to batch several next-hopranking messages together before executing Select-Best asshown in out evaluation (§7.2). Also, observe that, since BGPtreats each IP prefix destination independently, multiple in-stances of the two RSes can be easily instantiated to dispatchroutes in parallel, thus enhancing the system throughput.However, multiple route announcements towards the sameIP prefix have to be processed sequentially. We discuss theSelect-Best in detail later in this section (§5.5).Benefits of our design approach. Through Export-All,an IXPmember gains full visibility of the available routes andcan possibly select the best one according to any arbitrarylocal import policy (e.g., next-hop preferences, shortest route,avoid specific AS networks). Then, by taking part in Select-Best, the IXP member can incorporate IXP information intoits route selection process. By implementing Export-Alland Select-Best via carefully optimized SMPC and keepingLocal-Ranking’s potentially highly complex computationoutside of the SMPC framework, this pipeline preserves themember’s and the IXP’s privacy, and is efficiently executable.

Before we get into the details of the two SMPC-based com-ponents of sixpack, we first formalize our model of export

policies (and next-hop rankings in §5.3. We then describeExport-All in §5.4 and Select-Best in §5.5. Due to spacelimit, a formal description of our protocol and proofs of itssecurity are given in our TR [65].

5.3 Routing Policies ModelExport-policy. The Export-All component of sixpackdispatches routes according to the export policies pertain-ing to the routes that it received from its members. Eachroute carries its own export policy specification, i.e., theset of members to whom that route can be exported. SinceBGP computes routes independently for each destination IPprefix, w.l.o.g., we henceforth assume through this sectionthat there only exists a single destination IP prefix π . More-over, the BGP route computation only depends on the lastroute announced by a neighbor. Hence, our model only needsto store the set of routes currently available at the RS andthe currently specified export policies. To achieve efficientSMPC computation, we model BGP export policies as fol-lows. LetM = {m1, . . . ,m |M |} be the set of IXPmembers andR = {r1, . . . , r |R |} be the set of available routes. We definethe export policy matrix P , with |M | rows and |R | columns.Entry Pi, j in the matrix, for 1 ≤ i ≤ |M | and 1 ≤ j ≤ |R |, is 1if route r j is exportable to membermi and 0 otherwise.

As an example of an export policymatrix, consider Fig. 2(a)on p. 3 wheremA,mB ,mC ,mD are IXP members and rA, rBare routes announced by mA and mB , respectively. Whileroute rA is exported tomC only, route rB is exported tomCandmD . Observe that rA and rB are not exported tomA andmB , respectively, i.e., to the IXP member they originate from.

In the Export-All component, all permissible routes areexported. Each IXP membermi should receive each route r jfor which Pi, j = 1. In Fig. 2(a),mC receives both rA and rB ,andmD receives only rB , whilemA andmB do not receive anyroute. In the Select-Best component, each membermi isentitled to receive any route r j with Pi, j = 1. The actual routethat will be received depends on the ranking over routes andIXP performance-related information.Next-hop ranking (a.k.a. local preferences over routes).Select-Best receives as input, from each participating IXPmember, its next-hop ranking with respect to the destinationIP prefix. Each next-hop correspond to a route announcedby a member, thus next-hop rankings model local preferencesover the received routes. For each IP prefix π , we model pref-erences over the available routes at the RS as a matrix Ψ withsize |M | × |M |. Each element ψi, j of that matrix representsmembermi ’s local preference (value) for routes announcedbymj , where routes announced by members with higher lo-cal preference are preferred over routes announced by mem-bers with lower local preference. Using Select-Best, eachIXP membermi thus receives a single route r announced by

6

SIXPACK: Securing Internet eXchange Points Against Curious onlooKers CoNEXT’17, December, 2017, Seoul, South Korea

Route  Server  RS1

Route  Server  RS2 Member  mD (receive)c

Member  mC (receive)route  rAMember  mA (send)

nonce  nAroute  rAexport:  mC

⊕

EXPORT-­‐ALLSMPC1

EXPORT-­‐ALLSMPC2

route  rA

⊕

route  rA

route  rA route  rAdecrypt

decrypt

Kdm

C2

⊕

kA

kAkA

kA

D1

C1

D2

D1

C1encrypt

key  kA

⊕

nonce  nKA

key  kA

route  rA kA

export:  mC nAkey  kA nKA

nKA

export:  mC nA

nonce  nA

nonce  nKA

C2

D2

Figure 4: The Export-All component. The RSes export to IXPmembers all permissible routes.

𝑃",$𝑘"

𝑘&' 𝑘()*,",$MUX

(a) Building Block Xj

X1

X|M|

(b) Circuit StructureFigure 5: The Ex-port-All circuit

mj such thatψi, j is the highest priority value in row i of Ψ,for which Pi, j = 1 holds. Ties are broken deterministically.The matrix Ψ can easily be extended to represent prefer-

ences over routes based on a combination of members’ localpreferences and IXP performance-related recommendations.For instance, if each preference value ψ is encoded as anρ = 8-bit integer, we can use the four most significant digitsofψ to create 16 different classes of members’ local prefer-ences over routes and use the four least significant bits tocreate another 16 additional classes for the IXP performance-related recommendations. In this way, the IXP informationis used only to break ties among routes with the same rank.Alternatively, IXP information can be given higher priorityand members’ local preferences used to break ties.

5.4 The Export-All ComponentThrough Export-All, the RSes export to each member allpermissible (i.e., exportable) routes while keeping each mem-ber’s export policy private. We note that this problem couldalso be solved by using public-key cryptography if we as-sume that each sending member knows the public keys of allother IXP members. However, a public-key solution alonecannot incorporate IXP performance-related informationwithout revealing it – a concern for 60% of the surveyedoperators. Furthermore, SMPC circumvents all key manage-ment challenges, protects against side-channel attacks, andeasily integrates with the Select-Best SMPC component.Observe that, in the Export-All component, not only is

the computation per-prefix independent, i.e., the computationis executed independently for each destination IP prefix, butit is per-route independent, in the sense that the announce-ment of a specific route to a member does not depend onwhat other routes to the same prefix are announced to thatmember. Hence, w.l.o.g., Export-All is described belowwithrespect to a single route to a single prefix π .Fig. 4 illustrates an example of the Export-All compu-

tation. Two independent RSes, RS1 and RS2 (center of thefigure), perform the redistribution of a route frommA accord-ing to its export policy. We consider the scenario presented

in Fig. 2(a). The computation operates over a policy that iskept private using SMPC and the route is dispatched in sucha way that neither the RSes nor the IXP members can distin-guish whether a route is announced to any other member ornot. Each member attempts to decrypt the information re-ceived from the RSes (right side of the figure). This operationsucceeds iff the route is actually exported to that member,which then learns the route. We now discuss in more detailthe different parts of Export-All.Encrypted routes. Each route that needs to be distributedis first encrypted. W.l.o.g., we assume that membermA wantsto send a route rA as shown in Fig. 4. Then,mA encrypts rAusing a route-specific key kA and a symmetric encryptionscheme (we use AES) and sends the route to RS1, which, inturn, redistributes it to all the members. These can decryptrA only if they possess the route key kA. sixpack guaranteesthat an IXP member receives the key kA only if the routecan be exported to it. We use a “dummy key” kdm to notify amember when a route cannot be exported to it. Recall that areceiving member does not have visibility of the routes inplain text, so it does not know which routes are announced.Even if a member colludes with one of the two RS entities,it cannot distinguish whether a certain route is exported toany of the other IXP members.Exporting keys via SMPC.We now leverage SMPC to dis-patch the keykA in a privacy-preservingmanner. Specifically,we devise a tailored SMPC circuit (shown in Fig. 5) that isexecuted by both RS1 and RS2. The Export-All circuit con-sists of one multiplexer per member, which outputs eitherthe valid or dummy key kdm depending on entry Pi, j , wherei (j) is the member announcing (receiving) a route.

To generate the SMPC input (left of Fig. 4), membermAtakes rA’s export policy and creates a noncenA that is XORedwith it. The result is sent to RS2, while nA is sent to RS1. Ob-serve that neither RS1 nor RS2 is able to decrypt the exportpolicy as they are assumed to not collude. Both RSes storethe share of the export policy received by the member. Anal-ogously,mA generates a noncemKA that is XORed with kA.The result is sent to RS2, while nKA is sent to RS1. We show

7

CoNEXT’17, December, 2017, Seoul, South Korea M. Chiesa, D. Demmler et al.

Member  mC (send)

⊕SELECT-­‐BESTSMPC1

SELECT-­‐BESTSMPC2

Member  mC (receive)

⊕route  rA

route  rBselector

IXP  reccomendationsrB >  rA

nonce  pC

mC rankingmB =  mA

nonce  pC

export:  mC

export:  mC,mD

nAnB

mC rankingmB =  mA pC nonce  nA

nonce  nBoutC1

mC rankingmB =  mA pC

outC2

Route  Server  RS2

Route  Server  RS1

Figure 6: The Select-Best component. The RSes export to each IXP member thebest preferred permissible route.

MUX

MUX≥ 𝜓#$%

𝑘#$%𝑘'𝑘(𝜓),'𝜓),(

(a) Route comparison circuit *

*

X **

X *X *

*

*

X

(b) Circuit Structure

Figure 7: Select-Best circuit.

in Fig. 2 on page 3 an example of (a) the export policies oftwo routes rA and rB , (b) the nonces chosen bymA andmB ,respectively, and (c) the resulting inputs to RS2.

Once the two RSes receive their shares of the export policyand key kA, SMPC1 and SMPC2 (center of Fig. 4) outputto every membermX two shares X1 and X2 of the output,respectively. XORing X1 with X2 (right of Fig. 4) producesa key that can be used to decrypt the encrypted route rA iffPX ,A = 1, or the dummy key kdm, otherwise. In Fig. 4,mCreceives C1 and C2; when XORed, they give kA. Similarly,mD receives D1 and D2; when XORed, they give kdm, leadingmD to discard the encrypted route rA. Note that our designof the Export-All circuit can execute on multiple routesat once from different members for more efficiency. In fact,Export-All takes as input an export policy matrix P (§5.3),which may consist of just one column as a special case.

5.5 The Select-Best ComponentTo leverage the superior IXP’s visibility into dataplane condi-tions, we design Select-Best, a privacy-preserving compo-nent that allows IXP members to select the best permissibleroute according to both their own local preferences and theIXP performance-related information.

To execute Select-Best, each IXP member will first trans-late its ranking of available routes (e.g., prefer shortest routes)to a ranking of the corresponding IXP members that an-nounced them, where members that are neither exportingnor announcing a route to this member are assigned thelowest preference. Analogously, the IXP translates its sen-sitive performance-related information, such as members’port utilization, to preference values. As explained in §5.3,these preferences values can be combined into a single pref-erence valueψ per route, where we envision the members’local preference is given precedence over the IXP’s inputs.This information is provided as input to Select-Best, whichcomputes in a privacy-preserving manner the best route. Wenote that this functionality cannot be realizedwith public-keycryptography alone and is thus an interesting and practicalreal-world application of SMPC.

Refer to Fig. 6 as an example of the Select-Best com-putation, where we consider the export policy scenario ofFig. 2(a). Namely, two membersmA andmB announced tworoutes rA and rB , respectively. Through Export-All,mC re-ceived both rA and rB whilemD only received rB , with thelatter deciding not to execute Select-Best. Based on portutilization levels and assumingmC equally ranks rA and rB ,the IXP gives route rB a higher preference.Choosing the best route via SMPC. We now leverageSMPC to select the best route of each IXP member in aprivacy-preserving manner. To this end, we devise a tai-lored SMPC circuit (shown in Fig. 7) that, for each memberm, takes as input the next-hop ranking (i.e., preferences overmembers) and the IXP route recommendations, and outputstom the identifier of the best route.

The first step (left of Fig. 7b) is similar to the Export-Allcircuit: based on the export policy, the preference of all non-exportable routes is set to zero. After that (right of Fig. 7b),we feed the resulting priorities as well as the route keys(which are used as identifiers) into a MaxIdx tree circuit [58,§3.3] and thus determine the best route, i.e., output the routekey with the highest preference. We also input the dummykey kdm, which is returned if the receiving member doesnot have any permissible route. The comparison among tworoutes rY and rW (announced by membersmY andmW , resp.)is performed by the * circuit (Fig. 7a), whereψZ ,Y andψZ ,Ware themZ ’s preferences over rY and rW , resp., while kY andkW are the keys of routes rY and rW , resp. The selection bitof the MUX is chosen such that the route key with the higherpreference value gets propagated to the next level of the tree.

Both input and output are considered private information,and are not visible to the RS in clear. The IXP membersare responsible for generating and then reconstructing theSMPC’s input and output through secret-sharing.

The input to the SMPC (i.e., the next-hop ranking) is gener-ated similarly to the input of the Export-All component. Inthe example (left side of Fig. 6),mC generates an |M | · ρ-bitsnonce pC that is XORed with its next-hop ranking (i.e., rowψC of the ranking matrix Ψ, §5.3). The XORed result is sent

8

SIXPACK: Securing Internet eXchange Points Against Curious onlooKers CoNEXT’17, December, 2017, Seoul, South Korea

to RS2, while pC is sent to RS1. Neither RS1 nor RS2 are ableto decrypt the ranking as they are assumed to not collude.

At this point, the two RSes combine the member’s prefer-ences with the IXP preferences, where only RS1, which runsat the IXP supplies the preferences based on performance in-formation; RS2 uses a vector of zeros. The two RSes executethe Select-Best circuit on the given inputs (center of Fig. 6).Then,mX receives two values outX 1 and outX 2; when XORed,they produce an identifier of the best route. In Fig. 6, mCreceives rB as the best route based on the IXP performance-based preference. Observe that ifmC had preferred rA overrB , it would have received an identifier to rA.

As for the other circuit, it is worth noting that our designof the Select-Best circuit can process multiple next-hoprankings at once from different members for improved effi-ciency. In fact, Select-Best takes as input a ranking matrixΨ (§5.3), which may consist of just one row as a special case.

6 IMPLEMENTATIONWe implemented the two SMPC components shown in Fig. 4and Fig. 6 in C++ using the ABY framework [27] in 700 LoCs.ABY is written in C++ and provides low-level primitives forbuilding Boolean circuits that are evaluated efficiently withthe GMWprotocol. With ABY’s SIMD evaluation, we can runthe same circuit for inputs from arbitrarily many members inparallel while vector operations allow the efficient processingof long bit strings, such as the route keys. The circuits thatwe built are optimized to process multi-bit values efficiently,which benefits the processing of route keys and comparingpreference values. The Select-Best circuit is evaluated in atournament fashion by arranging the preference comparisongates in a tree. Thereby we achieve a circuit depth that growslogarithmically in the number of members, thus resulting inoptimized latency for GMW. We designed our circuits to beas minimalistic and performant as possible, while still beingas expressive as needed for our use cases. Additionally, weextended the ABY framework with native input and outputoperations for outsourced SMPC, which saves two rounds ofcommunication for input sharing and output reconstruction.Our route server service, which wraps the SMPC com-

ponents and handles the distribution and processing of allthe BGP update information among the IXP members, isimplemented as 1,800 LoCs in Python. The RSes run as inde-pendent processes, each executing its own instance of SMPCas a daemon subprocess, and communicate via TCP sockets.

Currently, we do not minimize the BGP update processingtime by precomputing the SMPC setup phase. Although wedid not optimize our implementation to the fullest extentpossible, our evaluation (§7) shows that our approach al-ready scales to the size of the largest IXPs in the world. Wediscuss deployment consideration in §8. Further details onthe implementation are provided in our full paper [65].

7 EVALUATIONWe evaluate our sixpack prototype to demonstrate that ourapproach is both feasible and practical. We first provide in-sights into the performance of the SMPC part of sixpackby performing micro benchmarks across a realistic rangeof numbers of IXP members and inputs to our circuits. Wethen evaluate our system by replaying a real trace of BGP an-nouncements from one of the largest IXPs worldwide and byperforming a stress test. Our results highlight the following:(1) While SMPC is (as expected) the costliest part perfor-

mance-wise, our results show that the online phase is even atworst below 38ms. The maximum setup and online runtimewe measured in our evaluation were 131.9ms and 37.2ms,respectively for 32 inputs in the Select-Best component.

(2) Our still unoptimized sixpack prototype achieves BGPprocessing times below 90ms at the 99th-ile, and, specifically,below 23.6ms and 62.8ms for Export-All and Select-Bestat the 99th-ile, respectively. Furthermore, we measured neg-ligible bandwidth requirements. Finally, sixpack processesa full-routing-table of 250K prefixes in ≈11 minutes, com-parable to today’s RSes (see §7.3). We stress the fact thatour prototype can fairly easily be improved to achieve betterperformance by precomputing the SMPC setup phase.

It is worth comparing these numbers with the convergencetime of BGP on the Internet, which can be in the order ofminutes [70] , that is, several order of magnitude higher thantime overhead due to dispatching routes with sixpack.

7.1 IXP DatasetWe assess our system using a two-hour trace of BGP up-dates from one of the largest IXPs worldwide, which inter-connects more than 600 members. It contains 25,676 BGPupdate messages, consisting of 76,506 IP prefix announce-ments and withdrawals. The average number of BGP updatesper second is 3.57, the first and third quartiles are 2 and 4,respectively, while the minimum and maximum numbersper second are 1 and 29, respectively. The average numberof IP prefixes announcements or withdrawals per second is10.62, the first and third quartiles are 6 and 12, respectively,while the minimum and maximum numbers per second are2 and 379, respectively. In addition to that trace, our dataalso contains a snapshot of the RS routing table at the be-ginning of the trace of updates. The routing table containsroughly 400,000 routes towards ≈240,000 IP prefixes. Fromour dataset, we observed that for more than half of the IPprefixes there exists a single available route, with an averageof 1.9, the 95th percentile of 5 and a maximum of 25 routesper prefix. From our dataset, we observed that on averageeach member announces 626.5 routes, the 95th percentile is1581 and the maximum is roughly 150,000.

9

CoNEXT’17, December, 2017, Seoul, South Korea M. Chiesa, D. Demmler et al.

Table 1: SMPC micro benchmark overview.Setup phase Online phase

Approach # Inputs Runtime[ms] Comm.[KiB] Runtime[ms] Comm.[KiB]

Export-All 2 1.7 (±8 %) 28 0.6 (±17 %) 25

Select-Best

4 19.8 (±4 %) 1,492 5.3 (±4 %) 1068 34.6 (±6 %) 3,469 10.3 (±3 %) 24716 63.7 (±9 %) 7,409 19.3 (±3 %) 52832 122.4 (±8 %) 15,286 35.9 (±1 %) 1,091

7.2 SMPC MicrobenchmarksTo benchmark the SMPC circuits, we consider the scenario ofan IXP with 750 members, which is ≈ 1.5 times the numberof members connected to the RS encountered in the IXPdataset we analyzed. From the dataset, we observe that atmost 27 members export a route for the same IP prefix. Thus,we run benchmarks for a number of route keys up to 32.

We measure the runtime of setup and online phase ofthe SMPC. The setup phase is independent of both the pri-vate inputs and the function being evaluated and can beprecomputed at any time before the actual circuit evalua-tion. One invocation of SMPC corresponds to processing asingle IP prefix announcement. For all experiments, we per-form 50 executions of the circuit and report median valuesof the runtimes and their standard deviation. Measurementswere performed on two servers with a 2.6 GHz CPU and128GiB RAM, connected via a local 10Gbps network. Weevaluate with a preference value bit length of 8 bits. We userandom preferences as the SMPC performance does not de-pend on the actual input (otherwise it would be vulnerableto side-channel attacks). The reported communication is thesum of sent and received data by one party.Runtimes. Tab. 1 shows the setup and online runtimes aswell as the required communication. More detailed results,covering circuit gate counts, circuit depth, and required com-munication are shown in our technical report [65].

Our results show that the setup phase takes between 1.7msand 122.4ms, and depends on the number of inputs pro-cessed and the circuit used. In the best case, an IP prefix isannounced by a single member and the Export-All com-ponent can be used to dispatch the announcement to thelegitimate members since no route comparison is needed.In the Export-All component, each route is processed

independently, even those towards the same destination IPprefix. This case corresponds to the computation with just2 inputs (i.e., a given route and the dummy one) and requiresan online computation of only 0.6 ms and an amount oftransferred data of 25 KiB. However, as both setup and onlineruntimes grow sub-linear with the number of routes, it isbeneficial to compute onmany routes in parallel, e.g., at timeswhere several BGP announcements happen simultaneously.The runtimes of Select-Best are large due to the deeperSMPC circuit. Note that Select-Best may not be used whenan IP prefix is announced by a single route (i.e., 2 inputs).

We also verified that our SMPC circuits scale linearly withrespect to the number of members, i.e., with 1,500 members,Select-Best takes less than 70ms to process 32 routes.Memory Consumption. We further measure the memoryconsumption of our SMPC implementation. Even when pro-cessing large inputs of 32 routes and 2,000 members, thememory consumption of the dispatching operation, whichis consumed only during a route dispatch, remains below15MiB. We determined that memory consumption growssub-linear with increasing parameters, which shows that ourimplementation will remain practical in the future.

In summary, these performance numbers confirm the prac-ticality of our SMPC implementation and sixpack.

7.3 Prototype EvaluationTo assess the feasibility of sixpack, we focus on evaluatingits two main building blocks, i.e., Export-All and Select-Best, against a real-world trace of BGP updates collectedfrom one of the largest IXPs in the world. To assess sixpack’sscalability, we performed a stress test of Export-All andSelect-Best and considered edge case scenarios such as theconnection of a new member to the IXP network.Experimental setup. We performed our experiments onthree servers with 16 hyper-threaded cores at 2.60 GHz with128GiB of RAM and Ubuntu Linux 14.04, each two connectedthrough 10Gbps links. The average latency is 100 µs , sim-ilarly to latencies reported in co-location data centers [4].We use one server to replay the stream of BGP updates fromour dataset and to handle the receiving part of the sixpackmechanism. In each experiment, we load the full RS routingtable contained in our dataset into the two RS instances.Each RS instance runs on a server and consists of a set

of worker processes orchestrated by a single handler pro-cess. The latter one (i) redistributes received BGP updates tothe workers, (ii) guarantees that the other RS instance alsoknows which worker must be used to process a BGP update,(iii) synchronizes sixpack’s operations, (iv) redistributes theSMPC outputs to the members, and (v) guarantees that twoBGP updates for the same IP prefix are not processed simul-taneously by two different workers. Each worker runs aninstance of the SMPC party and computes its inputs.Each BGP update can be either an announcement or a

withdrawal of a set of IP prefix destinations that share thesame export policy. In order to evaluate Select-Best, weassign random local preferences among all the IXP members.Performance analysis on real-worldBGP trace. For eachBGP route announcement r , we measure the amount of timerequired to create the input values for the SMPC process-ing, the time required to run the SMPC computation, theamount of time the announcement is handled by a workerprocess, and the total time between the reception of r andthe transmission of its output to the IXP members.

10

SIXPACK: Securing Internet eXchange Points Against Curious onlooKers CoNEXT’17, December, 2017, Seoul, South Korea

0 10 20 30 400

0.20.40.60.81

Processing time [ms]

Shareof

IPprefi

xes

Export-All

0 20 40 60 80 1000

0.20.40.60.81

Processing time [ms]Sh

areof

IPprefi

xes

Select-Best

SMPC-input SMPC-run worker handler

Figure 8: Processing time CDFs for Export-All and Sel-ect-Best components with 8 workers per RS instance.We plot the processing time of each IP prefix from our

dataset in Fig. 8. We observe similar trends in both Export-All and Select-Best (i.e, Step I and Step III of sixpack), withthe latter one being 2 times slower than the former one. Inboth graphs, we see a large gap between worker and handlertimes. This is mostly due to the unoptimized utilization ofshared data structures in Python. Second, worker executionis dominated by the SMPC processing, with SMPC inputcreation requiring only a negligible amount of computationalresources. We also measure the time required to encrypt anddecrypt routing announcements, finding AES operations tobe a negligible amount of time compared to the handler.To summarize, the average processing times are 15.9ms

and 32.7ms for Export-All and Select-Best, respectively,while the 99th percentiles are 23.6ms and 62.8ms, respec-tively. This low running time reflects the fact that most ofthe routes are announced by few members (see §7.1),The per-formance of Select-Best is dominated by the SMPC compu-tation. We recall that the SMPC could be further optimizedby precomputing the setup phase ahead of time.

We also measured the amount of communication requiredby sixpack during the experiment. We found that for bothExport-All and Select-Best the average bandwidth re-quirements from an IXP member to the RSes are negligible(i.e., 20 Kbps). The requirements are higher for the communi-cation between the two RSes. In the Export-All component,the average bandwidth requirement is less than 2.79Mbpswhile in Select-Best it is no more than 10.9Mbps. Thesefigures show that even a 1Gbps link between the two RSeswould be more than sufficient for supporting sixpack.Stress test.To assess the scalability of our system,we floodedsixpack with announcements and counted the number ofroutes dispatched per second. Our evaluation follows two di-mensions: number of routes announced for the same IP prefix(including a dummy route) and number of parallel workersand plotted our results in Fig. 9. We observed that sixpackprocesses 873.75 route announcements per second in the Ex-port-All component and 137.5 announcements per secondfor IP prefixes announced through 8 routes. The bandwidthrequirements at full-speed are always below 1.5Gbps.

2 4 8 16 321

10

100

1,000

Number of available routes per prefix

Throug

hput

[msg/sec]

1 2 4 8 Workers

Figure 9: Throughput test of sixpack for different num-ber of parallel workers.Connecting a new member: comparing with today’sRSes.When a newmemberM connects to the RS, two opera-tions are performed: (i) the RS propagates to the newmemberall the best permissible routes towards the IP prefixes thatwere previously announced by the other members and (ii)the RS recomputes and propagates to all the members thebest route for each IP prefix announced byM . As for opera-tion (i), at large IXPs, ≈250 K best-route computations mustbe performed, one for each prefix known to the RS. Whilethis may sound problematic, based on our datasets, most ofthe IP prefixes are announced by a single member, henceenabling us to consistently leverage the Export-All com-ponent. Moreover, each best route is computed only for oneIXP member and not for all of them. This allows us to con-siderably speed up the SMPC execution. For instance, whileexecuting the Select-Best component for 500members with32 routes takes on average 158ms, the same computation fora single member takes just 9ms. We verified that operation(i) takes on average 92.8 s with our dataset. As for operation(ii), from our dataset, we observed that most of the customersdo not announce more than 1K BGP routes. Our stress testshows that such operations would not take more than a fewseconds. For large customers announcing ≈250K prefixes,we verified that the announcement operation takes roughly11minutes. In comparison, today’s RSes report convergencetimes ranging between 3 and 10 minutes [2, 25, 77], evenwithout incorporating import policies, performance-driveninformation, and privacy functionality.

8 FREQUENTLY ASKED QUESTIONSIs sixpack ineffective since routing policies can be eas-ily inferred? No. While many techniques have been de-signed to infer peering relationships and policies [6, 19, 28,29, 52, 68, 80, 83], such solutions not only have limited accu-racy [75] (e.g., globally visible AS-paths neither reveal localpreferences nor “negative” export policies, i.e., not exportinga route) but might also be detectable (e.g., BGPAS-path spoof-ing [52]). Finally, sixpack supports ranking routes based onthe IXP members’ port utilization, a fundamental perfor-mance metric that is challenging to infer in practice [22].

11

CoNEXT’17, December, 2017, Seoul, South Korea M. Chiesa, D. Demmler et al.

How do you envision sixpack being adopted by IXPs?Traditional RS service and sixpack can coexist. Each mem-ber can choose whether to share its routes in a privacy-preserving manner or not. We argue that the desire to main-tain peering relationships private will incentivize adoption.Members could run sixpack as a BGP proxy that interfacesbetween the BGP border router and the SMPC process. Mem-bers not using sixpack require no changes to their infrastruc-ture. Moreover, sixpack is facilitated by the increasing trendtowards SDN deployments at IXPs [3, 24, 39, 40, 61, 81].Whynot use Intel SGX inplace of SMPC? Software GuardeXtensions (SGX) is an instruction set [20, 51] that allowsprogrammers to perform computation on data stored withinprivate regions of memory that are not accessible by unau-thorized processes. Despite its promise, SGX is currently thesubject of many discussions regarding its real level of secu-rity. In contrast, SMPC is a well-established methodologywith proven security guarantees. A major concern regardingSGX programs is that timing or memory access patterns leakinformation about the private inputs. Our SMPC approach,in contrast, does not suffer from these problems. Anothergeneral concern regarding SGX is Intel’s role as the central-ized point of trust. SMPC allows any two entities to guaran-tee privacy-preservation. Because of the above SGX limita-tions, recent studies propose combining SGX with SMPC tostrengthen the privacy of outsourced computation [41, 57].Thus, we consider SGX as complementary to our solution.Does sixpack nullify the benefits of RSes? No. sixpackpreserves centralized route computation while tackling 3 outof 4 concerns from operators with RSes (i.e., route visibility,best route control, privacy), enhancing RS functionality withperformance information, and retaining easy management.Does sixpack leak information? Members only learnthat (encrypted) routes are announced. The RSes also learn IPprefixes and announcing members of each encrypted route.

9 RELATEDWORKGreat efforts are invested in making SMPC a practical ap-proach with real-life applications. Indeed, SMPC has beenapplied to a broad spectrum of applications [9–12].

In the networking realm, the first works to apply an SMPCapproach to interdomain routing were [7, 42]. These worksstudy how BGP routes across the whole Internet can becomputed in a privacy-preserving manner. While a new andexciting direction, the achieved runtimes are impractical,even on small topologies and high limitation on BGP expres-siveness. We focus instead on IXPs, the crucial crossroadsof the Internet that run computation on private, business-sensitive routing information. We argue that applying SMPCto this narrower context is a promising approach to privacy-preserving interdomain route-computation and we built aprototype that handles real-world traces from a large IXP.

Another routing-related study is SPIDER [88], a distributedmechanism for verifying if a peering agreement betweenASes (involving, e.g., a requirement to always export theshortest route available) is respected by the involved par-ties without revealing control-plane information (e.g., whichroutes are available). Applying SPIDER to our context couldaid IXP members in verifying that the RS is indeed executingthe protocol (in contrary to, e.g., selecting an un-optimalroute for each member). Our focus in this paper is differ-ent: we guarantee that the RS does not learn anything aboutmembers’ export and import policies. Finally, while SGXcan be used to preserve the privacy of interdomain routingpolicies [54, 63], we discussed its limitations in §8.

Past studies utilized SMPC to address privacy concerns in avariety of other networking-related problems [13, 16, 67, 76].Unlike these studies, our focus is on guaranteeing the privacyof peering policies. Additionally, the protocols proposed inthese studies are either evaluated under questionable condi-tions, or exhibit runtimes in the order of seconds, whereasRSes are required to operate at faster runtimes.

Homomorphic encryption schemes [30, 31, 37] are cryptosystems that computes over encrypted data. However, suchgeneral schemes are slow for practical applications [32]. Evenimplementations tailored to simple regular expression eval-uation, as those presented in BlindBox [79], are either toorestrictive for supporting Select-Best, i.e., by only providingexact string match, or leak information whenever a match isfound, which corresponds to the IXP learning routes when-ever a best route is selected by amember. Finally, [43] designsa shortest-path vector-based routing protocol that uses ho-momorphic encryption but is not as expressive as BGP.We note that in dealing with the privacy of export poli-

cies, our work solves an orthogonal problem to that of se-curing BGP routing from IP-prefix hijacks and BGP path-manipulation, which is an active topic of research [15, 33, 46].Finally, while higher visibility over routes has been pro-posed(e.g., BGP Add-Paths [47]), when deploying such tech-niques at the RS, no confidentiality about the export policiesand IXP performance-related information is guaranteed.

10 CONCLUSIONSWe presented sixpack, a privacy-preserving IXP RS designwith provable guarantees. sixpack dispatches routes accord-ing to highly expressive members’ routing policies and IXPperformance-related information. We showed that an effi-cient realization of sixpack with Secure Multi-Party Compu-tation can be attained through a careful redistribution of theroute dispatching responsibilities between the RS and IXPmembers. We devised optimized SMPC circuits tailored to RScomputation. We built a sixpack prototype and assessed itspractical feasibility with a real-world trace of BGP updatescollected from one of the largest world-wide IXPs.

12

SIXPACK: Securing Internet eXchange Points Against Curious onlooKers CoNEXT’17, December, 2017, Seoul, South Korea

REFERENCES[1] B. Ager, N. Chatzis, A. Feldmann, N. Sarrar, S. Uhlig, and W. Willinger.

2012. Anatomy of a Large European IXP. In SIGCOMM’12. ACM.[2] AMS-IX 2012. Follow up :AMS-IX Route-Server Performance Test

Euro-IX 20th. (2012). https://ripe64.ripe.net/presentations/49-Follow_Up_AMS-IX_route-server_test_Euro-IX_20th_RIPE64.pdf.

[3] AMS-IX 2016. AMS-IX: Megaport and AMS-IX Partner to ProvideGlobal SDN-Enabled Elastic Interconnection and Internet ExchangeService. (Jan. 2016). https://ams-ix.net/newsitems/233.

[4] AMS-IX 2016. AMS-IX: Real-Time-Statistics. (Feb. 2016). https://ams-ix.net/technical/statistics/real-time-stats.

[5] AMSIX 2017. Amdsterdam Internet eXchange Infrastructure. (2017).https://ams-ix.net/technical/ams-ix-infrastructure.

[6] R. Anwar, H. Niaz, D. R. Choffnes, Í. S. Cunha, P. Gill, and E. Katz-Bassett. 2015. Investigating Interdomain Routing Policies in the Wild.In IMC 2015.

[7] G. Asharov, D. Demmler, M. Schapira, T. Schneider, G. Segev, S. Shenker,and M. Zohner. 2017. Privacy-Preserving Interdomain Routing atInternet Scale. Proceedings on Privacy Enhancing Technologies (PoPETs)2017, 3 (2017).

[8] D. Barrera, L. Chuat, A. Perrig, R. M. Reischuk, and P. Szalachowski.2017. The SCION Internet Architecture. Commun. ACM (2017).

[9] D. Bogdanov, M. Jõemets, S. Siim, and M. Vaht. 2015. How the EstonianTax and Customs Board Evaluated a Tax Fraud Detection SystemBased on Secure Multi-party Computation. In FC’15 (LNCS), Vol. 8975.Springer, 227–234.

[10] D. Bogdanov, L. Kamm, S. Laur, P. Pruulmann-Vengerfeldt, R. Talviste,and J. Willemson. 2014. Privacy-Preserving Statistical Data Analysison Federated Databases. In Annual Privacy Forum (APF’14) (LNCS),Vol. 8450. Springer, 30–55.

[11] D. Bogdanov, R. Talviste, and J. Willemson. 2012. Deploying SecureMulti-Party Computation for Financial Data Analysis - (Short Paper).In FC’12 (LNCS), Vol. 7397. Springer, 57–64.

[12] P. Bogetoft, D. L. Christensen, I. Damgård, M. Geisler, T. Jakobsen,M. Krøigaard, J. D. Nielsen, J. B. Nielsen, K. Nielsen, J. Pagter, M.Schwartzbach, and T. Toft. 2009. Secure Multiparty Computation GoesLive. In FC’09 (LNCS), Vol. 5628. Springer, 325–343.

[13] M. Burkhart, M. Strasser, D. Many, and X. Dimitropoulos. 2010. SEPIA:Privacy-preserving Aggregation of Multi-domain Network Events andStatistics. In USENIX Security 2010.

[14] N. Büscher and S. Katzenbeisser. 2015. Faster Secure Computationthrough Automatic Parallelization. In USENIX Security’15.

[15] K. Butler, T. R. Farley, P. McDaniel, and J. Rexford. 2010. A Survey ofBGP Security Issues and Solutions. Proc. IEEE 98, 1 (2010), 100–122.

[16] M. Canini, V. Jovanović, D. Venzano, G. Kumar, D. Novaković, and D.Kostić. 2011. Checking for Insidious Faults in Deployed Federated andHeterogeneous Distributed Systems. Technical Report 164475. EPFL.

[17] David R. Choffnes and Fabian E. Bustamante. 2009. On the Effec-tiveness of Measurement Reuse for Performance-Based Detouring. InINFOCOM’09.

[18] Cisco [n. d.]. BGP Best Path Selection Algorithm. ([n. d.]).[19] L. Cittadini, G. Di Battista, T. Erlebach, M. Patrignani, and M. Rimon-

dini. 2010. Assigning AS relationships to satisfy the Gao-Rexfordconditions. In International Conference on Network Protocols (ICNP’10).IEEE, 113–123.

[20] V. Costan and S. Devadas. 2016. Intel SGX Explained. CryptologyePrint Archive, Report 2016/086. (2016). http://ia.cr/2016/086.

[21] D. Croce, E. Leonardi, and M. Mellia. 2011. Large-Scale AvailableBandwidth Measurements: Interference in Current Techniques. IEEETrans. Network and Service Management 8, 4 (2011), 361–374.

[22] D. Croce, M. Mellia, and E. Leonardi. 2010. The Quest for Bandwidth Es-timation Techniques for Large-scale Distributed Systems. SIGMETRICS

Perform. Eval. Rev. 37, 3 (Jan. 2010).[23] DECIX 2013. Deutscher Commercial Internet Exchange Infrastruc-

ture. (2013). https://apollon.de-cix.net/news/blog-post/2013/07/26/de-cix-apollons-current-topology/.

[24] DECIX 2015. Project ENDEAVOUR. (Jan. 2015). https://www.de-cix.net/en/about-de-cix/research-and-development/endeavour.

[25] DECIX 2016. An IXP Route Server Test Frame-work. (2016). https://www.de-cix.net/_Resources/Persistent/fba89bc19381b6784df99d2a78d4a11ebb7583c2/DE-CIX-route-server-testframework.pdf.

[26] DECIX 2017. Deutscher Commercial Internet Exchange. (2017). https://www.de-cix.net/.

[27] D. Demmler, T. Schneider, and M. Zohner. 2015. ABY – A Frame-work for Efficient Mixed-Protocol Secure Two-Party Computation. InNDSS’15. The Internet Society.

[28] X. Dimitropoulos, D. Krioukov, M. Fomenkov, B. Huffaker, Y. Hyun, K.Claffy, and G. Riley. 2007. AS Relationships: Inference and Validation.SIGCOMM Comput. Commun. Rev. 37, 1 (Jan. 2007), 29–40.

[29] L. Gao. 2001. On Inferring Autonomous System Relationships in theInternet. IEEE/ACM Trans. Netw. 9, 6 (Dec. 2001), 733–745.

[30] S. Garg, C. Gentry, S. Halevi, M. Raykova, A. Sahai, and B. Waters. 2013.Candidate Indistinguishability Obfuscation and Functional Encryptionfor All Circuits. In FOCS’13. IEEE, 40–49.

[31] C. Gentry. 2009. Fully Homomorphic Encryption Using Ideal Lattices.In STOC’09. ACM, 169–178.

[32] C. Gentry, S. Halevi, and N. P. Smart. 2012. Homomorphic evaluationof the AES circuit. In CRYPTO’12 (LNCS), Vol. 7417. Springer, 850–867.

[33] P. Gill, M. Schapira, and S. Goldberg. 2011. Let the Market DriveDeployment: a Strategy for Transitioning to BGP Security. In SIG-COMM’11.

[34] P. Gill, M. Schapira, and S. Goldberg. 2014. A Survey of InterdomainRouting Policies. Computer Communication Review (2014).

[35] O. Goldreich. 2004. The Foundations of Cryptography - volume 2, BasicApplications. Cambridge University Press.

[36] O. Goldreich, S. Micali, and A. Wigderson. 1987. How to Play anyMental Game or A Completeness Theorem for Protocols with HonestMajority. In STOC’87. ACM, 218–229.

[37] S. Goldwasser, Y. Kalai, R. A. Popa, V. Vaikuntanathan, and N. Zel-dovich. 2013. Reusable Garbled Circuits and Succinct Functional En-cryption. In STOC’13. ACM, 555–564.

[38] Google Espresso 2017. Cloud Native Networking. (2017). AminVahdat’s keynote at Open Networking Summit. Available at http://bit.ly/2qIZigQ.

[39] A. Gupta, R. MacDavid, R. Birkner, M. Canini, N. Feamster, J. Rexford,and L. Vanbever. 2016. An Industrial-Scale Software Defined InternetExchange Point. In NSDI’16. USENIX, 1–14.

[40] A. Gupta, L. Vanbever, M. Shahbaz, S. P. Donovan, B. Schlinker, N.Feamster, J. Rexford, S. Shenker, R. Clark, and E. Katz-Bassett. 2014.SDX: A Software Defined Internet Exchange. In SIGCOMM’14. ACM,551–562.

[41] D. Gupta, B. Mood, J. Feigenbaum, K. R. B. Butler, and P. Traynor. 2016.Using Intel Software Guard Extensions for Efficient Two-Party SecureFunction Evaluation. In Financial Cryptography and Data Security - FC2016 International Workshops, BITCOIN, VOTING, and WAHC.

[42] D. Gupta, A. Segal, A. Panda, G. Segev, M. Schapira, J. Feigenbaum, J.Rexford, and S. Shenker. 2012. A new approach to interdomain routingbased on secure multi-party computation. In HotNets’12. ACM, 37–42.

[43] W. Henecka and M. Roughan. 2013. STRIP: Privacy-preserving vector-based routing. In ICNP 2013.

[44] A. Holzer, M. Franz, S. Katzenbeisser, and H. Veith. 2012. Securetwo-party computations in ANSI C. In CCS’12. ACM, 772–783.

[45] Y. Huang, D. Evans, J. Katz, and L. Malka. 2011. Faster Secure Two-Party Computation Using Garbled Circuits. In USENIX Security’11.

13

CoNEXT’17, December, 2017, Seoul, South Korea M. Chiesa, D. Demmler et al.

USENIX, 539–554.[46] G. Huston, M. Rossi, and G. Armitage. 2011. Securing BGP – A Lit-

erature Survey. Communications Surveys Tutorials, IEEE 13, 2 (2011),199–222.

[47] Internet-Draft 2014. Advertisement of Multiple Paths in BGP. (Oct.2014). https://tools.ietf.org/html/draft-ietf-idr-add-paths-10.

[48] Internet Draft 2017. Making Route Servers Aware of Data Link Failuresat IXPs. (2017). https://tools.ietf.org/html/draft-ietf-idr-rs-bfd-02.

[49] Interxion 2017. Interxion Colocation Services. (2017). http://www.interxion.com/.

[50] M. Jain and C. Dovrolis. 2008. Path Selection Using Available Band-width Estimation in Overlay-Based Video Streaming. Computer Net-works 52, 12 (2008), 2411–2418.

[51] P. Jain, S. J. Desai, M.-W. Shih, T. Kim, S. M. Kim, J.-H. Lee, C. Choi, Y.Shin, B. B. Kang, and D. Han. 2016. OpenSGX: An Open Platform forSGX Research. In NDSS 2016.

[52] U. Javed, I. Cunha, D. R. Choffnes, E. Katz-Bassett, T. E. Anderson,and A. Krishnamurthy. 2013. PoiRoot: investigating the root cause ofinterdomain path changes. In SIGCOMM’13. ACM, 183–194.

[53] M. Keller. 2015. The Oblivious Machine – or: How to Put the C intoMPC. IACR Cryptology ePrint Archive (2015), 467. http://ia.cr/2015/467.

[54] S. Kim, Y. Shin, J. Ha, Taesoo Kim, and D. Han. 2015. A First StepTowards Leveraging Commodity Trusted Execution Environments forNetwork Applications. In HotNets’15. ACM.

[55] S. M. Kim, J. Han, J. Ha, T. Kim, and D. Han. 2017. Enhancing Se-curity and Privacy of Tor’s Ecosystem by Using Trusted ExecutionEnvironments. In NSDI 2017.

[56] R. Kloeti, M. Rost, P. Georgopoulos, B Ager, S. Schmid, and Dimitropou-los X. 2016. Stitching Inter-Domain Paths over IXPs. In SOSR’16. ACM.

[57] P. Koeberl, V. Phegade, A. Rajan, T. Schneider, S. Schulz, and M. Zh-danova. 2015. Time to Rethink: Trust Brokerage Using Trusted Execu-tion Environments. In Trust and Trustworthy Computing (TRUST’15)(LNCS), Vol. 9229. Springer, 181–190.

[58] V. Kolesnikov, A.-R. Sadeghi, and T. Schneider. 2009. Improved GarbledCircuit Building Blocks and Applications to Auctions and ComputingMinima. In CANS’09 (LNCS), Vol. 5888. Springer, 1–20.

[59] B. Kreuter, B. Mood, A. Shelat, and K. Butler. 2013. PCF: a portablecircuit format for scalable two-party secure computation. In USENIXSecurity’13. USENIX, 321–336.

[60] T. Lee, C. Pappas, D. Barrera, P. Szalachowski, and A. Perrig. 2016.Source Accountability with Domain-brokered Privacy. In CoNEXT2016.

[61] LightReading. 2015. Pica8 Powers French TOUIXSDN-Driven Internet Exchange. (June 2015). http://www.lightreading.com/white-box/white-box-systems/pica8-powers-french-touix-sdn-driven-internet-exchange/d/d-id/716667.

[62] C. Liu, X. Shaun Wang, K. Nayak, Y. Huang, and E. Shi. 2015. ObliVM:A Programming Framework for Secure Computation. In S&P’15. IEEE.

[63] M., R. Di Lallo, G. Lospoto, H. Mostafaei, M. Rimondini, and G. DiBattista. 2017. PrIXP: Preserving the Privacy of Routing Policies atInternet eXchange Points. In IFIP/IEEE International Symposium onIntegrated Network Management, IM.

[64] M. Chiesa, D. Demmler, M. Canini, M. Schapira, T. Schneider 2017.Internet Routing Privacy Survey. (2017). http://bit.ly/2rjT7Nj.

[65] M. Chiesa, D. Demmler, M. Canini, M. Schapira, T. Schneider. 2017.Securing Internet eXchange Points Against Curious onlooKers. (Jan.2017). http://bit.ly/sixpack-tech-rep.

[66] S. Machiraju and R. H. Katz. 2004. Reconciling Cooperation with Con-fidentiality in Multi-Provider Distributed Systems. Technical ReportUCB/CSD-04-1345. EECS Department, University of California, Berke-ley. http://www.eecs.berkeley.edu/Pubs/TechRpts/2004/5834.html

[67] S. Machiraju and R. H. Katz. 2004. Verifying Global Invariants inMulti-Provider Distributed Systems. In HotNets’04.

[68] H. V. Madhyastha, E. Katz-Bassett, T. Anderson, A. Krishnamurthy, andA. Venkataramani. 2009. iPlane Nano: Path Prediction for Peer-to-peerApplications. In NDSI’09. USENIX, 137–152.

[69] D. Malkhi, N. Nisan, B. Pinkas, and Y. Sella. 2004. Fairplay - SecureTwo-Party Computation System. In USENIX Security’04. 287–302.

[70] Z. M. Mao, R. Bush, T. Griffin, and M. Roughan. 2003. BGP Beacons.In SIGCOMM’03. ACM, 1–14.

[71] J. B. Nielsen, P. S. Nordholt, C. Orlandi, and S. S. Burra. 2012. ANew Approach to Practical Active-Secure Two-Party Computation. InCRYPTO’12 (LNCS), Vol. 7417. Springer, 681–700.

[72] P. Papageorge, J. McCann, and M. Hicks. 2009. Passive AggressiveMeasurement with MGRP. SIGCOMM’09 39, 4 (2009), 279–290.

[73] A. K. Paul, A. Tachibana, and T. Hasegawa. 2016. An Enhanced Avail-able Bandwidth Estimation Technique for an End-to-End NetworkPath. IEEE Trans. Network and Service Management 13, 4 (2016), 768–781.

[74] P. Richter, G. Smaragdakis, A. Feldmann, N. Chatzis, J. Boettger, and W.Willinger. 2014. Peering at Peerings: On the Role of IXP Route Servers.In IMC 2014.

[75] M. Roughan, W. Willinger, O. Maennel, D. Perouli, and R. Bush. 2011.10 Lessons from 10 Years of Measuring and Modeling the Internet’s Au-tonomous Systems. IEEE Journal on Selected Areas in Communications29, 9 (2011), 1810–1821.

[76] M. Roughan and Y. Zhang. 2006. Privacy-Preserving PerformanceMeasurements. In Workshop on Mining Network Data (MineNet’06).ACM, 329–334.

[77] S. Sugimoto and W. Ishida 2014. Performance Evaluation of BIRDand GoBGP. (2014). https://www.euro-ix.net/m/uploads/2016/04/24/EuroIX_GoBGP_20160419.pdf.

[78] T. Schneider and M. Zohner. 2013. GMW vs. Yao? Efficient SecureTwo-Party Computation with Low Depth Circuits. In FC’13 (LNCS),Vol. 7859. Springer, 275–292.

[79] J. Sherry, C. Lan, R. A. Popa, and S. Ratnasamy. 2015. BlindBox: DeepPacket Inspection over Encrypted Traffic. In SIGCOMM’15. ACM, 14.

[80] R. Sherwood, A. Bender, and N. Spring. 2008. Discarte: a disjunctiveinternet cartographer. In SIGCOMM’08. 303–314.

[81] J. Stringer, Q. Fu, C. Lorier, and C. E. Rothenberg. 2013. Cardigan:Deploying a Distributed Routing Fabric. In Hot Topics in SoftwareDefined Networking (HotSDN’13). ACM, 169–170.

[82] S. Tao and R. Guérin. 2004. On-line Estimation of Internet Path Per-formance: An Application Perspective. In INFOCOM’04.

[83] F. Wang and L. Gao. 2003. On Inferring and Characterizing InternetRouting Policies. In Internet Measurement Conference (IMC’03). ACM,15–26.

[84] H. Wang, K. S. Lee, E. Li, C. L. Lim, A. Tang, and H. Weatherspoon.2014. Timing is Everything: Accurate, Minimum Overhead, AvailableBandwidth Estimation in High-speed Wired Networks. In InternetMeasurement Conference (IMC ’14). 407–420.

[85] X. Wang and M. K. Reiter. 2004. Mitigating Bandwidth-ExhaustionAttacks Using Congestion Puzzles. In CCS’04.

[86] C. Xing, L. Yang, and M. Chen. 2009. Estimating Internet Path Proper-ties for Distributed Applications. InWiCOM’09. 4179–4182.

[87] A. C. Yao. 1986. How to Generate and Exchange Secrets. In FOCS’86.IEEE, 162–167.

[88] M. Zhao, W. Zhou, A. J. T. Gurney, A. Haeberlen, M. Sherr, and B. T.Loo. 2012. Private and Verifiable Interdomain Routing Decisions. InSIGCOMM’12. ACM, 383–394.

[89] M. Zhao,W. Zhou, A. J. T. Gurney, A. Haeberlen, M. Sherr, and B. T. Loo.2016. Private and Verifiable Interdomain Routing Decisions. IEEE/ACMTrans. Netw. 24, 2 (2016), 1011–1024.

14