Download - TOR IEEE Paper

Comparison of Low-Latency Anonymous Communication Systems -

Practical Usage and Performance

Thorsten Ries, Andriy Panchenko, Radu State and Thomas Engel

Interdisciplinary Centre for Security Reliability and TrustUniversity of Luxembourg

Email: {thorsten.ries, andriy.panchenko, radu.state, thomas.engel}@uni.lu

Abstract

The most popular system for providing practical low-latency anonymity on the Internet is Tor. However,many other tools besides Tor exist as both free andcommercial solutions. In this paper, we consider�ve most popular low-latency anonymisation servicesthat represent the current state of the art: single-hop proxies (Perfect Privacy and free proxies) andOnion Routing based solutions (Tor, I2P, and Jon-Donym). We assess their usability and rank themin regard to their anonymity. We also assess theire�ciency and reliability. To this end, we de�ne aset of metrics and present extensive measurementsbased on round-trip time, inter-packet delay variationand throughput. Apart from the technical realization,economic aspects are also crucial for anonymous com-munication systems. In order to attract more users,which is mandatory in order to improve anonymityper se, systems need to exhibit a certain payo�. Wetherefore de�ne an economic model that takes all rel-evant aspects into consideration. In this paper, wedescribe the results obtained, lessons learned, andprovide guidance for selecting the most appropriatesystem with respect to a set of requirements.

1 Introduction

For various reasons, people want to protect their iden-tity when communicating over the Internet. Doingso, they protect their privacy. Freedom of expressionmay be one motivation, while another reason may bea company or customer with the need to stay anony-mous1 for certain business transactions.

Based on this need, the aim of this paper is to com-pare existing implementations of anonymising sys-tems with respect to users' requirements such as per-formance and usability, also taking into account as-pects of anonymity and security as well as the realcosts, i.e., monetary costs the user faces. To this end,we assessed �ve tools that represent the di�erent ap-proaches and the current state-of-the-art in practicalanonymisation: free proxies, Perfect Privacy 2, Jon-Donym 3, Tor 4, and I2P 5.

1The term anonymity derives from the Greek word ανωνυµιαand means "without a name" or "namelessness"

2http://www.perfect-privacy.com3https://anonymous-proxy-servers.net/en/index.html4http://torproject.org5http://www.i2p2.de/

Copyright c©2011, Australian Computer Society, Inc. This pa-per appeared at the 9th Australasian Information Security Con-ference (AISC 2011), Perth, Australia, January 2011. Confer-ences in Research and Practice in Information Technology (CR-PIT), Vol. 116, Colin Boyd and Josef Pieprzyk, Ed. Reproduc-tion for academic, not-for pro�t purposes permitted providedthis text is included.

In recent years, research in anonymity has beenvery active, with many approaches developed. How-ever, only a very few of these reached wide-scale de-ployment and are used in practice. The predominantsystem in use today is Tor, developed by Dingledineet al. (2004). Tor is considered to be a low-latencyanonymisation tool, which means that data is sup-posed to be delivered within a reasonable time, al-lowing the usage of interactive applications such asweb browsing.

In contrast, high-latency systems such as Mixmas-ter and Mixminion, developed by Moeller et al. (2003)and Danezis et al. (2003) respectively, provide a highdegree of anonymity and should be considered for ex-change of �more sensitive� information. As a draw-back, communications like anonymous web browsingwould not practically possible because of the longdelays. Beside these, several other anonymisationtools exist as both free or commercial solutions follow-ing di�erent design approaches; current low-latencyapproaches can basically be divided into single-hopproxies and Onion Routing approaches, initially in-vented by Reed et al. (1998).

The easiest solution to hiding the identity of a useris the use of a single proxy server. Tra�c is routedthrough a node that strips o� the origin IP addressand presents its own instead. The main problem ofsingle-host proxies is that they are a single point offailure in regard to availability and trust.

The current step in the evolution of anonymousnetworks is Onion Routing, where messages are en-crypted in layers and sent from one node to the next.At each hop one layer of encryption is removed (oradded, depending on the direction) and the result fur-ther forwarded.

Further, users need to distinguish between serviceswhere one entity operates both the anonymisationnodes, and the information service (e.g., Perfect Pri-vacy) and services where nodes can be operated byindependent third parties (e.g., Tor, I2P).

However, independent of the used anonymisationtechnique, users' identities may still be discoveredusing other techniques such as information leakageat the application layer. This can be accomplishedthrough analysis of the HTTP headers or by inter-section attacks, using language or font presets for in-stance as proved by Raymond (2000) andWright et al.(2003). Therefore, either a service to alter HTTPheader information should be provided by proxy ser-vice operators, or it is recommended to use �lteringproxy on the user side before sending the data to theanonymisation network.

This super�cial classi�cation of anonymisationsystems already shows the complexity a user facesdeciding upon an appropriate solution. During thisselection process, several aspects are usually consid-ered. In addition to the most important aspect, thedegree of anonymity and performance plays a large

Proceedings of the Ninth Australasian Information Security Conference (AISC 2011), Perth, Australia

77

role, as do reliability, usability, and economic aspects(see Fig. 1).

Anonymisation Tool

LNA

WSP

PP Jondo (Free) Jondo (Com.) TorI2PFP

ISP

Gov.

LNA

IX

WSP

NO

SO

ISP

Gov.

LNA

IX

WSP

NO

SO

ISP

Gov.

LNA

IX

WSP

NO

SO

ISP

Gov.

LNA

IX

WSP

NO

SO

LNA

WSP

Gov.NOSO

EPEP

EPEP

EPEPEx

posu

re

Gov.NOSOIX IXISP ISP

Performance

Anonymity

Usability

Cost

AnonymisationSystem

Reliability

SO Service Operator

NO Node Operator

Gov. Government / Secret Service

IX

ISP

LNAWSP

Internet Exchagne

Internet Service Provider

Local Network AdministratorWeb Service Provider

EP External Party

Figure 1: Aspects of systems selection

In this paper, we examine all relevant incentivesand combine them to provide usage guidance onalready-deployed anonymisation networks by classify-ing the systems and showing their strength and weak-nesses. Applying this guidance, users can select theanonymisation service that best suits their needs in aconcrete situation.

The remainder of this paper is organized as fol-lows: �rstly, we give an overview of the tools wecompare. Section 2 describes the anonymisation sys-tems we used for comparison, and is followed by anoverview of related work (Section 3). In Section 4,we brie�y examine usability in regard to its impacton acceptance of the tools. Further, we measureand evaluate the performance in terms of round triptime (RTT), Inter-Packet Delay Variation (IPDV),and throughput. We classify anonymisers in regardto their e�ciency in Section 5 before addressing prac-tical issues of anonymity and security in Section 6.In Section 7, we calculate and discuss the aspect ofreliability, before all relevant aspects are combined todescribe the economic impact on a user's decision pro-cess in Section 8. Finally, Section 9 concludes withthe lessons learned and future work.

2 Anonymisation Systems

The simplest way of hiding someone's identity is touse of a proxy server. The receiver of the messageonly gets the IP address of an intermediate server,not of the sender. The main drawback is that adver-saries can easily de-anonymise users by compromis-ing a server or simply providing one. However, thisservice may still assure a basic level of anonymity.Due to the simple setup, proxy servers are very com-mon, either as free or commercial solutions and can beeasily found in the Internet. The providers of theseproxy servers are mostly unknown, so one does notknow how trustworthy they actually are. Commer-cial services exist too, such as Perfect Privacy, whichcurrently provides 48 servers in 23 countries world-wide, allowing users to choose either their preferredproxy or a self-de�ned cascade of proxies, meaningthat several proxy servers are combined into a chain.This may increase anonymity and security against anexternal adversary, but still has the drawback thatthe service as a whole is operated by a single entity.Perfect Privacy o�ers a variety of ways of connecting:users can simply use the servers as an HTTP- SOCKSproxy6. In addition, users can connect to the proxy

6SOCKS is an Internet Protocol to facilitate routing of packetsusing a proxy server

server via OpenVPN, PPTP VPN, or SSH. As longas the user does not use their own layer of encryption,the tra�c from the proxy server to the destination isnot encrypted and consequently completely visible tothe server provider. This is true for all anonymisationtools presented here.

Another low-latency anonymisation approach pro-vides the possibility of active mixing7 of the tra�ctogether with Onion Routing. A popular exampleof using this approach is JonDonym. Started as anopen source project at the TU Dresden, JonDonym(formerly known as JAP) became a popular tool togain anonymity in the Internet. Users can choose be-tween several �xed paths, known as cascades, withnodes provided by JonDonym operators and nodesoperated by other organizations or individuals. Cur-rently, there are 34 nodes in the network forming 16cascades. The operators of JonDonym provide twokinds of service: a free service, usually having twonodes in a cascade with several hundreds users anda commercial service with usually three nodes in acascade. Compared to the free service, the numberof concurrent participants is relatively low (less than100). Even though tra�c mixing is supported in thisapproach, to the best of our knowledge, it is not ac-tivated because of performance issues.

Today's most widely used anonymisation system isTor. Also based on onion routing, Tor tries to providean acceptable degree of anonymity, while allowing theuse of interactive web applications. Recently, Dingle-dine (2009) showed that Tor has about 300,000 usersdaily and about 2,000 relaying nodes. The main dif-ference from JonDonym is its volunteer-based nodeoperation. In order to achieve optimal system per-formance, Tor currently relies on directory servers,which gather all relevant information about the net-work and provide information about the performanceof nodes to the clients.

I2P is a system similar to Tor and JonDonym. Incontrast to JonDonym and Tor, the main objectiveof I2P is communication within its own network andnot with external services. As a consequence, there iscurrently only one outbound HTTP gateway respon-sible for all outgoing web tra�c. Another di�erencefrom Tor and JonDonym is its fully-distributed net-work, which has no centralised server for coordinationand organisation. Hence, the network consists of a setof nodes that communicate with each other in orderto achieve anonymity. All tra�c is encrypted usinggarlic encryption, which combines multiple messagesinto one single message to make tra�c analysis moredi�cult.

3 Related Work

In the many years since the establishment of the Inter-net, network performance has been an extensive �eldof research, showing di�erent issues and optimizationsin a large number of publications, e.g., by Keshav(1999). In recent years, logical networks, also knownas overlay networks, were introduced to allow the easycreation of additional network services without mod-i�cation of the underlying network. These have be-came a popular topic of interest in network researchand shifted several network paradigms to the appli-cation layer. Peer-to-peer networks and other overlaynetwork topologies were introduced to improve dataexchange or to add additional functionality. Amongthese is anonymity, which elicited so much interest,that a special �eld of research, anonymous communi-cation, was established.

Several surveys on anonymous communication sys-tems exist, e.g., conducted by Kelly (2009) or Ren

7actively delaying and batching messages

CRPIT Volume 116 - Information Security 2011

78

et al. (2009). In the work of Pries et al. (2008),in which the authors describe the concepts of basicanonymous communication, as well as implementedsystems, the need of low-latency anonymous commu-nication systems is highlighted. However, most sur-veys focus on MixNet based schemes based on the ap-proach of Chaum (1981) for anonymous remailers andOnion Routing (particularly on Tor); minor work hasbeen conduction on other network routing-based tech-niques like Crowds and P2P networks such as Tarzan,which was developed by Freedman et al. (2002) andMorphMix, an approach by Rennhard et al. (2002).Due to its widespread usage with about 300,000 usersdaily, existing performance measurements in anony-mous communication mainly concentrate on Tor. Themain objective is the improvement of performance, forinstance using alternative methods of path selection.Very often, authors of related publications concen-trate on throughput improvements in Tor and eitherpropose algorithms to achieve higher performance orhigher anonymity as shown by Snader et al. (2008).In contrast, the importance of latency in anonymi-sation networks as performance metric is highlightedby Murdoch et al. (2008). However, both publicationsconsider only a single property, while our study com-bines these with the variance to determine the overallperformance.

Other relevant matters in choosing the appropri-ate anonymisation system are rarely considered. Theoptimal system needs to be reliable, and also has tobe usable and cheap. Economic aspects are coveredby Acquisti et al. (2003) to build a general model inorder to describe the incentives for participation inanonymous networks. This approach was elaboratedby Ngan et al. (2010), going one step further and de-scribing incentives for relaying tra�c within Tor withthe aim of an overall performance improvement.

The work of Dingledine et al. (2006) emphasizesthe usability and the network e�ect in anonymisationnetworks. The authors argue the importance of us-ability to increase the user base and, consequently,on the achievable anonymity. Related to both us-ability and performance is the time needed for send-ing and receiving messages. Even when just sur�ngin the Internet, users expect an appropriate perfor-mance. If these expectations are not met, users willmost likely not use the service. Various studies haveattempted to �nd out the maximum tolerable time forloading a website. Di�erent numbers can be found inliterature, depending on the culture, etc., but recentstudies studies, e.g., by Kopsel (2006) and Wendol-sky et al. (2007) conclude that about four seconds isa maximum tolerable delay for most users.

To the best of our knowledge, to date there hasbeen no practical comparison of all relevant aspects(degree of anonymity, performance, usability, reli-ability, and cost) of already deployed low-latencyanonymisation tools. This paper aims to close thisgap.

4 Usability

As already mentioned, usability is a crucial aspectsince it is essential to attract more users, which isa prerequisite for improving anonymity. The higherthe number of participants, the better the theoreticalanonymity due to the increased size of the anonymityset (as in the work of P�tzmann et al. (2009)). Conse-quently, providers of anonymity services aim to have ahigh number of users, which, which incurs the cost ofa degradation in the system's performance. However,even before evaluating the systems' features, the userinformally evaluates the usability of the anonymisa-tion system during installation and initial con�gura-

tion. This is of particular importance, as she mayalready form a negative opinion of the system andmay reject its further use.

To evaluate usability, we use the cognitive walk-through (CW) method, developed by Wharton et al.(1992). Hereby users try to accomplish tasks withthe aim of identifying usability issues. The particularevaluation was divided into three steps:

1. CW1: Installation of the anonymising software.

2. CW2: Con�guration of the browser/other soft-ware.

3. CW3: Veri�cation of the anonymised connection.

In the following, we describe these steps more in de-tail.

CW1: Installation of the anonymising software

Although some prerequisite software installationsmay be challenging to inexperienced users, all systemsprovide well documented websites to support usersduring the installation process. Very often, step-by-step instructions are given, which vary from a sim-ple double-click (JonDonym) to some more advancedcon�guration being necessary (Tor and I2P).

CW2: Con�guration of the browser/othersoftware

As already mentioned, we tested both free and com-mercial systems. The two commercial systems, Per-fect Privacy and the premium service of JonDonym,have to be paid for in advance. This can by doneby credit card or anonymously by using vouchers (seeSection 6). Thus, the process of paying makes someadditional e�ort necessary, but is relatively easy tohandle.

The aim of Tor is to protect data transport. Forweb browsing, there are no speci�c measures to hidepotentially unmasking information as such as browsertype, language settings, and so forth, which is sent bydefault to the web server. Therefore the developershighly recommend the installation of a local proxyserver that modi�es or deletes this information be-fore sending the data. After the installation of thelocal proxy server, the �nal step is the same for allother tested systems: the users have to con�gure theapplication (in this case, the browser) in order to usea proxy server. Depending on the browser, the step ofproxy con�guration may be di�cult for a less sophisti-cated user the �rst time because of the sometimes notobvious location of these settings. Only Tor simpli�esthis process by installing an add-on (Torbutton) thatthat allows the proxy settings to be easily switchedon and o�.

CW3: Veri�cation of the anonymised connec-tion

Once the user has con�gured the browser or the ad-ditional software, she needs to verify whether theanonymisation service is running properly. On dedi-cated web sites that reveal the IP address of the con-necting user, it is easy to check the system's func-tionality. Some of these web sites8 provide additionalinformation about security/anonymity issues, like theconnecting IP address, HTTP header information andwhether Java/Javascript is turned on in the browser.Except for �nding an appropriate website, this stepwas found to be relatively easy to accomplish.

8E.g., http://test.anonymity.com


79

5 Performance

Probably the most important aspect for users on theInternet, even when acting anonymously, is perfor-mance. In particular, Round Trip Time (RTT), Inter-Packet Delay Variance (IPDV) and throughput havea signi�cant in�uence on the overall performance asperceived by users. Because this has a direct impacton the user's satisfaction, we examine these parame-ters in detail and calculate the overall e�ciency.

5.1 Testbed environment

All measurements described in this section wereperformed between a client (running Ubuntu 10.04,Intel Core2 Duo, 3GHz, connected at 100Mbit/sto the campus network and with 300Mbit/s tothe outside world) located at the University ofLuxembourg and two web servers, one located inLuxembourg and one on St. Vincent Island. Thebasic measurement setup is depicted in Figure 2.In order to allow the comparison of all tools underthe same conditions, we used the HTTP protocol asthe least common denominator supported by all tools.

Client Node A Node B Node C{ {Webserver 1

Webserver 2

Figure 2: Testbed setup with either one proxy or achain of intermediate nodes

For RTT measurements, we chose the Apache webserver benchmarking tool9. It allows measurementof the time a request needs to get about 200 bytesfrom an HTTP server. Even though this approachinvolves a certain overhead, it allows a relativecomparison of the systems. In order to consider time-shifts and varying network usage at di�erent timesof a day, we repeated the measurements over six days.

Measurements of IPDV were conducted everyminute over a period of four days, using a dedicatedclient-server application. We measured the inter-arrival time between every sequence sent with a onesecond interval in between. The main motivationof the following measurements is the question ofwhether it is possible to use applications such asVoIP over the anonymisation systems.

Finally, we measured the throughput for three con-secutive days using GNU wget10. We chose to down-load �les of two sizes (100KB and 1MB) to examinethe interaction between the amount of the transferreddata and the TCP slow start algorithm. We usedthese �le sizes to identify di�erences between smalland large �les based on a recent report that statesthat the average size of a web site is 320KB ( Google(2010)). Thus, we cover cases of both smaller andlarger �les.

All measurements were performed using the al-ready described anonymisation tools, applying thefollowing settings:

• Free proxies (FP) were chosen from a web pagelisting free proxy servers ranked by their perfor-mance11. As these servers typically have a high

9http://httpd.apache.org/docs/2.0/programs/ab.html10http://www.gnu.org/software/wget/11http://proxy.speedtest.at/proxybyPerformance.php?o�set=0

�uctuation, we had to switch between serversduring the test, causing signi�cant downtimes(see Section 7).

• Perfect Privacy (PP) currently o�ers servers at23 locations worldwide. Some locations provideonly a single server, others up to eight for thepurpose of load balancing. We used three ran-domly chosen nodes out of 48, located in Ams-terdam, Moscow, and Chicago.

• JonDonym, using three di�erent random pre-mium service cascades (out of nine), having threenodes each. Measurements were not performedusing free cascades because the user limit is oftenreached and, consequently, the service continu-ally becomes unavailable.

• Tor with its default con�guration, changing cir-cuits at least every 10 minutes.

• I2P, which also changes internal paths every 10minutes, but uses always the same single out-bound server with estimated 1,000 concurrentusers 12.

In addition, we performed the same measurementswithout any anonymisation tool. This informationserves as the reference value to calculate the e�ciencyand performance losses of anonymisation tools.

200 400 600 800 1000

0.0

0.2

0.4

0.6

0.8

1.0

CDF−RTT−Luxembourg

RTT [ms]

CD

F

NoneFree ProxyPerfect Privacy

JondoTorI2P

Figure 3: CDF Luxembourg

5.2 Round Trip Time

Network latency and RTT have a fundamental im-pact on end-to-end performance in computer net-works. Voice over IP (VoIP) applications for instancerequire a RTT of less than 600ms13 to provide ade-quate quality.

Evaluating our measurements, signi�cant di�er-ences were identi�ed: the commercial approaches,Perfect Privacy and JonDonym, show the lowestaverage RTTs together with the free proxy, whileTor and I2P are signi�cantly slower by a factor ofthree to four. The Cumulative Distribution Function(CDF) plots (Fig. 3 and 4) show the fraction ofmeasurements of RTT that are below a certainvalue. Taking Tor and I2P as an example, the testson the server in Luxembourg show that Tor canachieve lower RTTs, but between 550ms and 1s, I2P

12http://stats.i2p.to/13http://www.itu.int/itudoc/itu-t/aap/sg12aap/history/g.114/g114.html


80

200 400 600 800 1000

0.0

0.2

0.4

0.6

0.8

1.0

CDF−RTT−St. Vincent

RTT [ms]

CD

F


JondoTorI2P

Figure 4: CDF St. Vincent

Time

Milli

seco

nds

(ms)

8am 12pm 4pm 8pm 12am 4am 8am

025

0050

0075

0010

000

1500

020

000

(a) Perfect Privacy

Time

Milli

seco

nds

(ms)

025

0075

0012

500

1750

022

500

2750

0

8am 12pm 4pm 8pm 12am 4am 8am

(b) Tor

Figure 5: RTTs measured during one day using Per-fect Privacy(a) and Tor(b).

performs better, meaning that, for instance in 60%of the measurements, Tor showed a RTT of about800ms, while I2P achieved about 720ms. The resultsshow also that VoIP is only possible with PerfectPrivacy (Amsterdam), FP, and JonDonym with somerestrictions, because their RTTs are less than 400msfor at least 80% of the measurements.

During our measurements, Perfect Privacy showsthree distinct levels of RTTs with each level �uctuat-ing in only a narrow band of a few milliseconds (Fig.5(a)). Due to the usage of TCP packets, this patternis most likely created by packet retransmits. Thetimeout of TCP packets on GNU/Linux is 3000msand would explain the very constant additional de-lays. This suggests that there was congestion on thecommunication line or the proxy server. The samebehaviour has been observed on other Perfect Pri-vacy proxy servers as well. Tor instead shows a widervariance of RTT values (see Fig. 5(b)) due to the va-riety of possible circuits. Possible retransmits are noteasily detectable in this plot.

5.3 Inter-Packet Delay Variation

Interactive real-time applications such as VoIP de-pend heavily on a constant IPDV. While multime-dia streaming applications can compensate di�eringIPDV by the use of bu�ers, this is not possible inVoIP. In the sense of Quality of Service of VoIP, IPDVshould be <100ms to avoid distortion14.

Figure 6(a) shows the IPDV observed at both

14http://www.gig-ip.com/help/voip_and_qos_sensors.htm

−20 0 20 40 60 80 100

0.00

0.05

0.10

0.15

0.20

No Anonymisation − Luxembourg and St. Vincent

PDV [ms]

Den

sity

LuxembourgSt. Vincent

−100 −50 0 50 100

0.00

0.02

0.04

0.06

Perfect Privacy (Amsterdam), Jondo, Free Proxy − Luxembourg

PDV [ms]D

ensi

ty

Perfect PrivacyJondoFree proxy

−2000 −1000 0 1000 2000

0e+

001e

−04

2e−

043e

−04

4e−

045e

−04

Tor, I2P − St. Vincent

PDV [ms]

Den

sity

TorI2P

Figure 6: Inter-Packet Delay Variations

servers without anonymisation service. The serverin St. Vincent has a smoother and wider distribution,probably caused by the longer distance between clientand server, compared to the server in Luxembourg.However, Figure 6 suggests that apart from PerfectPrivacy and JonDonym, no other anonymisation ser-vice would be able to comply with the recommendedvalue of IPDV for VoIP applications. The values forTor and I2P are far too high for this kind of communi-cation (Fig. 6(c)), most likely due to congestion. Per-fect Privacy, the free proxy, and JonDonym in partic-ular provide a IPDV of less than 50ms (cf. Fig. 6(b)),and satisfy the requirements for carrying VoIP tra�c.

5.4 Throughput

In order to evaluate application performance withindi�erent scenarios, we measured throughput whiletransferring �les with the sizes of 100KB and 1MB.Due to similarities of the results, only the throughputresults to St. Vincent server are shown here.

The �rst CDF graph (Figure 7(b)) shows thethroughput of an anonymisation system while trans-


81

0 100 200 300 400

0.0

0.2

0.4

0.6

0.8

1.0

Throughput − 100K − St. Vincent

Throughput [KB/s]

CD

F

0 200 400 600 800 1000 1200

0.0

0.2

0.4

0.6

0.8

1.0

Throughput − 1MB − St. Vincent

Throughput [KB/s]

CD

F


JondoTorI2P

Figure 7: CDF Throughput

ferring 1MB of data. I2P shows generally thelowest throughput, while the maximum throughputwas achieved by the Perfect Privacy proxy in Am-sterdam (1044KB/s). This value was even higherthan the throughput measured without anonymisa-tion (746KB/s). This is an astonishing result, be-cause our tests were conducted utilising the HTTPheader option no-cache, so that there should be nocaching on the proxy server. The only explanationcould be the usage of a more powerful connection viaAmsterdam compared to the native connection.

As can be seen in Figure 7(a), the values for trans-mission of 100KB �les were signi�cantly lower, forPerfect Privacy and without anonymisation tool, bya factor of 4 - 4.5. This e�ect may result from TCPslow start, when the hosts involved try to achievethe highest throughput for this particular connectionby adapting TCP window size. In Tor and I2P, thethroughput is almost constant for the di�erent �lesizes. The reason for this behaviour, again, is pre-sumed to be congestion within the network.

5.5 Systems e�ciency

Even though low-latency anonymisation systems ingeneral provide the possibility is being used for in-teractive applications such as sur�ng on the Inter-net, the question remains how they compare to thebehaviour and performance without any anonymisa-tion. In this section, we calculate the e�ciency ofRTT and throughput. We did not include IPDV be-cause as long as the required threshold value is notexceeded (e.g., 100ms for VoIP), the actual value isof no particular interest to the average user.

For the throughput, we calculate the e�ciency(ET) for the transfer of 100KB and 1MB data respec-tively as ratio of the mean throughput (Tmean(ASi))of the corresponding anonymisation system ASito the throughput (Tno-anon) measured withoutanonymisation tool (161KB/s and 734KB/s) using eq.1.

ET (ASi) =Tmean(ASi)

Tno−anon(1)

RTT e�ciency (ERTT) is calculated similarly toET: it is the ratio of the RTT without anonymiza-tion (RTTno_anon) and the mean RTT of ASi(RTTmean(ASi)).

ERTT (ASi) =RTTno−anon

RTTmean(ASi)(2)

To further guide users' decisions, Table 1 lists thee�ciencies of throughput and RTT. The value of 1 isthe reference value, accomplished without anonymi-sation and the higher the value, the more e�cient isthe system.

Apart from I2P, all tools show an acceptable e�-ciency for 100KB �les, but the rates decrease tremen-dously for the large �les. Here, JonDonym and Torshow throughput e�ciencies of less than 30%, I2Ponly 5%.

RTT total e�ciency values are calculated as themean of Luxembourg and St. Vincent. Compared tobandwidth e�ciencies, these values are even worse.While again the Perfect Privacy server in Amsterdamperformed well, all other ERTT values are below 40%.Altogether, these �gures show that current anonymi-sation systems still su�er from poor performance.

Table 1: Throughput and RTT E�ciency

Anonymisation System ET (100KB) ET (1MB) ERTTFree Proxy 1.19 0.88 0.27PP Amsterdam 1.37 1.17 0.89PP Moscow 0.88 0.71 0.37PP Chicago 0.85 0.37 0.40JonDonym 0.75 0.30 0.36Tor 0.55 0.19 0.07I2P 0.18 0.05 0.08

6 Anonymity and Security

Anonymity may be quanti�ed using di�erent metrics,as a survey of Kelly et al. (2008) shows, but none ofwhich is comprehensive. In this section, we establisha thorough classi�cation of anonymity for all testedanonymisation services. Because a quantitative com-parison of all services is di�cult up to impossible (asthere is no existing metric that would consider allpossible attacks) using existing approaches such asentropy, which is described by Diaz et al. (2002), weperformed an educated anonymity/security appraisaland ranked attackers in regard to their ability andcosts to de-anonymise users. This ranking is based onour subjective assessment and may di�er from otherclassi�cations.

The idea is simple: we identi�ed the di�erentroles of adversaries against systems for anonymouscommunication. We then ranked these adversarieswith respect to their power. In order to quantify theanonymity, we ranked the power of an adversary ona scale between 0 and 1. The value of 1 means thatthe adversary can de-anonymise the involved entitieswith a high probability, whereas the value of 0 meansthat the adversary is generally harmless with respect


82

to the considered anonymisation technique. For in-stance, while a web service provider has limited powerto identify a user coming from an anonymisation net-work, an Internet exchange (IX) and a Governmenthas much greater power and abilities. Figure 8 showsthe results of our appraisal.

Anonymization Tool

LNA

WSP

PP Jondo (Free) Jondo (Com.) TorI2PFP

ISP

GOV

LNA

IX

WSP

NO

SO

ISP

GOV

LNA

IX

WSP

NO

SO

ISP

GOV

LNA

IX

WSP

NO

SO

ISP

GOV

LNA

IX

WSP

NO

SO

LNA

WSP

GOVNOSO

EP

EP

EPEP

EPEPPo

wer

GOVNOSOIX IXISP ISP

Performance

Anonymity

Usability

Cost

AnonymisationSystem

Reliability

SO Service Operator

NO Node Operator

GOV Government / Secret Service

IX

ISP

LNAWSP

Internet Exchange

Internet Service Provider

Local Network AdministratorWeb Service Provider

EP External Party

1

0

Figure 8: Classi�cation of anonymisation systems

No single low-latency anonymisation techniquecan provide an adequate protection against an at-tacker having a government or anonymisation serviceoperator status. Therefore, users of anonymisationsystems are required to trust the service operator. Us-ing for instance Tor, users get good protection againstthe Web Service Provider, the Local Network Admin-istrator (LNA), as well as the ISP. This is due tothe encryption used between the sender and the �rstTor node. A node operator and the External Party(EP) have some more power, as they can add as manynodes to the network as they have resources. Here,an External Party is de�ned as an entity outside theanonymisation system, that is trying to become a partof it. Hence, every other entity we consider in our cat-egorization can be seen as an EP too.

An even more powerful attacker is the InternetExchange, as it can observe a considerable amountof tra�c between the Tor nodes. Recent studies,e.g., by Edman et al. (2009) show that there isa certain risk that provider of large AutonomousSystems (AS) can control a signi�cant number ofentry and exit nodes, hence this is also true for thecorresponding Internet Exchange. Service Operatorand Government or Secret Service are the mostpowerful players. They may have enough power tobias path selection, analyse all network tra�c, breakthe encryption, or even apply non-technical means toachieve their goal (e.g., rubber-hose cryptanalysis15).We also di�erentiate between the the two availableversions of JonDonym. While the free versionprovides a path-length of two nodes, the premiumservices always use three nodes. However, we rankthe LNA higher for the premium service due tosimpli�ed �ngerprinting, as proved by Panchenkoet al. (2009) because of a smaller number of users.

Calculating the degree of anonymity using thisclassi�cation, single values are weighted, summed andnormalized:

15Torture of a person to extract cryptographic secrets, e.g., apassword

A = 1−

n∑i=1

(wi ∗ ai)

n∑i=1

wi

, (3)

where ai is the power of attacker i, wi the weightan user puts on the attacker ai and n is the numberof attackers considered (here: eight). Table 2 showsthe degree of anonymity of the tested anonymisationservices for two particular cases. Case 1 (C1) showsthe values without consideration of any user-basedweighting; C2 could be an employee using serviceslike eBay at the workplace. Here, we mainly con-sider the LNA and the WSP as critical, i.e., givingthem a higher weight (LNA: 10, WSP: 3). The otheridentities are weighted 1. Overall, Tor achieves thehighest degree of anonymity and the free proxy theworst. The degree of anonymity of Tor is even higherthan in I2P, mainly given because of the single out-bound node I2P provides. Due to the higher rankingof a LNA, the free version of JonDonym may be moreappropriate in C2.

Table 2: Degree of anonymity

Anonymisation System A (C1) A (C2)Free Proxy 0.18 0.32Perfect Privacy 0.26 0.42JonDonym 0.33 0.54JonDonym 0.36 0.53I2P 0.44 0.62Tor 0.47 0.66

In addition to the degree of anonymity, other as-pects, such as anonymous payment for the use of com-mercial anonymisation tools, are of relevance too, asthey may directly in�uence the anonymity. For exam-ple, providing the real name and/or the bank accountnumber would reveal the identity of the user to thecompany o�ering an anonymisation service.

The commercial service providers in this compari-son, Perfect Privacy and JonDonym, o�er an alterna-tive by also accepting payments by anonymous pay-ment schemes such as PaySafeCard16 or UKash17.Users can anonymously buy a code in an ordinaryshop and to pay for the anonymisation service withthis code as with pre-paid telephone cards, but with-out any personal registration being required. Anotherpossibility for ensuring anonymity during the pay-ment process is the usage of anonymous credit cards,which work either like pre-paid cards or like gift cards.Pre-paid cards need to be charged before usage, giftcards can be bought already containing a certain bal-ance.

Considering the di�erence between free and com-mercial service operators, we cannot preempt theuser's decision to which service is more trustworthy;users have to trust the operator in both cases. Onlythe operator's intention may vary, and range fromcommercially-driven to belief in expression of freedomor the hope of creating a trap to harvest sensitive in-formation.

7 Network Reliability

The next essential aspect, which is particularly im-portant for user satisfaction, is the reliability of thenetwork. We assess it in terms of the failure rate. To

16http://www.paysafecard.com/17http://www.ukash.com


83

calculate the failure rate, all unanswered RTT bench-marking requests were counted during the period ofexperiment execution.

A common parameter to describe the failure rateis MTBF, which expresses the Mean Time BetweenFailures of a system. In this context, MTBF is cal-culated as the sum of the uptime periods divided bythe number of downtimes:

MTBF =Σ(tdown − tup)

ndown(4)

where ndown is the number of failures.

We also calculated the Mean Time To Recovery(MTTR), which is computed in the same way asMTBF (Equation 5). In order to evaluate reliability,both factors need to be considered.

MTTR =Σ(tup − tdown)

nup(5)

Tables 3 and 4 show that the loss of RTT con-nections occurred by the free proxy, which is the re-sult of proxy servers going o�-line periodically fromtime to time. During our experiment, we twice had toswitch to a new proxy server. Another issue is relatedto the connection to the webserver in St. Vincent,which showed problems for two hours when some ofthe packets did not get through. The relatively highpacket loss of JonDonym was caused by a service in-terruption of more than two hours. This interruptiononly a�ected JonDonym tra�c to the server in Lux-embourg while all other services were working well,including the connections to St. Vincent. This resultsuggests that there was a problem of connectivity be-tween the �nal JonDonym relay and the server. How-ever, the numbers only present a snapshot and maynot necessarily re�ect the long-term behaviour.

Table 3: Snapshot of MTBF and MTTR - St. Vincent(SV)

AS PL MTBF (SV) MTTR (SV)None 0.33% 72:54:35 0:00:39Free Proxy 2.22% 0:10:05 0:04:14PPA 0.29% 1:20:56 0:01:04PPM 0.37% 1:05:35 0:01:12PPC 0.29% 1:21:14 0:01:12JonDonym 0.36% 1:04:51 0:01:07Tor 0.36% 0:49:49 0:01:00I2P 0.56% 0:19:22 0:01:23

Table 4: Snapshot of MTBF and MTTR - Luxem-bourg (Lux)

AS PL MTBF (Lux) MTTR (Lux)None 0% 144:23:04 0:00:00Free Proxy 2.06% 0:38:54 0:00:59PPA 0.05% 1:02:06 0:01:00PPM 0.06% 0:05:04 0:01:00PPC 0.02% 2:45:55 0:01:10JonDonym 0.76% 3:44:16 1:05:04Tor 0.06% 1:30:04 0:04:44I2P 0.22% 0:02:04 0:45:00

Fig. 9 illustrates the number of unanswered RTTrequests during the measuring period. A high numberof lost messages without the use of any anonymisationsystem is a sign of a general network problem. How-ever, the �gure also shows the in�uence of path selec-tion on reliability. It is again possible to see a high

None PPA PPM PPC Jondo Tor I2P FP

Packet loss

010

020

030

040

050

060

070

0 St. VincentLuxembourg

Figure 9: Lost packets during 6 days

number of lost packets for FP. This result con�rmsour observations during previous tests.

Another observation from reliability tests is thein�uence of the server location. Excluding the freeproxy due its outages, connections to the server in St.Vincent summed up over all services show a higherloss in total (822) compared to Luxembourg (561).The reason for this issue may be a general networkproblem, not related to any anonymisation service.Normalising these results, the number of unansweredRTTs for all services, except I2P and the free proxyserver, is quite low.

8 Economic aspects

Apart from the technical aspects of �nding the appro-priate anonymisation tool, users have also to decideon the economic value/cost. Some services rely on anactive participation where users pay indirectly (Ci) byproviding, e.g., computational resources. Using I2Pfor instance requires provision of bandwidth and com-putational power in order to use the network, while inTor, users have the choice of either donating resourcesby acting as a relay node or as a client only. Payingindirect costs may be negligible in most cases, butmay, on the other hand, limit the maximum achiev-able performance. We calculate Ci as the sum of Cr,Ce and Cb, where Cr is the relaying/routing costs, Cecosts of de- or encryption (computational e�ort) andCb the costs of providing additional bandwidth.

Ci = Σ(Cr, Ce, Cb) (6)

Using commercial anonymisation systems, usershave to pay fees. These direct costs Cd are basedon usage time or data volume. For instance the busi-ness model of Perfect Privacy is based on a monthlyfee, o�ering a data �at rate. JonDonym instead billsaccording to the amount of data. Consequently, theoverall costs are calculated as a sum of the two costs:

C = (Ci + Cd) (7)

The payo� costs for every tool are then calculatedas follows:

Pi =waA ∗ weE ∗ wuU

wrR− wcC (8)

where A is the degree of anonymity, E the e�-ciency, R the reliability and U the usability; w{a,e,u,r,c}


84

are the di�erent weights the user de�nes dependingon her particular needs. Consequently, the calcula-tion strongly depends on user requirements and hasto take into account the actual situation.

At �rst glance, users main goal may be a highdegree of anonymity together with a high e�ciencyand reliability of the system at low cost. This doesnot mean that users will not pay for such a service;statistics for Perfect Privacy show about 25,000 con-current connections. Even though this might be notthe accurate statistic, it still shows that a fast andreliable service can attract many users willing to paya certain amount of money. Another example is Tor,with about 300,000 daily users. Tor is also known forits high anonymity and reliability, even at no cost,but with the drawback of poor performance. For thelaunch of future anonymisation systems, especially forcommercial products, the operators need to take allthese aspects into consideration.

9 Conclusion and future work

In this paper, we have de�ned a set of metrics in orderto compare and evaluate �ve already deployed state-of-the-art anonymous communication systems in re-gard to their anonymity, performance, reliability, us-ability, and cost.

Besides the installation, which is relatively simplefor all systems, the usability of the tools is generallygood and should not be a reason for non-use. Usabil-ity does not vary not much from the users' point ofview; they always have to con�gure their application,i.e., web browser, to use a proxy server, a processwhich is practically the same for all tools.

In order to provide a comprehensive comparisonof the anonymisation tools, we ranked them in regardto the power of possible adversaries. Our classi�ca-tion is subjective and may vary from other opinions,but gives users an indication of strength and weak-nesses of corresponding anonymisers. A future goalmight be to simplify the presentation of the resultsand present them to users in a more appropriate way.This will be subject of further work. In addition, wemeasured throughput, RTT, IPDV and failure rateand calculated e�ciencies. The results show that theproxy-based anonymisation systems outperform theOnion Routing approaches in throughput and RTT,but provide less anonymity. This trade-o� appliesto all systems and in the end, the user must decidewhich system best �ts her requirements. However,web browsing is possible using all the tested tools,even though I2P in particular has long response times.Latency-critical applications like VoIP, which rely onhighly responsive networks, are only usable to a cer-tain extent with the the systems we examined.

An important �nding is the e�ciency of thethroughput performance of single proxy solutions.They perform as well the native communication,sometimes even better.

We also observed that the selected anonymisationpath and the recipient's location have a strong in-�uence on performance and reliability. The PerfectPrivacy proxy in Amsterdam, which outperformedthe communication without anonymisation, demon-strates this. In general, all Perfect Privacy proxies wetested, as well as Tor, showed particularly reliabilityin terms of successful connections. While JonDonymand I2P were slightly less reliable, the most unreliableservice was the free proxy service due to a high �uctu-ation of nodes. This demonstrates the main problemwith single-proxy solutions but makes them applica-ble to high-performance short term downloads.

Economics in anonymity is still an under-investigated �eld of research, with only a few publica-

tions. In this paper, we show that besides real costsin terms of money, all relevant aspects such as per-formance, anonymity, reliability, usability, and cost,need to be evaluated in order to calculate system'spayo�. However, as already mentioned, this calcula-tion strongly depends on individual users' preferences.

To summarise our results, we established a com-parison table, containing all examined anonymisationsystems. We classi�ed the systems in �ve groups,expressed on a scale of very good (++), good (+),average(0 ), bad(-) and very bad(- -). Table 5 showsthis classi�cation.

Table 5: Evaluation of anonymous communicationsystems

Anonymisation system V A E R CFree Proxy 0 - - + - - ++Perfect Privacy 0 0 + + -JonDonym 0 0 - + -Tor 0 ++ - - ++ ++I2P 0 + - - + ++

U = Usability, A = Anonymity, E = E�ciency,R = Reliability and C = Cost.

Overall, Tor shows the best results, followed byI2P. They score well in all disciplines except perfor-mance, which is their main weak point. Here, singleproxy solutions score with the best performance.Unfortunately, their degree of anonymity is poor andadditionally reliability leaves much to be desired.JonDonym performs averagely, showing no particularstrength or weakness. However, it is very di�cult toconsider all users' requirements and it is �nally upto them to evaluate the results in order to �nd themost appropriate solution.

To conclude, future work will be necessary in thefollowing areas:

• Extending the usability evaluations by also in-volving less sophisticated users,

• Further investigation of the very high throughputvia certain anonymisation paths,

• Economic aspects need to be evaluated in moredetail, especially in regard to business purposes,

• It may be worthwhile to include social aspectsinto the proposed payo� function, e.g., particulargroup behaviour in an anonymous network.

Overall, this comparison shows the need and mo-tivation to spend further e�ort on the improvementof existing anonymisation services or to work onalternative solutions.

Acknowledgement. This work has been par-tially supported by EC FP7 EFIPSANS project(INFSO-ICT-215549). Furthermore, we would like tothank Dominic for his extensive proof reading.

References

Acquisti, A., Dingledine, R. & Syverson, P. (2003),On the Economics of Anonymity, in R.N. Wright,eds, `Financial Cryptography', Springer LectureNotes in Computer Science, pp. 84�102.

Chaum, D.L. (1981), Untraceable electronic mail, re-turn addresses, and digital pseudonyms , in `ACMJournal of Communication', ACM, pp. 84�90.


85

Danezis, G., Dingledine, R. & Mathewson, N. (2003),Mixminion: Design of a Type III Anonymous Re-mailer Protocol , in `Proceedings of the 2003 IEEESymposium on Security and Privacy', IEEE 'Com-puter Society, pp. 2-15.

Díaz, C., Seys, S., Claessens, J. & Preneel, B. (2002),Towards Measuring Anonymity in '`R. Dingledine& P. Syverson, eds, `Privacy Enhancing Technolo-gies', Springer Lecture Notes in Computer Science,pp. 54�68.

Dingledine, R., Mathewson, N. & Syverson, P. (2004),Tor: the second-generation onion router in 'Pro-ceedings of the 13th conference on USENIX Secu-rity Symposium - Volume 13', USENIX Associa-tion, San Diego, CA, p. 21.

Dingledine, R. & Mathewson, N. (2006), AnonymityLoves Company: Usability and the Network E�ectin 'Proceedings of the Fifth Workshop on the Eco-nomics of Information Security (Weis 2006)'.

Dingledine, R. (2009), Tor and circumvention:Lessons learned at '26th Chaos CommunicationCongress', Berlin, Germany.

Edman, M. & Syverson, P. (2009), As-awareness inTor path selection , in `Proceedings of the 16thACM conference on Computer and communicationssecurit (CSS '09)', ACM, pp. 380�389.

Freedman, M.J. & Morris, R. (2002), Tarzan: apeer-to-peer anonymizing network layer , in `Pro-ceedings of the 9th ACM conference on computerand communications security (CCS '02)', ACM, pp.193�206.

Google (2010), Web metrics:Size and number of resources,http://code.google.com/speed/articles/web-metrics.html,[Online; last accessed 2010-10-01].

Kelly, D.J., Raines, R.A., Grimaila, M.R., Bald-win, R.O. & Mullins, B.E. (2008), A survey ofstate-of-the-art in anonymity metrics , in `Proceed-ings of the 1st ACM workshop on network dataanonymization - NDA '08', ACM Press, p. 31.

Kelly, D. (2009), A taxonomy for and analysis ofanonymous communications networks, Ph.D., AirForce Institute of Technology.

Keshav, S. (1999), On individual and aggregate TCPperformance , in `Proceedings of Seventh Inter-national Conference on Network Protocols', IEEEComputer Society Press, pp. 203�212.

Köpsell, S. (2006), Low Latency Anonymous Commu-nication How Long Are Users Willing to Wait? , in`Journal of Emerging Trends in Information andCommunication', Volume 3995 Springer, LectureNotes in Computer Science, Berlin / Heidelberg,pp. 221�237.

Möller, U., Cottrell, L., Palfrader, P. & Sassamanm L.(2003), Mixmaster Protocol � Version 2, in 'IETFInternet Draft', IETF.

Murdoch S.J. & Watson, R.N.M. (2008), Metricsfor Security and Performance in Low-LatencyAnonymity Systems, in `N. Borisov & I. Goldberg,eds, `Privacy Enhancing Technologies', SpringerLecture Notes in Computer Science, pp. 115�132.

Ngan, T.-W., Dingledine, R. & Wallach, D.S. (2010),Building Incentives into To, in R. Sion, ed, `Fi-nancial Cryptography', Springer Lecture Notes inComputer Science, pp. 238�256.

Panchenko, A., Herrmann, D., Wendolsky, R. & Fed-errath, H. (2009), Website �ngerprinting: attackingpopular privacy enhancing technologies with themultinomial naïve-bayes classi�er, in `Proceedingsof the 2009 ACM workshop on Cloud computingsecurity', ACM, Chicago, Il, USA, pp. 31�42.

P�tzmann, A. & Hansen, M. (2009), A termi-nology for talking about privacy by data mini-mization: Anonymity, Unlinkability, Undetectabil-ity, Unobservability, Pseudonymity, and Iden-tity Management , Online: 'http://dud.inf.tu-dresden.de/Anon_Terminology.shtml'.

Pries, R., Yu, W., Graham, S. & Fu, X. (2008),On performance bottleneck of anonymous commu-nication networks , in `22nd IEEE InternationalSymposium on Parallel and Distributed Process-ing, IPDPS 2008', IEEE Computer Society Press,Miami, Florida USA, pp. 1�11.

Raymond, J.-F. (2000), Tra�c Analysis: Protocols,Attacks, Design Issues, and Open Problems , in`Proceedings of Designing Privacy Enhancing Tech-nologies: Workshop on Design Issues in Anonymityand Unobservability', Springer Verlag, pp. 10-29.

Reed, M.G., Syverson, P.F. & Goldschlag, D.M.(1998), Anonymous connections and onion routing, in `IEEE Journal on Selected Areas in Communi-cations', IEEE Computer Society, pp. 482�494.

Ren J. & Wu, J. (2009), Survey on anonymous com-munications in computer networks, in `ComputerCommunications' Volume 33, Issue 4, pp. 420-431.

Rennhard, M. & Plattner, B. (2002), IntroducingMorphMix: peer-to-peer based anonymous Inter-net usage with collusion detection, in `Proceedingsof the 2002 ACM workshop on Privacy in the Elec-tronic Society (WPES '02)', ACM, pp. 91�102.

Snader, R. & Borisov, N. (2008), A Tune-up for Tor:Improving Security and Performance in the TorNetwork , in `Proceedings of the Network and Dis-tributed Security Symposium - NDSS '08', InternetSociety.

Wendolsky, R., Herrmann, D. & Federrath, H.(2007), Performance Comparison of Low-LatencyAnonymisation Services from a User Perspective, in`', N. Borisov & P. Golle, eds, `Privacy EnhancingTechnologies', Springer Lecture Notes in ComputerScience, pp. 233�253.

Wharton, C., Bradford, J., Je�ries, R. & Franzke, M.(1992), Applying cognitive walkthroughs to morecomplex user interfaces: experiences, issues, andrecommendations, in `Proceedings of the SIGCHIconference on Human factors in computing sys-tems', ACM Press, Monterey, CA, pp. 381�388.

Wright, M., Adler, M, Levine, B. N. & Shields,C. (2003), Defending Anonymous CommunicationAgainst Passive Logging Attacks , in `Proceedingsof the 2003 IEEE Symposium on Security and Pri-vacy', IEEE Computer Society, pp. 28�.


86