Modeling Network Diversity for Evaluating theRobustness of Networks against Zero-Day Attacks

Lingyu Wang1, Mengyuan Zhang1, Sushil Jajodia2, Anoop Singhal3, andMassimiliano Albanese2

1 Concordia Institute for Information Systems Engineering, Concordia University{wang,mengy zh}@ciise.concordia.ca

2 Center for Secure Information Systems, George Mason University{jajodia,malbanes}@gmu.edu

3 Computer Security Division, National Institute of Standards and Technologyanoop.singhal@nist.gov

Abstract. The interest in diversity as a security mechanism has recently beenrevived in various applications, such as Moving Target Defense (MTD), resist-ing worms in sensor networks, and improving the robustness of network routing.However, most existing efforts on formally modeling diversity have focused on asingle system running diverse software replicas or variants. At a higher abstrac-tion level, as a global property of the entire network, diversity and its impact onsecurity have received limited attention. In this paper, we take the first step to-wards formally modeling network diversity as a security metric for evaluatingthe robustness of networks against potential zero day attacks. Specifically, wefirst devise a biodiversity-inspired metric based on the effective number of dis-tinct resources. We then propose two complementary diversity metrics, based onthe least and the average attacking efforts, respectively. Finally, we evaluate ouralgorithm and metrics through simulation.

Keywords: Security Metrics, Diversity, Network Security, Zero Day Attack, Net-work Robustness

1 Introduction

Computer networks are playing the role of nerve systems in many critical infrastruc-tures, governmental and military organizations, and enterprises. Protecting such missioncritical networks means more than just patching known vulnerabilities and deployingfirewalls or IDSs. Evaluating the network’s robustness against potential zero day attacks(i.e., attacks exploiting previously unknown vulnerabilities) is equally important. Thefact that Stuxnet employs four different zero day vulnerabilities [10] has clearly demon-strated the real-world significance of defending networks against zero day attacks. Onthe other hand, dealing with unknown vulnerabilities is clearly a challenging task.

In a slightly different context, software diversity has previously been regarded as asecurity mechanism for improving the robustness of a software system [24] (a more de-tailed review of related work will be given later in Section 6). By comparing outputs [8]or behaviors [13] of multiple software replicas or variants with diverse implementationdetails, security attacks may be detected and tolerated as Byzantine faults [7]. Although

the earlier diversity-by-design approaches are usually regarded as impractical due tothe implied development and deployment cost, recent works show more promising re-sults on employing either opportunistic diversity already existing among different soft-ware systems [14], or automatically generated diversity, e.g., through randomization ofaddress space [4, 33], instruction set [20], or data space [5]. More recently, diversityhas found new applications in Moving Target Defense (MTD) [19], resisting sensorworms [39], and improving the robustness of network routing [6].

However, at a higher abstraction level, as a global property of the entire network,the concept of network diversity and its impact on security has received less attention.In this paper, we take the first step towards formally modeling network diversity as asecurity metric, for the purpose of evaluating a network’s robustness with respect tozero day attacks. More specifically, following the discussion of several use cases, wepropose a series of network diversity metrics as follows.

– First, we propose a network diversity metric function by adapting well known math-ematical models of biodiversity in ecology. The metric is based on the number ofdistinct resources in a network, while considering the uneven distribution of re-sources and similarity between different resources. This first metric is more suitablefor evaluating the scale of potential infection by a malware, and it is also a buildingblock of the other two metrics. The main limitation is that it ignores potential causalrelationships between resources in a network.

– Second, we design a network diversity metric based on the least attacking effortrequired for compromising critical assets, while taking into account the causal re-lationships between resources. We also study the complexity and design heuristicalgorithms for computing the metric efficiently. This second metric is suitable formeasuring a network’s capability of resisting intrusions or malware that employmultiple related zero day attacks. The main limitation is that, by focusing on theleast attacking effort, it only provides a partial picture about the threat and cannotreflect the average attacking effort.

– Third, we devise a Bayesian network-based model to define network diversity as aconditional probability based on the effect of diversity on the average attack like-lihood. This probabilistic metric provides a complementary measure to the abovemetric by depicting the average attacking effort required by attackers. We showhow this metric can be instantiated from existing standard vulnerability databases.

– Finally, we evaluate the proposed heuristic algorithm and metrics through simula-tion results, and we discuss practical limitations of our approach.

The main contribution of this paper is twofold. First, to the best of our knowledge,this is the first effort on formally modeling network diversity as a security metric fordefending networks against zero day attacks. Second, the modeling effort not only im-proves understanding of the network diversity concept, but may lead to better, quantita-tive approaches to employing diversity for improving network security.

The rest of this paper is organized as follows. Section 2 describes use cases anddefines a biodiversity-inspired metric. Section 3 then presents the least attacking effort-based metric. Section 4 presents the probabilistic diversity metric. Section 5 presentssimulation results. Section 6 reviews related work, and finally Section 7 discusses mainlimitations of this work and concludes the paper.

2

2 Preliminaries

This section motivates the study through several use cases and defines a biodiversity-inspired network diversity metric.

2.1 Use Cases

Use Case 1: Stuxnet and SCADA Security. Stuxnet is one of the first malware thatemploy multiple (four) different zero day attacks [10], which clearly indicates, in amission critical system, such as supervisory control and data acquisition (SCADA) inthis case, the risk of zero day attacks and multiple unknown vulnerabilities is veryreal. Therefore, it is important to provide network administrators a systematic way forevaluating such a risk. On the other hand, this is clearly a challenging task due to thelack of prior knowledge about vulnerabilities or attacking methods.

We next take a closer look at Stuxnet’s attack strategies to illustrate how a net-work diversity metric may help here. Stuxnet targets the programmable logic con-trollers (PLCs) on control systems of gas pipelines or power plants [10]. Such PLCs aremostly programmed using Windows machines not connected to the network. Therefore,Stuxnet adopts a multi-stage approach, by first infecting Windows machines owned bythird parties (e.g., a contractor or insider), next spreading to the organization’s Win-dows machines through the LAN, and finally covering the last hop to targeted machines,which are disconnected from the LAN, through removable flash drives [10].

Clearly, a sufficient presence of vulnerable Windows machines inside the organiza-tion is a necessary condition for Stuxnet to propagate and eventually infect the targetedPLCs. On the other hand, the degree of software diversity along potential attack pathsleading from the network perimeter to the PLCs can be regarded as a critical metric ofthe network’s capability of resisting a threat like Stuxnet. Our objective in this paper isto provide a rigorous study of such diversity metrics.

Use Case 2: Worm Propagation. To make our discussion more concrete, we will referto the running example shown in Figure 1 from now on. Suppose our main concern isthe potential propagation of worms or bots inside a network. A common belief is thatthe more diversified the network is, the higher degree of robustness it will have againsta worm propagation. In other words, we can just count the number (percentage) ofdifferent resources inside the network, and use that count as a diversity metric. Althoughsuch a definition of diversity is natural and intuitive, it clearly has limitations.

For example, in Figure 1, suppose host 1, 2 and 3 are all Web servers running IIS,and host 4 a storage server. Clearly, the above count-based diversity metric will indicatea lack of diversity among the three Web servers and suggest replacing IIS with othersoftware, such that a worm will unlikely infect all three. However, assuming all threeWeb servers would read from a network share on the storage server (host 4), then even-tually a worm can still propagate to all four hosts through the network share, even if itcannot infect all three Web servers directly.

The lesson here is, a naive approach, such as counting the number of resources ina network, may produce misleading results because it ignores the causal relationshipsbetween resources. Therefore, after we discuss the count-based metric in Section 2.2,we will address this limitation with a goal oriented approach in Section 3.

3

host0

host1

host2

firewall1 firewall2

host3

host4

Fig. 1. The Running Example

Use Case 3: Targeted Attack. Suppose now we are more concerned with a targetedattack on the storage server, host 4, launched by human attackers. Following above dis-cussions, an intuitive solution is to diversify resources along any path leading to thecritical asset (host 4), for example, between hosts 1 (or 2, 3) and host 4. Although thisis a valid observation, realizing it will demand a rigorous study of the causal relation-ships between different resources, because host 4 is only as secure as the weakest path(representing the least attacking effort) leading to it. We will propose a formal metricand corresponding algorithm based on such an intuition in Section 3.

On the other hand, the least attacking effort by itself is not sufficient. Suppose nowhost 1 and 2 are diversified to run IIS and Apache, respectively, and firewall 2 will onlyallow host 1 and 2 to reach host 4. Although the least attacking effort has not changed,this diversification effort has provided attackers more opportunities to reach host 4 (byexploiting either IIS or Apache). That is, misplaced diversity may hurt security, whichwill be captured by a probabilistic metric we will introduce in Section 4.

Use Case 4: MTD through Combinations of Web, Application, and Database Servers.In this case, suppose host 1 and 2 are Web servers, host 3 an application server, and host4 a database server. A Moving Target Defense (MTD) approach attempts to achievebetter security by varying in time the software components at those three tiers [19].A common misconception here is that the combination of different components at thethree tiers will increase diversity, and the degree of diversity is equal to the productof diversity at those three tiers. However, this is usually not the case. For example, asingle flaw in the application server (host 3) may result in a SQL injection that compro-mises the database server (host 4) and consequently leaks the root user’s password. Inaddition, diversity over time may actually provide attackers more opportunities to findflaws. The lesson here is again that, an intuitive observation may be misleading, andformally modeling network diversity is necessary.

2.2 Biodiversity-Inspired Metrics

Although the modeling of network diversity has attracted only limited attention, a cor-responding concept in ecology, biodiversity, and its positive impact on the ecosystem’sstability has been investigated for many decades [9]. While many lessons may poten-

4

tially be borrowed from the rich literature of biodiversity, we focus on adapting existingmathematical models of biodiversity to the modeling of network diversity in this paper.

The number of different species in an ecosystem is known as species richness [30].Similarly, given a set of distinct resource types (we will consider similarity betweenresources later) R in a network, we call the cardinality | R | the richness of resourcesin the network. An obvious limitation of this richness metric is that it ignores the rel-ative abundance of each resource type. For example, the two sets {r1, r1, r2, r2} and{r1, r2, r2, r2} have the same richness of 2 but clearly different levels of diversity.

To address this limitation, the Shannon-Wiener index, which is essentially the Shan-non entropy using natural logarithm, is used as a diversity index to group all systemswith the same level of diversity, and the exponential of the diversity index is regarded asthe effective number metric [16]. The effective number basically allows measuring di-versity in terms of the number of equally-common species, even if in reality all speciesare not equally common. In the following, we borrow this concept to define the effectiveresource richness and our first diversity metric.

Definition 1 (Effective Richness and d1-Diversity). In a network G composed of aset of hosts H = {h1, h2, . . . , hn}, a set of resource types R = {r1, r2, . . . , rm}, andthe resource mapping res(.) : H → 2R, let t =

∑ni=1 | res(hi) | (total number of

resource instances), and let pj =|{hi:rj∈res(hi)}|

t (1 ≤ i ≤ n, 1 ≤ j ≤ m) (relative

frequency of each resource). We define the network’s diversity as d1 = r(G)t , where

r(G) is the the network’s effective richness of resources, defined as

r(G) =1

∏n1 p

pi

i

One limitation of the effective number-based metric is that similarity between dif-ferent resource types is not taken into account and all resource types are assumed tobe entirely different, which is not realistic (e.g., the same application can be config-ured to fulfill totally different roles, such as NGinx as a reverse proxy or a web server,respectively, in which case these should be regarded as different resources with highsimilarity). To remove this limitation, we borrow the similarity-sensitive biodiversitymetric recently introduced in [22] to re-define resource richness. With this new defini-tion, the above diversity metric d1 can now handle similarity between resources.

Definition 2 (Similarity-Sensitive Richness). In Definition 1, suppose a similarityfunction is given as z(.) : [1,m] × [1,m] → [0, 1] (a larger value denoting highersimilarity and z(i, i) = 1 for all 1 ≤ i ≤ m), let zpi =

∑mj=1 z(i, j)pj . We define the

network’s effective richness of resources, considering the similarity function, as

r(G) =1

∏n1 zp

pi

i

Note that we will simply use “the number of distinct resources” to refer to therichness of resources from now on. It is to be understood that such a term can always bereplaced with the effective richness concepts given in Definition 1 and 2 to handle theuneven distribution of different resource types and the similarity between resources; inother words, these are not limitations of our models.

5

3 Network Diversity Based on Least Attacking Effort

This section models network diversity based on the least attacking effort. Section 3.1defines the metric, and Section 3.2 discusses the complexity and algorithm.

3.1 The Model

In order to model diversity based on the least attacking effort while considering causalrelationships between different resources, we first need a model of such relationshipsand possible zero day attacks. Our model is similar to the attack graph model [32, 2],although our model focuses on remotely accessible resources (e.g., services or appli-cations that are reachable from other hosts in the network), which will be regarded asplaceholders for potential zero day vulnerabilities, instead of known vulnerabilities (wewill discuss how to integrate known vulnerabilities into our model in Section 4). Tobuild intuitions, we revisit Figure 1 by making following assumptions:

– Accesses from outside firewall 1 are allowed to host 1 but blocked to host 2;– Accesses from host 1 or 2 are allowed to host 3 but blocked to host 4 by firewall 2;– Hosts 1 and 2 provide http service;– Host 3 provides ssh service;– Host 4 provides both http service and rsh service;

Figure 2 depicts a corresponding resource graph model, which is syntacticallyequivalent to an attack graph, but models zero day attacks rather than known vulnera-bilities. Each pair in plaintext is a self-explanatory security-related condition (e.g., con-nectivity 〈source, destination〉 or privilege 〈privilege, host〉), and each triple insidea box is a potential exploit of resource 〈resource, source host, destination host〉; theedges point from the pre-conditions to a zero day exploit (e.g., from 〈0, 1〉 and 〈user, 0〉to 〈http, 0, 1〉), and from that exploit to its post-conditions (e.g., from 〈http, 0, 1〉 to〈user, 1〉). Note we have omitted exploits or conditions involving firewall 2 for sim-plicity. We simply regard resources of different types as entirely different (their simi-larity can be handled using the effective resource richness given in Definition 2). Also,we take the conservative approach of considering all resources (services and firewalls)to be potentially vulnerable to zero day attacks. Definition 3 formally introduces theconcept of resource graph.

Definition 3 (Resource Graph). Given a network composed of a set of hosts H , a setof resources R with the resource mapping res(.) : H → 2R, a set of zero day exploitsE = {〈r, hs, hd〉 | hs ∈ H,hd ∈ H, r ∈ res(hd)} and the collection of their pre-and post-conditions C, a resource graph is a directed graph G(E ∪C,Rr ∪Ri) whereRr ⊆ C × E and Ri ⊆ E × C are the pre- and post-condition relations, respectively.

Next we consider how attackers may potentially attack a critical network asset, mod-eled as a goal condition, with the least effort. In Figure 2, we follow the simple rule thatany zero day exploit may be executed if all its pre-conditions are satisfied, and execut-ing the exploit will cause all its post-conditions to be satisfied. We may then observe sixattack paths as shown in Table 1 (the second and third columns can be ignored for now

6

<http,0,1>

<0,1> <user,0> <0,F>

<firewall,0,F>

<ssh,1,4> <http,0,2>

<2,4><user,2>

<user,4> <4,5>

<http,1,2>

<user,1><1,4> <0,2><1,2>

<rsh,4,5> <http,4,5>

<ssh,2,4>

<user,5>

Fig. 2. An Example Resource Graph

and will be explained shortly). Intuitively, each attack path is a sequence of exploitswhose pre-conditions are all satisfied, either initially, or as post-conditions of precedingexploits in the same path. Definition 4 formally introduces the concept of attack path.

Attack Path # of Steps # of Resources1. 〈http, 0, 1〉 → 〈ssh, 1, 4〉 → 〈rsh, 4, 5〉 3 32. 〈http, 0, 1〉 → 〈ssh, 1, 4〉 → 〈http, 4, 5〉 3 23. 〈http, 0, 1〉 → 〈http, 1, 2〉 → 〈ssh, 2, 4〉 → 〈rsh, 4, 5〉 4 34. 〈http, 0, 1〉 → 〈http, 1, 2〉 → 〈ssh, 2, 4〉 → 〈http, 4, 5〉 4 25. 〈firewall, 0, F 〉 → 〈http, 0, 2〉 → 〈ssh, 2, 4〉 → 〈rsh, 4, 5〉 4 46. 〈firewall, 0, F 〉 → 〈http, 0, 2〉 → 〈ssh, 2, 4〉 → 〈http, 4, 5〉 4 3

Table 1. Attack Paths

Definition 4 (Attack Path). Given a resource graph G(E∪C,Rr∪Ri), we call CI ={c : c ∈ C, (�e ∈ E)(〈e, c〉 ∈ Ri)} the set of initial conditions. Any sequence of zeroday exploits e1, e2, . . . , en is called an attack path in G, if (∀i ∈ [1, n])(〈c, ei〉 ∈ Rr →(c ∈ Ci ∨ (∃j ∈ [1, i − 1])(〈ej , c〉 ∈ Ri))), and for any c ∈ C, we use seq(c) for theset of attack paths {e1, e2, . . . , en : 〈en, c〉 ∈ Ri}.

We are now ready to consider how diversity should be defined based on the leastattacking effort, which intuitively corresponds to the shortest path. However, there areactually several possible ways for choosing such shortest paths and for defining themetric, as we will illustrate through our running example in the following.

7

– First, as shown in the second column of Table 1, path 1 and 2 are the shortest interms of the steps (i.e., the number of zero day exploits). Clearly, the shortest path interms of steps does not reflect the least attacking effort, since path 4 may actuallytake less effort than path 1, as attackers may reuse their exploit code, tools, andskills while exploiting the same http service on three different hosts.

– Next, as shown in the third column, path 2 and 4 are the shortest in terms of thenumber of distinct resources 1. This option is more reasonable than the above one,since it takes into consideration the saved effort in reusing the same exploits. How-ever, although both path 2 and 4 have the same number of distinct resources (2),they clearly do not reflect the same diversity.

– Another attractive option is to base on the minimum ratio # of resources# of steps (which is

given by path 4 in this example), since such a ratio reflects the potential improve-ments in terms of diversity (e.g., the ratio 2

4 of path 4 indicates there is 50% poten-tial improvement in diversity). However, although not shown in this example, wecan easily imagine a very long attack path minimizing such a ratio (e.g., an attackpath with 9 steps and 3 distinct resources will yield a ratio of 1

3 , less than that ofpath 4) but does not reflect the least attacking effort (e.g., the aforementioned attackpath will require more effort than path 4 since it has more distinct resources).

– Finally, yet another option is to pick the shortest path that minimizes both the num-ber of distinct resources (path 2 and 4) and the above ratio # of resources

# of steps (path 4).While this may seem to be the most reasonable choice, a closer look will revealthat, although path 4 does represent the least attacking effort, it does not representthe maximum amount of potential improvement in diversity, because once we startto diversify path 4, the shortest path may change to be path 1 or 2.

Based on these discussions, we define the network diversity by combining the firsttwo options above. Specifically, the network diversity is defined as the ratio betweenthe minimum number of distinct resources on a path and the minimum number of stepson a path (note these can be different paths). Going back to our running example above,we find path 2 and 4 to have the minimum number of distinct resources (two), and alsopath 1 and 2 to have the minimum number of steps (three), so the network diversityin this example is equal to 2

3 (note that it is a simple fact that this ratio will neverexceed 1). Intuitively, the numerator 2 denotes the network’s current level of robustnessagainst zero day exploits (no more than 2 different attacks) , whereas the denominator3 denotes the network’s maximum potential of robustness (tolerating no more than 3different attacks) by increasing the amount of diversity (from 2

3 to 1). More formally, weintroduce our second network diversity metric in Definition 5 (note that, for simplicity,we only consider a single goal condition for representing the given critical asset, whichis not a limitation since multiple goal conditions can be easily handled through addinga few dummy conditions [1]).

Definition 5 (d2-Diversity). Given a resource graph G(E ∪ C,Rr ∪ Ri) and a goalcondition cg ∈ C, for each c ∈ C and q ∈ seq(c), denote R(q) for {r : r ∈

1 Note that, although we will refer to the number of distinct resources for simplicity, it is to beunderstood that this can be replaced by the effective richness concept in Definition 2.

8

R, r appears in q}, the network diversity is defined as (where min(.) returns the mini-mum value in a set)

d2 =minq∈seq(cg) | R(q) |minq′∈seq(cg) | q′ |

3.2 The Complexity and Algorithm

Since the problem of finding the shortest paths (in terms of the number of exploits) inan attack graph (which is syntactically equivalent to a resource graph) is known to beintractable [32], not surprisingly, the problem of determining the network diversity d2 isalso intractable, as stated in Theorem 1. However, we note that, for a specific network,the two problems are not necessarily comparable in terms of their relative hardness. Forexample, in a network with all resources being distinct, it is trivial that d2 = 1, whereasthe shortest paths (in terms of the number of steps) may not be easy to find. On the otherhand, the proof of Theorem 1 is based on special cases where finding the shortest pathsis trivial, whereas determining the network diversity is still intractable.

Theorem 1. Given a resource graph G(E ∪ C,Rr ∪ Ri), determining the networkdiversity d2 is NP-hard.

Proof: The NP-complete Minimum Set Covering (MSC) problem [15] can be reducedto this problem through a construction similar to that in [40]. Specifically, the MSCproblem is to determine that, given a finite set S = {c1, c2, . . . , cn} and a collectionSC = {r1, r2, . . . , rm} where ri ⊆ S(1 ≤ i ≤ m), whether there exists a minimumSC ′ ⊆ SC satisfying that every ci ∈ S is a member of some rj ∈ SC ′. For anygiven MSC instance, we construct a special resource graph G(E ∪ C,Rr ∪ Ri) inwhich we let C = S ∪ {s, d}, where s denotes an initial condition and d the goalcondition, and whenever ci ∈ rj is true, we create an exploit that involves resourcerj , with pre-condition ci−1 (or s for i = 1) and post-condition ci. Finally, we add anadditional exploit that involves an extra resource r0, with pre-condition cn and post-condition d. Since every attack path q in this special resource graph has the same length| q |= n + 1, we need to find a path q that minimizes the set of distinct resourcesinvolved in q, denoted as R(q) (which also minimizes | R(q) | / | q |). Moreover, apath that minimizes | R(q) | clearly provides a solution to the MSC problem. Therefore,we can determine the network diversity d2 if and only if we can solve the MSC problem,which concludes the proof. �

Although determining network diversity is computationally infeasible in general, inmost cases the network diversity of a given network may still be computed or estimatedwithin a reasonable time using heuristics. In particular, Algorithm Heuristic Diversityshown in Figure 3 employs the heuristic of only maintaining a limited number of localoptima at each step in order to keep the complexity manageable. Specifically, the algo-rithm starts by marking all exploits and conditions as unprocessed (lines 1-2) and allinitial conditions as processed (line 3-4). Functions σ() and σ′() represent two collec-tions of attack paths (as sets of exploits, since the order of exploits is unimportant here)leading to an exploit or condition, to be used to calculate the minimum number of re-sources and steps, respectively. Therefore, for each initial condition c, such collectionsσ() and σ′() are both initialized as empty sets (line 5).

9

Procedure Heuristic DiversityInput: Resource graph G(E ∪ C,Rr ∪Ri), goal condition cg , parameter kOutput: d2Method:1. For each e ∈ E and c ∈ C \ CI

2. Mark e and c as unprocessed3. For each c ∈ CI

4. Mark c as processed5. Let σ(c) = σ′(c) = φ6. While (∃e ∈ E)(e is unprocessed) and (∀c ∈ C)((c, e) ∈ Rr ⇒ c is processed)7. Let {c ∈ C : (c, e) ∈ Rr} = {c1, c2, . . . , cn}8. Let σ(e) = ShortestK({q1 ∪ q2 ∪ . . . ∪ qn ∪ {e} : qi ∈ σ(ci), 1 ≤ i ≤ n}, k)9. Let σ′(e) = ShortestK′({q1 ∪ q2 ∪ . . . ∪ qn ∪ {e} : qi ∈ σ(ci), 1 ≤ i ≤ n}, k)10. Mark e as processed11. For each c s.t. (e, c) ∈ Ri

12. If (∀e′ ∈ E)((e′, c) ∈ Ri ⇒ e′ is processed) Then13. Let σ(c) = ShortestK(

⋃e′ s.t. (e′,c)∈Ri

σ(e′), k)14. Let σ′(c) = ShortestK′(

⋃e′ s.t. (e′,c)∈Ri

σ(e′), k)15. Mark c as processed

16.Returnminq∈seq(cg)|R(q)|minq′∈seq(cg)|q′|

Fig. 3. A Heuristic Algorithm for Computing the Network Diversity d2

The main loop cycles through each unprocessed exploit whose pre-conditions haveall been processed (line 6). For each such exploit e, all of its pre-conditions are firstplaced in a set (line 7). The collection of attack paths σ(e) (and σ′(e)) is then con-structed from the attack paths of those pre-conditions (line 8 and 9). Specifically, sincethe exploit e requires all the pre-conditions to be satisfied, an attack path leading to emust be the union of n attack paths (q1∪ q2∪ . . .∪ qn, each of which leads to one of thepre-conditions (qi ∈ σ(ci)). The function ShortestK() simply picks the top k solu-tions, that is, the k paths with the minimum number of distinct resources (ShortestK ′()for paths with the minimum number of steps). After this, the exploit e is marked asprocessed (line 10). Next, the inner loop cycles through each post-condition of e (line11-15) in a similar way (the differences arise from the fact that a condition c may besatisfied by any of the exploits implying it alone). The final result is calculated basedon the two collections of attack paths leading to the goal condition (line 16).

The complexity of the algorithm is dominated by the main loop (lines 6-15). Theouter loop will execute at most | E | times since it only cycles through unprocessedexploits while each cycle will mark one exploit as processed. The inner loop executesat most | C | times, and its complexity is dominated by line 13 and 14 which calculatethe union over at most k paths leading to each of the | E | or less exploits. Consideringthe maximum length of each path | E |, the complexity of the inner loop is thus | C |· | E |2 ·k. However, this complexity is actually dominated by line 8 and 9, in which atmost k paths may lead to every one of the | C | or less conditions, and this results in atmost k|C| candidates for ShortestK() (and ShortestK ′()) to choose from. Therefore,heuristics will be needed in designing the ShortestK() (and ShortestK ′()) function

10

such that it only evaluates a limited number of candidates in picking the top k solutions.However, in practice, the number of pre-conditions of most exploits is expected to be aconstant (compared to the size of the resource graph), and hence the overall complexity| E | ·(| C | · | E |2 ·k + k|C|) would still be manageable.

4 Probabilistic Network Diversity

In Section 2.1, we have shown that the least attacking effort-based metric only providesa partial picture of the threat and is insufficient by itself. In this section, we developa metric to capture the average attacking effort by combining all attack paths. For thispurpose, we take a probabilistic approach to modeling network diversity. More specif-ically, we define network diversity as the conditional probability p that, given that anattacker can compromise a given critical asset in the network, he/she would still be ableto do so even if all the resources were to be made different (i.e., every type of resourcewould appear at most once). This probability p represents the level of diversity currentlypresent in the network, and a higher value means higher diversity (in the special case ofp = 1, the network is already perfectly diverse, since further diversification effort willnot reduce the attack likelihood with respect to the given critical asset).

Clearly, the aforementioned conditional probability is equal to the ratio between twoprobabilities, the probability that an attacker may compromise the given critical assetwhen all resource instances in the network are different, and the probability that he/shecan do so in the current network. Both probabilities represent the attack likelihood withrespect to the goal condition, and can be modeled using a Bayesian network constructedbased on the resource graph (a similar approach using attack graph is given in [11]).

Definition 6 formally introduces network diversity following this intuition. In thedefinition, the first set of conditional probabilities represent the probability that an ex-ploit e can be successfully executed, given that all its pre-conditions are already satis-fied. The second and third set together represent the simple fact that an exploit cannotbe executed unless all its pre-conditions are already satisfied, whereas a condition canbe satisfied as the post-condition of one or more executed exploits. Finally, the fourthset represents the conditional probability that an exploit e2 may be executed by an at-tacker who has already successfully executed another exploit e1 which involves thesame resource (i.e., the attack likelihood while reusing a previous exploit).

Definition 6 (d3 Diversity). Given a resource graph G(E ∪ C,Rr ∪Ri), and

1. for each e ∈ E, a given conditional probability P (e | ∧{c:〈c,e〉∈Rr} c = TRUE),2. conditional probabilities P (e | ∧{c:〈c,e〉∈Rr} c = FALSE) = 0,3. conditional probabilities P (c | e = TRUE ∧ 〈e, c〉 ∈ Ri) = 1, and4. for any e1, e2 ∈ E involving the same resource r, conditional probabilities P (e1 |

e2 = TRUE ∧ (∧

{c:〈c,e1〉∈Rr} c) = TRUE) (and P (e2 | e1 = TRUE ∧∧

{c:〈c,e2〉∈Rr} c = TRUE)),

Given any cg ∈ C, the network diversity d3 is defined as d3 = p′

p where p denotes theconditional probability of cg being satisfied given that all the initial conditions are true,and p′ denotes the probability of cg being satisfied given that all initial conditions are

11

true and the above fourth set of probabilities not given (i.e., without considering theeffect of reusing any exploit).

Figure 4 demonstrates the proposed metric model using our running example, byassuming all resource instances to be different (even if they may be under the samename) except the http service, which is the same on three different hosts. In the figure,on the left side is the case when the effect of reusing an exploit is not considered inthe above definition, and on the right side the case when the same effect is considered.In the figure, each number inside the box represents the first set of given conditionalprobabilities (assigned with arbitrary values in this example). The dotted lines in theright figure show the last set of given conditional probabilities. The number beside eachexploit or condition represents the probability calculated through statistical inferencesusing the Bayesian network. Finally, we show part of the two conditional probability ta-bles (CPTs) to illustrate the difference between not considering the effect of reusing thehttp exploit (e.g., probability 0.5 in the left CPT), and considering it (e.g., probability0.9 in the right CPT). The diversity in this case will be calculated as d3 = 0.007

0.0103 .To instantiate the above model, we need to obtain the first and last set of conditional

probabilities in Definition 6. For the former, we can adopt the simple approach in [11] tobase the probability on the Common Vulnerability Scoring System (CVSS) [27] scores(available in public databases [29]). For zero day exploits, we can assign a nominalvalue as follows. Since a zero day vulnerability is commonly interpreted as a vulnera-bility not publicly known or announced, we can interpret this using the CVSS base met-rics [27], as a vulnerability with a remediation level unavailable, a report confidenceunconfirmed, and a maximum overall base score (and hence produce a conservativemetric value). We therefore obtain a nominal value of 0.8, converting to a probability of0.08 (for reference purpose, the lowest existing CVSS score in [29] is currently 1.7). Fi-nally, the last set of conditional probabilities models the attack likelihood while reusingan exploit on different machines and therefore can be assigned with a higher value thanthe corresponding attack probability in the first set.

5 Simulation

In this section, we study the performance of our proposed heuristic algorithm and brieflycompare the three proposed metrics via simulations, while leaving more detailed com-parative studies of those metrics to future work. All simulation results are collectedusing a computer equipped with a 3.0 GHz CPU and 8GB RAM in the Python environ-ment under Ubuntu 12.04 LTS. We calculate the Bayesian network-based metric usingOpenBayes [12]. To generate a large number of resource graphs, we first construct asmall number of seed graphs based on real networks, and then we obtain larger graphsfrom these seed graphs by injecting new hosts and assigning resources in a random butrealistic fashion (e.g., we vary the number of pre-conditions of each exploit within asmall range since real world exploits usually have a few pre-conditions).

The objective of the first two simulations is to evaluate the accuracy (approximationratio between the result obtained using our algorithm and that using brute force) of ourheuristic algorithm (in Figure 3). The left-hand side of Figure 5 shows the approxima-tion ratio in increasing k (the parameter of the algorithm that represents the number of

12

<http,0,1>0.08

<0,1> <user,0> <0,F>

<firewall,0,F>0.08

<ssh,1,4>0.08

<http,0,2>0.5

<2,4>user(2)

<user,4> <4,5>

<http,1,2>0.5

<user,1><1,4> <0,2><1,2>

<rsh,4,5>0 08

<http,4,5>0 5

<ssh,2,4>0.08

0.08

0.006 0.04

0.08

0.04

0.078

0.006

0.012

<http,4,5>

<user,4> <4,5> T F

T T 0.5 0.5

<user,5>

0.08 0.5

0.000099 0.006

0.007

T F 0 1

F T 0 1

F F 0 1

<http,0,1>0.08

<0,1> <user,0> <0,F>

<firewall,0,F>0.08

<ssh,1,4>0.08

<http,0,2>0.5

<2,4>user(2)

<user,4> <4,5>

<http,1,2>0.5

<user,1><1,4> <0,2><1,2>

<rsh,4,5>0 08

<http,4,5>0 5

<ssh,2,4>0.08

0.08

0.006 0.072

0.08

0.04

0.109

0.009

0.015

<http,4,5>

<http,0,1> <http,1,2> <user,4> <4,5> T F

T T T T 0.9 0.1

T T T F 0 1

T T F T 0 1

T F T T 0.9 0.1

T T F F 0 1

<user,5>

0.08 0.5

0.00117 0.0099

0.0103

T F T F 0 1

T F F T 0 1

T F F F 0 1

Fig. 4. Modeling Network Diversity Using Bayesian Networks

local optima stored at each step). We also examine the results under different in-degrees(i.e., the maximum number of pre-conditions of any exploit). We can see that the ap-proximate ratios increase with the in-degrees, and they decrease to an acceptable level(lower than 1.03) when k reaches about 4, and the trends stay flatten when k > 6 (al-most equal to 1). Therefore, k can be chosen as around 5 in practice. The right-handside of Figure 5 shows that the approximation ratio grows when the resource graph getslarger, which is expected (with a fixed parameter k, the relative amount of local optimastored at each step will decrease when the size of resource graphs increases, and henceworse performance). We only show k up to 3 since from the previous simulation it isclear that the approximation ratio will be close to 1 when k is 4 or greater.

The objective of next simulation is to evaluate the processing time of the heuristicalgorithm. Since the approximation ratio will stay flatten when k ≥ 6, we only showthe processing time for k ≤ 6. The left-hand side of Figure 6 shows that the processingtime is still acceptable (about 10 seconds) when k = 6 with around 1000 nodes. Forin-degrees of 3 and 4, the trend is much closer to linear. Although it is well knownthat inference using Bayesian networks is intractable in general, the right-hand side ofFigure 6 shows that our processing time for computing the Bayesian network-basedmetrics (using OpenBayes [12]) exhibits an acceptable trend (mostly due to the specialstructure of resource graphs).

The last two simulations compare the results of all three metrics proposed in this pa-per. To convert the Bayesian network-based metric d3 to a comparable scale of the othertwo, we use log0.08(p

′)log0.08(p)

(i.e., the ratio based on equivalent numbers of zero day exploits)instead of d3. In the left-hand side of Figure 7, the scatter points marked with X in the

13

Fig. 5. Approximation Ratio in k under Different In-degrees (Left) and in Graph Size under Dif-ferent k (Right)

Fig. 6. Processing Time for Computing d2 in Graph Size under Different k (Left) and ProcessingTime for Computing d3

red color are the individual values of d2. The blue points marked with Y are the valuesof d3 (converted as above). Also shown are their average values, and the average valueof the effective richness-based metric d1. While all three metrics follow a similar trend(diversity will decrease in larger graphs since there will be more duplicated resources),the Bayesian network-based metric d3 somehow reflects an intermediate result betweenthe two other extremes (d1 can be considered as the average over all resources, whereasd2 only depends on the shortest path). The right-hand side of Figure 7 shows the aver-age value of the three metrics in increasing number of distinct resources for resourcegraphs of a fixed size. All three metrics capture the same effect of increasing diversity,and their relationships are similar to that in the previous simulation.

6 Related Work

The research on security metrics has attracted much attention lately. Unlike existingwork which aim to measure the amount of network security [18, 38], this paper focuseson diversity as one particular property of networks which may affect security. Nonethe-

14

Fig. 7. Comparison of Metrics (Left) and the Effect of Increasing Diversity (Right)

less, our work borrows from the popular software security metric, attack surface [25],the general idea of focusing on interfaces (remotely accessible resources) rather thaninternal details (e.g., local applications). Our least attacking effort-based diversity met-ric is derived from the k-zero day safety metric [36, 35], and our probabilistic diversitymetric is based on the attack likelihood metric [11, 37]. Another notable work evaluatessecurity metrics against real attacks in a controlled environment [17], which providesa future direction to better evaluate our work. One limitation of our work lies in thehigh complexity of analyzing a resource graph; high level models of resource depen-dencies [21] may provide coarser but more efficient solutions to modeling diversity.

The idea of using design diversity for fault tolerance has been investigated for a longtime. The N-version programming approach generates N ≥ 2 functionally equivalentprograms and compares their results to determine a faulty version [3], with metricsdefined for measuring the diversity of software and faults [28]. The main limitation ofdesign diversity lies in the high complexity of creating different versions, which maynot justify the benefit [23]. The use of design diversity as a security mechanism hasalso attracted much attention [26]. The general principles of design diversity is shownto be applicable to security as well in [24]. The N-Variant system extends the idea ofN-version programming to detect intrusions [8], and the concept of behavioral distancetakes it beyond output voting [13]. Different randomization techniques have been usedto automatically generate diversity [4, 20, 33, 5].

In addition to design diversity and generated diversity, recent work employ oppor-tunistic diversity which already exists among different software systems. The practi-cality of employing OS diversity for intrusion tolerance is evaluated in [14] and thefeasibility of using opportunistic diversity already existing between different OSes totolerate intrusions is demonstrated. Diversity has also been applied to intrusion tolerantsystems which usually implement some kinds of Byzantine Fault Tolerant (BFT) repli-cation as fault tolerance solutions. Considering single-machine environments based onmultiple cores and virtualization, diversified replications are employed as a method tooffer Byzantine-fault tolerance to software attacks [7]. A generic architecture for imple-menting intrusion-tolerant Web servers based on redundancy and diversification prin-ciples is introduced using redundant proxies and diversified application servers with

15

redundancy levels selected according to threat levels [31]. Components-off-the-shelf(COTS) diversity is employed to provide an implicit reference model, instead of theexplicit model usually required, for anomaly detection in Web servers [34].

7 Conclusion

In this paper, we have taken a first step towards formally modeling network diversity asa security metric for evaluating networks’ robustness against zero day attacks. We firstdevised an effective richness-based metric based on the counterpart in ecology. We thenproposed a least attacking effort-based metric to address causal relationships betweenresources and a probabilistic metric to reflect the average attacking effort. Finally, weevaluated our algorithms and metrics through simulations.

The main limitations of this work are the following.

– First, our models depend on the availability and accuracy of many inputs, such asthe modeling of resources and their relationships (to form the resource graph), thedegree of difference and similarity between different types of resources (to calcu-late the effective richness), which may be challenging to characterize in practice.

– Second, we have employed simulations to evaluate our models, although it is cer-tainly ideal to conduct experiments with real-world networks and attacks. Unfor-tunately, there does not currently exist any publicly available benchmark datasetcontaining a significant number of representative real networks, and with both vul-nerabilities and attack information.

– Third, we have focused on modeling diversity, but did not address other factorsthat may also affect decisions regarding diversity, such as the cost (in terms ofdeployment and maintenance) and impact to functionality.

– Fourth, we regard all resources as equally likely to have zero day vulnerabilities,which can easily be extended by assigning different weights to resources, whentheir likelihood of having vulnerabilities can be estimated from past experiences.

As future work, we will address those limitations by refining and extending themodels, employing real vulnerabilities for experiments and case studies, and studyingvarious applications of the proposed diversity metrics.Acknowledgements The authors thank the anonymous reviewers for their valuablecomments. The work of Sushil Jajodia and Massimiliano Albanese was partially sup-ported by the Army Research Office under grants W911NF-13-1-0421, W911NF-09-1-0525, and W911NF-13-1-0317, and by the Office of Naval Research under grantsN00014-12-1-0461 and N00014-13-1-0703. The work of Sushil Jajodia and LingyuWang was also supported by the National Institute of Standards and Technology un-der the grant 60NANB14D060. The work of Lingyu Wang and Mengyuan Zhang wasalso supported by Natural Sciences and Engineering Research Council of Canada un-der Discovery Grant N01035. Commercial products are identified in order to adequatelyspecify certain procedures. In no case does such identification imply recommendationor endorsement by the National Institute of Standards and Technology, nor does it implythat the identified products are necessarily the best available for the purpose.

16

References

1. M. Albanese, S. Jajodia, and S. Noel. A time-efficient approach to cost-effective networkhardening using attack graphs. In Proceedings of the 42nd Annual IEEE/IFIP InternationalConference on Dependable Systems and Networks (DSN 2012), pages 1–12, 2012.

2. P. Ammann, D. Wijesekera, and S. Kaushik. Scalable, graph-based network vulnerabilityanalysis. In Proceedings of ACM CCS’02, 2002.

3. A. Avizienis and L. Chen. On the implementation of n-version programming for softwarefault tolerance during execution. In Proc. IEEE COMPSAC, volume 77, pages 149–155,1977.

4. S. Bhatkar, D.C. DuVarney, and R. Sekar. Address obfuscation: An efficient approach tocombat a broad range of memory error exploits. In Proceedings of the 12th USENIX securitysymposium, volume 120. Washington, DC., 2003.

5. S. Bhatkar and R. Sekar. Data space randomization. In Proceedings of the 5th InternationalConference on Detection of Intrusions and Malware, and Vulnerability Assessment, DIMVA’08, pages 1–22, Berlin, Heidelberg, 2008. Springer-Verlag.

6. J. Caballero, T. Kampouris, D. Song, and J. Wang. Would diversity really increase the robust-ness of the routing infrastructure against software defects? In Proceedings of the Networkand Distributed System Security Symposium, 2008.

7. B.G. Chun, P. Maniatis, and S. Shenker. Diverse replication for single-machine byzantine-fault tolerance. In USENIX Annual Technical Conference, pages 287–292, 2008.

8. B. Cox, D. Evans, A. Filipi, J. Rowanhill, W. Hu, J. Davidson, J. Knight, A. Nguyen-Tuong,and J. Hiser. N-variant systems: A secretless framework for security through diversity. De-fense Technical Information Center, 2006.

9. C. Elton. The ecology of invasion by animals and plants. University Of Chicago Press,Chicago, 1958.

10. N. Falliere, L. O. Murchu, and E. Chien. W32.stuxnet dossier. Symantec Security Response,2011.

11. M. Frigault, L. Wang, A. Singhal, and S. Jajodia. Measuring network security using dynamicbayesian network. In Proceedings of 4th ACM QoP, 2008.

12. K. Gaitanis and E. Cohen. Open bayes 0.1.0. https://pypi.python.org/pypi/OpenBayes, 2013.13. D. Gao, M. Reiter, and D. Song. Behavioral distance measurement using hidden markov

models. In Recent Advances in Intrusion Detection, pages 19–40. Springer, 2006.14. M. Garcia, A. Bessani, I. Gashi, N. Neves, and R. Obelheiro. OS diversity for intrusion

tolerance: Myth or reality? In 2011 IEEE/IFIP 41st International Conference on DependableSystems & Networks (DSN), pages 383–394, 2011.

15. M.R. Garey and D.S. Johnson. Computers and intractability: A guide to the theory of NP-Completeness. San Francisco: W.H. Freeman, 1979.

16. M.O. Hill. Diversity and evenness: a unifying notation and its consequences. Ecology,54(2):427–432, 1973.

17. H. Holm, M. Ekstedt, and D. Andersson. Empirical analysis of system-level vulnerabilitymetrics through actual attacks. IEEE Trans. Dependable Secur. Comput., 9(6):825–837,November 2012.

18. N. Idika and B. Bhargava. Extending attack graph-based security metrics and aggregatingtheir application. IEEE Transactions on Dependable and Secure Computing, 9:75–85, 2012.

19. S. Jajodia, A.K. Ghosh, V. Swarup, C. Wang, and X.S. Wang. Moving Target Defense:Creating Asymmetric Uncertainty for Cyber Threats. Springer, 1st edition, 2011.

20. G.S. Kc, A.D. Keromytis, and V. Prevelakis. Countering code-injection attacks withinstruction-set randomization. In Proceedings of the 10th ACM conference on Computerand communications security, pages 272–280. ACM, 2003.

17

21. N. Kheir, N. Cuppens-Boulahia, F. Cuppens, and H. Debar. A service dependency model forcost-sensitive intrusion response. In ESORICS, pages 626–642, 2010.

22. T. Leinster and C.A Cobbold. Measuring diversity: the importance of species similarity.Ecology, 93(3):477–489, 2012.

23. B. Littlewood, P. Popov, and L. Strigini. Modeling software design diversity: A review. ACMComput. Surv., 33(2):177–208, June 2001.

24. B. Littlewood and L. Strigini. Redundancy and diversity in security. Computer Security–ESORICS 2004, pages 423–438, 2004.

25. P.K. Manadhata and J.M. Wing. An attack surface metric. IEEE Trans. Softw. Eng.,37(3):371–386, May 2011.

26. R.A. Maxion. Use of diversity as a defense mechanism. In Proceedings of the 2005 Workshopon New Security Paradigms, NSPW ’05, pages 21–22, New York, NY, USA, 2005. ACM.

27. P. Mell, K. Scarfone, and S. Romanosky. Common vulnerability scoring system. IEEESecurity & Privacy, 4(6):85–89, 2006.

28. S. Mitra, N.R. Saxena, and E.J. McCluskey. A design diversity metric and analysis of redun-dant systems. IEEE Trans. Comput., 51(5):498–510, May 2002.

29. National vulnerability database. available at: http://www.nvd.org, May 9, 2008.30. E.C. Pielou. Ecological diversity. Wiley New York, 1975.31. A. Saıdane, V. Nicomette, and Y. Deswarte. The design of a generic intrusion-tolerant archi-

tecture for web servers. IEEE Trans. Dependable Sec. Comput., 6(1):45–58, 2009.32. O. Sheyner, J. Haines, S. Jha, R. Lippmann, and J.M. Wing. Automated generation and

analysis of attack graphs. In Proceedings of the 2002 IEEE Symposium on Security andPrivacy, 2002.

33. The PaX Team. PaX address space layout randomization. http://pax.grsecurity.net/.34. E. Totel, F. Majorczyk, and L. Me. Cots diversity based intrusion detection and application

to web servers. In RAID, pages 43–62, 2005.35. L. Wang, S. Jajodia, A. Singhal, P. Cheng, and S. Noel. k-zero day safety: A network security

metric for measuring the risk of unknown vulnerabilities. IEEE Transactions on Dependableand Secure Computing, 11(1):30–44, 2013.

36. L. Wang, S. Jajodia, A. Singhal, and S. Noel. k-zero day safety: Measuring the security riskof networks against unknown attacks. In Proceedings of the 15th European Symposium onResearch in Computer Security (ESORICS), pages 573–587, 2010.

37. L. Wang, A. Singhal, and S. Jajodia. Measuring the overall security of network configura-tions using attack graphs. In Proceedings of 21th IFIP DBSec, 2007.

38. L. Wang, A. Singhal, and S. Jajodia. Toward measuring network security using attack graphs.In Proceedings of 3rd ACM QoP, 2007.

39. Y. Yang, S. Zhu, and G. Cao. Improving sensor network immunity under worm attacks: asoftware diversity approach. In Proceedings of the 9th ACM international symposium onMobile ad hoc networking and computing, pages 149–158. ACM, 2008.

40. S. Yuan, S. Varma, and J.P. Jue. Minimum-color path problems for reliability in mesh net-works. In 24th Annual Joint Conference of the IEEE Computer and Communications Soci-eties (INFOCOM), pages 2658–2669, 2005.

18