Security: Lessons Learned and Missed from Java

Nathanael Paul

David Evans{nate,evans}@cs.virginia.edu

University of Virginia

ACSAC 2004

0

5

10

15

20

25

30

35

40

45

50

1996 1997 1998 1999 2000 2001 2002 2003 2004

Java VM

.NET VM

Major Security Vulnerabilities (Cumulative)

3

Why the disparity in vulnerabilities?Hypotheses:

• No one uses/attacks .NET– Windows Update installs .NET framework– Attractive target with over 90% market share

• Microsoft is smarter than everyone else– Check their profit and market share

• Learned from past– .NET learned from experience with Java

4

Universal Security Principles[Saltzer and Shroeder, 1974] [McGraw and Viega, 2001]

• Keep it simple

• Complete Mediation

• Least Privilege

• Secure Weakest Link

• Defense in Depth

5

Virtual MachinesPlatforms that allow untrusted code to execute with restrictions enforced by the virtual machine (VM)

Java VM

Operating System

Protected Resource

ClassLoaderSecurity

exception

Security exception

Verify Exception

JAR Assembly

Policy Manager

Class

Verifier

Class Loader

JIT VerifierVerify Exception

CLR

Operating System

Protected Resource

Security exception

Java .NET

6

Source

bytecodes

Verifier

Low-level Code Safety

• Must ensure programs are type, memory, and control safe using data-flow analysis

• High-level policy enforcement depends on low-level code safety

VM

7

Verifier is (should be) Conservative

.NET/Java programs

Safe programs

Verifiable programs

Bug

8

Object Creation and Initialization

• Virtual machine must ensure object is initialized before use– Security permissions restrict some

objects from being created– Improper initialization can create a

vulnerability• Bug in MSIE 4.0, 5.0, 6.0 [lsd-pl.net]• Similar bug in Sun and Netscape

Lesson 1: Keep it simple

9

• Java– new – create new object reference– dup – duplicate reference– invokespecial – calls constructor

• .NET– newobj is equivalent to Java’s new,

dup, and invokespecial instructions

Object Creation Instructions

10

Object Initialization Vulnerability [lsd-pl.net]

class LSDbug extends SecurityClassLoader {public LSDbug() {

try { LSDbug(5); } catch (SecurityException e) { this.loadClass(…);

} }

public LSDbug (int x) { super(); // throws Security Exception }}

11

Bootstrapping the VM

• Need to bootstrap the virtual machine

• Certain classes providing policy enforcement need full trust– Infinite recursion if checks needed on all

classes

Lesson 2: Least Privileges

12

Bootstrapping the VM

• Java 1.0– Fully trusted code on CLASSPATH– Current Java versions have

bootclasspath for backwards compatibility

• .NET’s trusted path is a cache of signed files

13

Location-based Vulnerability[Hopwood, 1996]

• Netscape cached files on local filesystem

• Guessing cached file names could allow arbitrary code execution

• Applet could execute cached files located on CLASSPATH

14

Monitoring Execution

Lesson 3: Fail-safe Defaults and Complete Mediation

15

Monitoring Execution

• Want policy extensible but complicates policy enforcement– Java 1.0 (HotJava) and 1.1 had all or nothing

trust for applets

• Reference monitor should be tamper-proof and always be invoked

16

Reference Monitor’s Enforcement

• Java’s reference monitor, the SecurityManager may be bypassed

SecurityManager sm = System.getSecurityManager();if (sm != null) {sm.checkListen(21); // listen on port 21?

}

• .NET’s SecurityManager cannot be inherited or instantiated

17

Failure to Monitor Vulnerability[Brumleve, 2000]

• SecurityManager.checkListen() allows creation of a ServerSocket object

• Flaw in ServerSocket.implAccept(Socket s)– Accepts connection to get remote address and port

number– Calls socket’s close() and throws

SecurityException if permissions violated– Subclass of Socket can override close() to keep

socket open

18

Principles Review• Keep it simple

– Object initialization– .jsr/swap vulnerability (see paper)

• Least privileges– Bootstrapping the VM– Stack Inspection

• Fail-safe Defaults and Complete Mediation– Brown Orifice– DoS attacks– Union/Intersection in Policy Resolution

19

Conclusions• Classic security principles still important today

• Hard to follow them in real systems– Easier to find complex solutions than simple ones– Tradeoffs between security and other goals

• Complete Mediation vs. Efficiency (policy expressiveness)

• Simplicity vs. Backwards compatibility (bootclasspath)

• Fail-safe defaults vs. Usability (Default Policies)

• Some reasons for optimism

20

Questions

?

21

Conclusions

• Why do we still have problems today?– Security vs. Efficiency– Defense in Depth vs. Simplicity [McGraw,

Viega]– Flexibility vs. Simplicity

• Evaluate principles in context [McGraw, Viega]

22

Object Initialization Vulnerability [lsd-pl.net]

• <init>()LSDbug →<init>(I)LSDbug → com/ms/security/SecurityClassLoader/<init>()LSDbug

• Security exception occurs (caught by <init>()LSDbug) since code does not have permission to instantiate ClassLoader

23

Granted Permissions in Policies• Permissions are granted, not excluded• Java’s policy is the union of all granted

permissions• .Net policy is the intersection of a 4-level

hierarchical policy– Enterprise– Machine– User– AppDomain

Lesson 3: Fail-safe defaults in Permission Resolution

24

Static/Dynamic Permissions

• Policy enforcement can be optimized– Need flexibility

• Static permissions– Must be known before run-time– Faster checking possible

• Dynamic– Can change on-the-fly– Checks delayed until run-time

Lesson 3: Fail-safe Defaults

25

Policy Implementation: Static/Dynamic Permissions

Granted in class loaders (e.g., AppletClassLoader)

Attached to assemblies and can be checked before run-time

Union of all permissions in policy files

Intersection of permissions in policy files

Sta

tic

Dy

nam

ic

Java .Net

26

notes

• Emphasize overall point (talk of analysis of lessons learned… one sentence – slide 2)

• Pointer• Don’t flip between overall pic• Make sure point out vulnerability is on Java• Wrap up each section (at end of vulnerability) better• Have better transitions• Mention a couple more of the s & s principles• Look more at audience• Point out no significant security vulnerabilities in .Net

(double check)

27

notes

• More principles– Defense in depth– Chain is only strong as weakest link– Secure failure (not seen in Java’s exceptions!)– Compartmentalization– Choke points (narrow interface to system)– Usability– Trust community (open design crytpo)– No security through obscurity– Educate user

28

Object Initialization Vulnerability [lsd-pl.net]

• LSDbug child class of SecurityClassLoader• Call constructor, call constructor, call

superclass constructor (exception occurs)

newdupinvokespecial LSDbug()

…invokespecial LSDbug(int)

…invokespecial SecurityClassLoader()

29

Object Initialization Vulnerability[lsd-pl.net]

• MSIE 4.0, 5.0, 6.0

• Create object of a security-critical class to escalate privileges

• Similar bug in Sun and Netscape implementations

30

Verifier is (should be) Conservative

.NET/Java programs

Safe programs

Verifiable programs

31

Complexity Increases Risk

.NET/Java programs

Safe programs

Verifiable programs

Bug