CSE543 - Introduction to Computer and Network Security...

33
CSE543 - Introduction to Computer and Network Security Page CSE543 - Introduction to Computer and Network Security Module: Operating System Security Professor Trent Jaeger 1

Transcript of CSE543 - Introduction to Computer and Network Security...

Page 1: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

CSE543 - Introduction to Computer and Network Security

Module: Operating System Security

Professor Trent Jaeger

1

Page 2: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

OS Security• So, you have built an operating system that enables

user-space processes to access hardware resources‣ Thru various abstractions: files, pages, devices, etc.

• Now, you want your operating system to enforce security requirements for your application processes‣ What do you do?

2

Page 3: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

OS Security • We learned about a few things that will help you• Your OS must implement a ‣ (Mandatory) Protection system

• That can enforce a‣ MAC policy

• How do we implement such an OS mechanism? ‣ Multics‣ Linux Security Modules

3

Page 4: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Access Policy Enforcement• A protection system uses a reference validation

mechanism to produce and evaluate authorization queries‣ Interface: Mediate security-sensitive operations by building

authorization queries to evaluate‣ Module: Determine relevant protection state entry (ACLs,

capabilities) to evaluate authorization query‣ Manage: Install protection state entries and reason about

labeling and transition states

• How do we know whether a reference validation mechanism is correct?

4

Page 5: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Security-Sensitive Operations• Broadly, operations that enable interaction among

processes that violate secrecy, integrity, availability• Which of these are security-sensitive? Why?‣ Read a file (read)‣ Get the process id of a process (getpid)‣ Read file metadata (stat)‣ Fork a child process (fork)‣ Get the metadata of a file you have already

opened? (fstat)‣ Modify the data segment size? (brk)

• Require protection for all of CIA?5

Page 6: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Reference Monitor• Defines a set of requirements on reference

validation mechanisms ‣ To enforce access control policies correctly

• Complete mediation‣ The reference validation mechanism must always be

invoked (before executing security-sensitive operations)• Tamperproof‣ The reference validation mechanism must be tamperproof

• Verifiable‣ The reference validation mechanism must be small

enough to be subject to analysis and tests, the completeness of which can be assured

6

Page 7: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Multiprocessor Systems• Major Effort: Multics‣ Multiprocessing system -- developed many OS concepts

• Including security

‣ Begun in 1965 • Research continued into the mid-70s

‣ Used until 2000‣ Initial partners: MIT, Bell Labs, GE (replaced by Honeywell)‣ Other innovations: hierarchical filesystems, dynamic linking

• Subsequent proprietary system, SCOMP, became the basis for secure operating systems design (XTS-400)

7

Page 8: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Multics Goals• Secrecy‣ Multilevel security

• Integrity ‣ Rings of protection

• Resulting system is considered a high point in secure systems design

8

Page 9: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Protection Rings• Successively less-privileged “domains”• Modern CPUs support 4 rings‣ Use 2 mainly: Kernel and user

• Intel x86 rings‣ Ring 0 has kernel

‣ Ring 3 has application code

• Example: Multics (64 rings in theory, 8 in practice)9

Page 10: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

What Are Protection Rings?• Coarse-grained, Hardware Protection Mechanism• Boundary between Levels of Authority‣ Most privileged -- ring 0‣ Monotonically less privileged above

• Fundamental Purpose‣ Protect system integrity

• Protect kernel from services

• Protect services from apps

• So on...

10

Page 11: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Ring 0

Ring 3

Protection Ring Rules• Program cannot call code of

higher privilege directly‣ Gate is a special memory

address where lower-privilege code can call higher• Enables OS to control where

applications call it (system calls)

11

Gate

No gate

Page 12: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Multics Interpretation• Kernel resides in ring 0• Process runs in a ring r‣ Access based on current ring

• Process accesses data (segment)‣ Each data segment has an access

bracket: (a1, a2)• a1 <= a2

‣ Describes read and write access to segment• r is the current ring• r <= a1: access permitted• a1 < r <= a2: r and x permitted; w denied• a2 < r: all access denied

12

1

2

3

4

5

6

7

0

a1

a2

RWX

R-X

---

Ring

Page 13: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Multics Interpretation (con’t)• Also different procedure segments

‣ with call brackets: (c1, c2), c1 <= c2

‣ and access brackets (a1, a2)

‣ The following must be true (a2 == c1)

‣ Rights to execute code in a new procedure segment• r < a1: access permitted with ring-crossing fault

• a1 <= r <= a2 = c1: access permitted and no fault

• a2 < r <= c2: access permitted through a valid gate

• c2 < r: access denied

• What’s it mean?‣ case 1: ring-crossing fault changes procedure’s ring

• increases from r to a1

‣ case 2: keep same ring number

‣ case 3: gate checks args, decreases ring number

• Target code segment defines the new ring

13

1

2

3

4

5

6

7

0

a1

a2

Allow

with

gate

No ring

fault

Denied

Ring

c2

c1

Ring

fault

Page 14: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Examples• Process in ring 3 accesses data segment‣ access bracket: (2, 4)‣ What operations can be performed?

• Process in ring 5 accesses same data segment‣ What operations can be performed?

• Process in ring 5 accesses procedure segment‣ access bracket (2, 4) ‣ call bracket (4, 6)‣ Can call be made?‣ How do we determine the new ring?‣ Can new procedure segment access the data segment

above?14

Page 15: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Now forward to UNIX ...

15

Page 16: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

UNIX Security Limitations• Circa 2000 Problems‣ Discretionary access control

‣ Setuid root processes

‣ Network-facing daemons vulnerable

• What can we do?

16

Page 17: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

UNIX Security Limitations• Circa 2000 Problems‣ Discretionary access control

‣ Setuid root processes

‣ Network-facing daemons vulnerable

• What can we do?‣ Reference validation mechanism that satisfies reference

monitor concept

‣ Protection system with mandatory access control (mandatory protection system)

17

Page 18: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Linux Security Modules• Reference validation mechanism for Linux‣ Upstreamed in Linux 2.6

‣ Support modular enforcement - you choose

• SELinux, AppArmor, POSIX Capabilities, SMACK, ...

• 150+ authorization hooks‣ Mediate security-sensitive operations on

• Files, dirs/links, IPC, network, semaphores, shared memory, ...

‣ Variety of operations per data type

• Control access to read of file data and file metadata separately

• Hooks are restrictive

18

Page 19: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

LSM & Reference Monitor• Does LSM satisfy reference monitor concept?

19

Page 20: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

LSM & Reference Monitor• Does LSM satisfy reference monitor concept?‣ Tamperproof

• Can MAC policy be tampered?• Can kernel be tampered?

20

Page 21: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Linux Security Modules

• Register (install) module

• Load policy (open and write to special file)

• Produce authorization queries at hooks21

Systems and Internet Infrastructure Security (SIIS) Laboratory Page

LSM API

14

Page 22: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Linux Security Modules

• Attacks on “register”

• Attacks on “install policy”

• Attacks on “system calls”22

Systems and Internet Infrastructure Security (SIIS) Laboratory Page

LSM API

14

Page 23: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Linux Security Modules

• To prevent attacks on registration

• And attacks on function pointers of LSM

• LSMs are now statically compiled into the kernel23

Systems and Internet Infrastructure Security (SIIS) Laboratory Page

LSM API

14

Page 24: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

LSM & Reference Monitor• Does LSM satisfy reference monitor concept?‣ Tamperproof

• Can MAC policy be tampered?• Can kernel be tampered?

‣ Verifiable• How large is kernel?• Can we perform complete testing?

24

Page 25: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

LSM & Reference Monitor• Does LSM satisfy reference monitor concept?‣ Tamperproof

• Can MAC policy be tampered?• Can kernel be tampered? By network threats?

‣ Verifiable• How large is kernel?• Can we perform complete testing?

‣ Complete Mediation• What is a security-sensitive operation?• Do we mediate all paths to such operations?

25

Page 26: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Linux Security Modules

26Systems and Internet Infrastructure Security (SIIS) Laboratory Page

LSM Hooks

13

Page 27: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

LSM & Complete Mediation• What is a security-sensitive operation?‣ Instructions? Which?‣ Structure member accesses? To what data?‣ Data types whose instances may be controlled

• Inodes, files, IPCs, tasks, ...• Approaches‣ Mediation: Check that authorization hook

dominates all control-flow paths to structure member access on security-sensitive data type

‣ Consistency: Check that every structure member access that is mediated once is always mediated• Several bugs found - some years later

27

Page 28: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

LSM & Complete Mediation

28Systems and Internet Infrastructure Security (SIIS) Laboratory Page

LSM Analysis •  Static analysis of Zhang, Edwards,

and Jaeger [USENIX Security 2002]!

‣  Based on a tool called CQUAL!

•  Found a TOCTTOU bug!

‣  Authorize filp in sys_fcntl!

‣  But pass fd again to fcntl_getlk!

•  Many supplementary analyses were necessary to support CQUAL!

21

/* from fs/fcntl.c */

long sys_fcntl(unsigned int fd,

unsigned int cmd,

unsigned long arg)

{

struct file * filp;

...

filp = fget(fd);

...

err = security ops->file ops

->fcntl(filp, cmd, arg);

...

err = do fcntl(fd, cmd, arg, filp);

...

}

static long

do_fcntl(unsigned int fd,

unsigned int cmd,

unsigned long arg,

struct file * filp) {

...

switch(cmd){

...

case F_SETLK:

err = fcntl setlk(fd, ...);

...

}

...

}

/* from fs/locks.c */

fcntl_getlk(fd, ...) {

struct file * filp;

...

filp = fget(fd);

/* operate on filp */

...

}

Figure 8: Code path from Linux 2.4.9 containing an ex-

ploitable type error.

THREAD-A:

(1) fd1 = open("myfile", O_RDWR);

(2) fd2 = open("target_file", O_RDONLY);

(3) fcntl(fd1, F_SETLK, F_WRLOCK);

KERNEL-A (do_fcntl):

(4) filp = fget(fd1);

(5) security_ops->file_ops

->fcntl (fd1);

(6) fcntl_setlk(fd1,cmd)

THREAD-B:

/* this closes fd1, dups fd2,

* and assigns it to fd1.

*/

(7) dup2( fd2, fd1 );

KERNEL-A (fcntl_setlk)

/* this filp is for the target

* file due to (7).

*/

(8) filp = fget (fd1)

(9) lock file

Figure 9: An example exploit.

chance of race conditions when the data structures are

not properly synchronized, which may result in poten-

tial exploits.

Here we present a type error of this kind. Many se-

curity checks that intend to protect the inode structure

are performed on the dentry data structure. For exam-

ple, the following code does the permission check on the

dentry structure, but does the “set attribute” operation

on the inode structure.

/* from fs/attr.c */

...

security_ops->inode_ops

->setattr(dentry, attr);

...

inode = dentry->d_inode;

inode_setattr(inode, attr);

...

It is also quite common in Linux to check on the file

data structure and operate on the inode data structure.

Page 29: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

LSM Enforcement• Several LSMs have been deployed‣ Most prominent: AppArmor, SELinux, Smack,

TOMOYO• The most comprehensive is SELinux‣ Used by RedHat Fedora and some others

29

Page 30: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

LSM Enforcement• Several LSMs have been deployed‣ Most prominent: AppArmor, SELinux, Smack,

TOMOYO• The most comprehensive is SELinux‣ Created by the NSA - Result of many years work‣ Used by RedHat Fedora and some others

30

Page 31: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

LSM Enforcement• Several LSMs have been deployed‣ Most prominent: AppArmor, SELinux, Smack,

TOMOYO• The most comprehensive is SELinux‣ Created by the NSA - Result of many years work‣ Used by RedHat Fedora and some others

31Systems and Internet Infrastructure Security (SIIS) Laboratory Page

SELinux Policy Rules •  SELinux Rules express an MPS

‣  Protection state – ALLOW subject-label object-label ops

‣  Labeling state – Assign new objects labels on creation

‣  Transition state – Define how a process may change label

•  All are defined explicitly

‣  Tens of thousands of rules are necessary for a standard Linux distribution

•  Remember, we are ignoring user processes too (other than confining them relative to the system)

•  Enforces a Least Privilege Policy

9

Page 32: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

LSM Enforcement• Several LSMs have been deployed‣ Most prominent: AppArmor, SELinux, Smack,

TOMOYO• The most comprehensive is SELinux‣ Created by the NSA - Result of many years work‣ Used by RedHat Fedora and some others

32Systems and Internet Infrastructure Security (SIIS) Laboratory Page

SELinux Transition State •  For user to run passwd program

‣  Only passwd should have permission to modify /etc/shadow

•  Need permission to execute the passwd program

‣  allow user_t passwd_exec_t:file execute (user can exec /usr/bin/passwd)

‣  allow user_t passwd_t:process transition (user gets passwd perms)

•  Must transition to passwd_t from user_t

‣  allow passwd_t passwd_exec_t:file entrypoint (run w/ passwd perms)

‣  type_transition user_t passwd_exec_t:process passwd_t

•  Passwd can the perform the operation

‣  allow passwd_t shadow_t:file {read write} (can edit passwd file)

10

Page 33: CSE543 - Introduction to Computer and Network Security ...trj1/cse543-f16/slides/cse543-os-security.pdf · CSE543 - Introduction to Computer and Network Security Page OS Security

CSE543 - Introduction to Computer and Network Security Page

Take Away• Goal: Build authorization into operating systems‣ Multics and Linux

• Requirements: Reference monitor‣ Satisfy reference monitor concept

• Multics‣ Hierarchical Rings for Protection

‣ Call/Access Bracket Policies (in addition to MLS)

• Linux‣ Did not enforce security (DAC, Setuid, root daemons)

‣ So, the Linux Security Modules framework was added

‣ Approximates reference monitor assuming network threats only -- some challenges in ensuring complete mediation

33