Minestrone: TesngtheSOUP - USENIX · MINESTRONE:Tes-ngtheSOUP,CSET13 Overview •...

1

Minestrone: Tes-ng the SOUP Azzedine Benameur, Nathan Evans, Ma?hew Elder

MINESTRONE: Tes-ng the SOUP, CSET 13

Agenda

• Overview • MINESTRONE: –  Architecture –  Sta-c and dynamic detec-on technologies

–  I/O Redirec-on –  External Replica Monitoring

• Test and evalua-on: –  Architecture –  Test suite –  Results

• Closing Thoughts

2


Overview

•  IARPA STONESOUP Program: “Securely Taking On New Executable SoLware of Uncertain Provenance”

–  Develop and demonstrate technology that provides comprehensive, automated techniques that allow end users to safely execute new soLware of uncertain provenance

–  Addressing 8 “weakness” classes across 3 target language classes

• Team: Columbia University (PI: Angelos Keromy-s) with Stanford University, George Mason University (GMU), and Symantec

• 4-‐year project, 3 phases 3


Overview: NSA Source Code Analysis Tool Evalua-on

• hYp://www.iarpa.gov/stonesoup_Merced_DHSAWGbrief.pdf • Evaluated suite of tools (Coverity Prevent, FindBugs, For-fy SCA, GrammaTech CodeSonar, Klocwork Insight, Ounce Labs Ounce, PMD) against both C/C++ and Java vulnerability test cases in different CWE (Common Weakness Enumera-on) classes

4


Overview: STONESOUP Program Targets

•  Target classes of vulnerabili-es (including example CWE numbers): –  Number handling (e.g., integer overflow/underflow, sign conversion: #190, #191 –  Resource drains (e.g., failure to release memory, structures, devices: #400, #404) –  Tainted data/input valida-on errors (#78, #134) –  Error handling (e.g., unhandled excep-ons/error status codes: #248, #252) –  SQL injec-on / command injec-on (#78, #89) –  Concurrency handling (e.g., race condi-ons, thread safety: #362, #366) –  Buffer overflows/underflows/out of bounds accesses/memory safety (#121, #122) –  Null pointer errors (#476)

•  Target language classes: –  Type-‐safe languages (Java, C#) –  Type-‐unsafe languages (C, C++) –  Binaries (x86, Windows or Linux)

5


MINESTRONE

• Architecture

• Sta-c and dynamic detec-on technologies

• Replica Diversifica-on

•  I/O Redirec-on

• External Replica Monitoring

6


UnknownSoftware

Lightweight Containers

Lightweight Containers

ISR + defensive instrumentation

ISR + defensive instrumentation

UnknownSoftware

KLEEcontinuoussymbolicexecution

Runtime

Offline/parallel

Repli

cate

d run

time

Remove/optimizeunneeded defenses

Path explorationpreference & control flow

information

MINESTRONESystem Composer

Symbiotes

AnomalyDetection

RaceAvoidance

RaceDetection

Informationflow trackingoptimization

Attackdetection

I/O & statereplication

Symbiotes AnomalyDetection

Deployedapplication

(N instances)

Backendanalysis

(M << N instances)

Instrumentedreplicas

(P < N instances)

ResourceExhaustionDetection

ResourceExhaustionDetection

KLEEprophylactic

analysis

Minestrone Overview: System Architecture

7


MINESTRONE

• Architecture




8


Detec-on technologies: Pin-‐based tools (theRing)

• REASSURE –  Self-‐contained Mechanism for Healing SoLware Using Rescue Points –  Detects program crashes and gracefully recovers

•  ISR: Instruc-on Set Randomiza-on –  Applica-on binary is randomized –  Shared libraries can be also randomized

• DFT –  Data flow tracking –  High performance

9


Detec-on technologies: con-nued

•  KLEE –  Symbolic Execu-on –  Fine-‐Grained detec-on

•  Dyboc –  Source to source transforma-on –  Moving stack buffers to heap –  Custom version of malloc(): pmalloc()

•  Valgrind (baseline) –  State of the art –  Memcheck

10


MINESTRONE

• Architecture for confinement




11


I/O Redirec-on: Network/Shared Memory/X11

• Paired-‐library: –  Interpose_writer: writes to file from no-‐sec environment

–  Interpose_reader: read from file in all replicas

12


MINESTRONE

• Architecture for confinement




13


External Replica Monitoring

14

• OpenVZ allows easy replica monitoring –  CPU from /proc/vz/vestat

– Memory using bean counters /proc/user_beancounters

–  Network from /vz/root/$replica_id/sys/class/net/venet0/sta-s-cs/tx_bytes

• Overhead comparison: –  Confinement between containers –  Fair scheduling


Test and Evalua-on

•  Architecture

•  Test suite

•  Results

15


Test and Evalua-on Process, cont’d.

• MITRE developed tes-ng framework and API • We/Symantec developed the interface to interact with the test harness

16


Test and Evalua-on

•  Architecture

•  Test suite

•  Results

17


Test and Evalua-on: Test suite

•  Vulnerability Classes: –  Null pointer, 113 hYp://samate.nist.gov/SRD/testsuites/stonesoup/stonesoup-‐c-‐np.zip –  Buffer Over/underflow, 231 hYp://samate.nist.gov/SRD/testsuites/stonesoup/stonesoup-‐c-‐mc.zip

•  Input source: –  Environment variable –  Command line arguments –  File –  Network –  Shared Memory –  Clipboard

18


Test and Evalua-on

•  Architecture

•  Test suite

•  Results

19


Test and Evalua-on: Results

20



21



22



23



24

MINESTRONE: Tes-ng the SOUP, CSET 13 25

Closing Thoughts


Lessons learned

• Symbolic execu-on limita-ons: –  Limited model

–  Very slow when it works (observed a 2700X overhead) • Wri-ng test suite from scratch is tricky: –  Stack not always ini-alized to 0 –  Provide the vulnerability loca-on to establish the ground truth

•  I/O Redirec-on/replay is not a solved problem: – Many implementa-on available, ioapps, Jockey –  Can you build it ? Do they work ?

• Enterprise products are not the silver bullet: –  Single mul--‐purpose tools don’t outperform single purpose tailed tools

26

Thank you!

Copyright © 2011 Symantec Corpora-on. All rights reserved. Symantec and the Symantec Logo are trademarks or registered trademarks of Symantec Corpora-on or its affiliates in the U.S. and other countries. Other names may be trademarks of their respec-ve owners. This document is provided for informa-onal purposes only and is not intended as adver-sing. All warran-es rela-ng to the informa-on in this document, either express or implied, are disclaimed to the maximum extent allowed by law. The informa-on in this document is subject to change without no-ce.

27

Azzedine Benameur [email protected] Nathan Evans [email protected] MaYhew Elder [email protected]

Minestrone: TesngtheSOUP - USENIX · MINESTRONE:Tes-ngtheSOUP,CSET13 Overview •...

Documents

Transcript of Minestrone: TesngtheSOUP - USENIX · MINESTRONE:Tes-ngtheSOUP,CSET13 Overview •...

Minestrone: TesngtheSOUP - USENIX · MINESTRONE:*Tes-ng*the*SOUP,*CSET*13* Overview •...

Documents

Transcript of Minestrone: TesngtheSOUP - USENIX · MINESTRONE:*Tes-ng*the*SOUP,*CSET*13* Overview •...

Minestrone: TesngtheSOUP - USENIX · MINESTRONE:Tes-ngtheSOUP,CSET13 Overview •...

Transcript of Minestrone: TesngtheSOUP - USENIX · MINESTRONE:Tes-ngtheSOUP,CSET13 Overview •...