44CON London 2015 - Software Defined Networking (SDN) Security
44CON London 2015 - 15-Minute Linux Incident Response Live Analysis
-
Upload
44con -
Category
Technology
-
view
643 -
download
1
Transcript of 44CON London 2015 - 15-Minute Linux Incident Response Live Analysis
15-Minute Linux Live Analysis
Dr. Phil (Polstra)Bloomsburg University of Pennsylvania
What is this talk about?
● Determining with some certainty if you have been hacked● In a matter of minutes● With minimal disturbance to the subject system● Automating the process with shell scripting
Why should you care?
● Someone calls you about a suspected breach● You need to need to figure out if they were hacked
● Quickly so as to avoid further harm to your client● Without destroying evidence● Without taking down a critical machine
Who is the handsome man before you?
● Digital forensics professor at Bloomsburg University● Programming since age 8 (in Assembly since age 10)● Known to hack electronics from age 12● Linux user from its start● Author USB
Forensics
Insert coolCover art
Roadmap
● Opening a case● Talking to the users● Mounting known-good binaries● Minimizing disturbance to the system● Collecting Data● Automation with scripting● Next steps if there is a breach
Opening a Case
● Decide on case name (date?)● Create a case folder on your laptop● Start making entries in your notebook
● Bound notebook with numbered pages● Easy to carry● Hard to insert/remove pages● No batteries required
First Talk to the Users
● They know the situation better than you● Might be able to tell a false alarm before digging in● Why did you call me?
● What they suspect● No internal experts or policy to use outsider?
● Why do you think there was an incident?● Everything they know about the subject system
USB Response Drive
● Contains known-good binaries● Minimum /bin, /sbin, /lib for same architecture● Might also grab /usr/sbin, /usr/bin, /usr/lib● Must be on an ext2, ext3, or ext4 partition
● Could contain a bootable Linux on another partition● This partition will probably be FAT ● Should be first partition● See http://linuxforensicsbook.com/hdvideos.html Chapter 1
Mounting Known-Good Binaries
● Insert response drive● Exec your bash binary● Set path to your binaries (and only your binaries)● Set LD_LIBRARY_PATH● Run all shell scripts as bash <script>
● Don't use she-bang (#!) in scripts!
Demo: Mounting Binaries
Minimize Disturbance to System
● You will always change the system a little● Goal is to
● Minimize memory footprint● Never write to subject media
● Two basic options● Save everything to your USB response drive● Send it over the network
Sending data over the network
● Better than USB drive due to caching● Use netcat
● Create a listener for “log” information on forensics workstation● Send “log” information from client● Also create a listener for interesting files on forensics
workstation– Spawn a new listener when files are sent
Setting Up Log Listener
● netcat -k -l 9999 >> case-log.txt● (-k) keep alive (-l) listen (>>) append● From subject
● {command} | netcat {forensic ws IP} 9999
● Let's use shell scripting to automate this● Shell not Python because we want to minimize memory
footprint
Automating the Log Listener
usage () {
echo "usage: $0 <case number>"
echo "Simple script to create case folder and start listeners"
exit 1
}
if [ $# -lt 1 ] ; then
usage
else
echo "Starting case $1"
fi
#if the directory doesn't exist create it
if [ ! -d $1 ] ; then
mkdir $1
fi
# create the log listener
`nc -k -l 4444 >> $1/log.txt` &
echo "Started log listener for case $1 on $(date)" | nc localhost 4444
# start the file listener
`./start-file-listener.sh $1` &
Automating the Log Client-Part 1
usage () {
echo "usage: $0 <IP> [log port] [fn port] [ft port]"
exit 1
}
# did you specify a file?
if [ $# -lt 1 ] ; then
usage
fi
export RHOST=$1
if [ $# -gt 1 ] ; then
export RPORT=$2
else
export RPORT=4444
fi
if [ $# -gt 2 ] ; then
export RFPORT=$3
else
export RFPORT=5555
fi
if [ $# -gt 3 ] ; then
export RFTPORT=$4
else
export RFTPORT=5556
fi
Automating the Log Client – Part 2
# defaults primarily for testing
[ -z "$RHOST" ] && { export RHOST=localhost; }
[ -z "$RPORT" ] && { export RPORT=4444; }
usage () {
echo "usage: $0 <command or script>"
echo "Simple script to send a log entry to listener"
exit 1
}
# did you specify a command?
if [ $# -lt 1 ] ; then
usage
else
echo -e "++++Sending log for $@ at $(date) ++++\n $($@) \n----end----\n" | nc $RHOST $RPORT
fi
Automating Sending Files
● Listener on forensics workstation listens for file name● When a new file name is received
● Create a new listener to receive file● Redirect file to one with correct name● Also log in the main case log (optional)
● On the client side● File name is sent● After brief pause send file to data listener port
Automating the File Listener
usage () {
echo "usage: $0 <case name>"
echo "Simple script to start a file listener"
exit 1
}
# did you specify a file?
if [ $# -lt 1 ] ; then
usage
fi
while true
do
filename=$(nc -l 5555)
nc -l 5556 > $1/$(basename $filename)
done
Automating the File Client
# defaults primarily for testing
[ -z "$RHOST" ] && { export RHOST=localhost; }
[ -z "$RPORT" ] && { export RPORT=4444; }
[ -z "$RFPORT" ] && { export RFPORT=5555; }
[ -z "$RFTPORT" ] && { export RFTPORT=5556; }
usage () {
echo "usage: $0 <filename>"
echo "Simple script to send a file to listener"
exit 1
}
# did you specify a file?
if [ $# -lt 1 ] ; then
usage
fi
#log it
echo "Attempting to send file $1 at $(date)" | nc $RHOST $RPORT
#send name
echo $(basename $1) | nc $RHOST $RFPORT
#give it time
sleep 5
nc $RHOST $RFTPORT < $1
Cleaning Up
# close the case and clean up the listeners
echo "Shutting down listeners at $(date) at user request" | nc localhost 4444
killall start-case.sh
killall start-file-listener.sh
killall nc
Collecting Data
● Date (date)● Clock skew on subject● Time zone on subject
● Kernel version (uname -a)● Needed for memory analysis● Might be useful for researching vulnerabilities
Collecting Data (continued)
● Network interfaces (ifconfig -a)● Any new interfaces?● Strange addresses assigned?
● Network connections (netstat -anp)● Connects to suspicious Internet addresses?● Strange localhost connections?● Suspicious ports?● Programs on wrong ports (i.e malware on port 80)
Collecting Data (continued)
● Open files (lsof -V)● What programs are using certain files/ports● Might fail if malware installed
● Running processes (ps -ef and/or ps -aux)● Things run as root that shouldn't be● No login accounts logged in and running things● Might fail if malware installed
Collecting Data (continued)
● Routing info (netstat -rn and route)● Routed through new interface● New gateways● Conflicting results = malware● Failure to run = malware
Collecting Data (continued)
● Mounted filesystems (mount and df)● Things mounted that shouldn't be (especially tempfs)● Mount options that have changed● Filesystem filling up● Disagreement = malware
● Kernel modules (lsmod)● New device drivers● Modules that have changed
Collecting Data (continued)
● Who is logged in now (w)● System accounts that shouldn't be allowed to login
● Who has been logging in (last)● System accounts that shouldn't be allowed to login● Accounts that don't normally use this machine
● Failed logins (lastb)● Repeated failures then success = password cracked
Collecting Data (continued)
● User login info (send /etc/passwd and /etc/shadow)● Newly created login● System accounts with shells and home directories● Accounts with ID 0● Accounts with passwords that shouldn't be there
Putting It Together with a Script
usage () {
echo "usage: $0 [listening host]"
echo "Simple script to send a log entry to listener"
exit 1
}
# did you specify a listener IP?
if [ $# -gt 1 ] || [ "$1" == "--help" ] ; then
usage
fi
# did you specify a listener IP?
if [ "$1" != "" ] ; then
source setup-client.sh $1
fi
# now collect some info!
send-log.sh date
send-log.sh uname -a
send-log.sh ifconfig -a
send-log.sh netstat -anp
send-log.sh lsof -V
send-log.sh ps -ef
send-log.sh netstat -rn
send-log.sh route
send-log.sh lsmod
send-log.sh df
send-log.sh mount
send-log.sh w
send-log.sh last
send-log.sh lastb
send-log.sh cat /etc/passwd
send-log.sh cat /etc/shadow
Running the Initial Scan
Have I been hacked?
Who is Johnn?
/etc/passwd
Why do these accounts have passwords?
/etc/shadow
Who's been logging in?Results from last
Who failed to login?
Results from lastb
Looks Like They Were HackedNow What?
Live Analysis
● Use techniques described to● Grab file metadata for key directories (/sbin, /bin, etc.)● Grab users' command history● Get system log files● Get hashes of suspicious files
● Dump RAM● Must compile LiME (off subject machine!)● Risky – can cause a crash
Dead Analysis
● Unless the machine absolutely cannot be taken offline it is strongly recommended that you shut it down and get a filesystem image
● If you cannot shutdown the machine● You can still get a filesystem image with dcfldd● You probably cannot use this evidence in court
More on Dead Analysis
● Filesystem analysis is much more mature and powerful than memory analysis
● The Linux support in Volatility is somewhat lacking● Relatively new addition to a new tool● Seems to fall down a lot with late 3.x and 4.x kernels
● None of the investigators I've talked to could come up with a case where evidence existed only in memory
Finding Out More
● Heard there was a new book out (1 kg+ of knowledge)● Resources on http://linuxforensicsbook.com● Feel free to talk to me later
Questions?