Topic :Logging Topic :Logging

:Sunil:Sunil

ISQS 6342

Topics for Discussion Topics for Discussion

Introduction Introduction System Log ManagementSystem Log ManagementSyslog Syslog Syslog-ngSyslog-ngConclusion Conclusion

INTRODUCTION INTRODUCTION Logging and System AuditingLogging and System Auditing

Logging is the recording of per-specified Logging is the recording of per-specified events or actions performed on or by a events or actions performed on or by a system.system.

System Auditing is verification of all System Auditing is verification of all events and actions recorded in the events and actions recorded in the Logging process to search for system Logging process to search for system abuse, theft etc abuse, theft etc

System Log Management System Log Management Systems must have a comprehensive Systems must have a comprehensive

, accurate and carefully watched logs , accurate and carefully watched logs . Logs serve several purposes :. Logs serve several purposes :

Helps in troubleshooting virtually all types of Helps in troubleshooting virtually all types of application and system problems.application and system problems.

Provide valuable early warning signs for system Provide valuable early warning signs for system abuse abuse

Finally if anything goes wrong it provides crucial Finally if anything goes wrong it provides crucial forensic data.forensic data.

System Log Management System Log Management Threats :Threats :

Scope Scope The type of threats and attacksThe type of threats and attacks Potential unauthorized attacks forPotential unauthorized attacks for

Access InformationAccess Information Manipulate InformationManipulate Information Render Information systems Render Information systems

unreliable/unusableunreliable/unusable

System Log Management System Log Management

Risk :Risk : Accidental unpredictable exposure to InformationAccidental unpredictable exposure to Information Violation to system integrity due to malfunctionViolation to system integrity due to malfunction

Vulnerability :Vulnerability : Suspect Flaw in the system ( hardware or Suspect Flaw in the system ( hardware or

software)software) Attack :Attack :

A specific execution of a plan to carry out a threatA specific execution of a plan to carry out a threat Penetration :Penetration :

A successful attackA successful attack

AttackersAttackers

Penetrator Not Penetrator Not Authorized to use Authorized to use Data/Program Data/Program Resource Resource

Penetrator Penetrator Authorized to use Authorized to use Data/Program Data/Program ResourceResource

Penetrator Not Penetrator Not Authorized to use Authorized to use of computerof computer

Case A :Case A :

External PenetrationExternal Penetration

Penetrator Not Penetrator Not Authorized to use Authorized to use Computer Computer

Case B :Case B :

Internal PenetrationInternal Penetration

Case C :Case C :

Misfeasance Misfeasance

Threat RepresentationThreat Representation Installation Access Controls

Systems Access Controls

Data Program Resources

Legitimate User

Clandestine User(Defeats Logical Controls)

Masquerade(Defeats Procedural Controls)

External Penetration

Gaining Access to System Gaining Access to System External PenetrationExternal Penetration

An Outsider attempting to access information of an An Outsider attempting to access information of an organization that he is not a part of. organization that he is not a part of.

An Employee who has access to the premises but is not an An Employee who has access to the premises but is not an authorized computer user.authorized computer user.

An individual Taps a communication lines to work on the An individual Taps a communication lines to work on the system. uses trial and error method of logging in system. uses trial and error method of logging in

Gaining Access to System Gaining Access to System Internal PenetrationInternal Penetration

More Frequent More Frequent Has Access ( May or may not be Limited) Has Access ( May or may not be Limited)

or has over come the barrier to or has over come the barrier to unauthorized accessunauthorized access

Classified into 3 classes Classified into 3 classes ( shown in increasing order of difficulty of detection )( shown in increasing order of difficulty of detection )

The MasqueraderThe Masquerader The Legitimate userThe Legitimate user The Clandestine userThe Clandestine user

LEGITIMATE USERLEGITIMATE USER

A Case of Misfeasance A Case of Misfeasance Traits :Traits :

User with proper authorizationUser with proper authorization No extra use of resourcesNo extra use of resources User with proper authorizationUser with proper authorization Trails do not show Trails do not show abnormalabnormal patterns of patterns of

system usage .system usage .

LEGITIMATE USER LEGITIMATE USER

Not easy nor feasible to catch Foul PlayNot easy nor feasible to catch Foul Play

Audit trails :Audit trails :

Excessive Time spent Excessive Time spent Excess amount of Data/Program Reference Excess amount of Data/Program Reference

The MasqueraderThe Masquerader Internal userInternal user

External Penetrator who has succeeded in penetrating External Penetrator who has succeeded in penetrating the installationthe installation

An Employee without full access to the system An Employee without full access to the system An Employee with full access to the system who wishes An Employee with full access to the system who wishes

to exploit another authorized users identity that he to exploit another authorized users identity that he might have obtained.might have obtained.

He has a He has a legitimatelegitimate login and password so as far as the login and password so as far as the system is concerned he is a legitimate user.system is concerned he is a legitimate user.

There is no particular feature to distinguish the There is no particular feature to distinguish the masquerader from a legitimate used.masquerader from a legitimate used.

The Masquerader The Masquerader Interestingly defined by “ Interestingly defined by “ ExtraExtra ” use of system ” use of system

resourcesresources

Detection Trails :Detection Trails : Use outside of normal timesUse outside of normal times Abnormal frequency of useAbnormal frequency of use Abnormal Volume of data referenceAbnormal Volume of data reference Abnormal pattern of reference to programs or dataAbnormal pattern of reference to programs or data

The system focuses on the legitimate users resources as The system focuses on the legitimate users resources as protected all devices and access to the systems . protected all devices and access to the systems .

This extra usage is detectableThis extra usage is detectable

Clandestine User Clandestine User

Most difficult to detectMost difficult to detect

Seized supervisory controls of machine Seized supervisory controls of machine

Alters system to evade Audit Trail Data Alters system to evade Audit Trail Data

Operates below the level audit data is takenOperates below the level audit data is taken

Seen as “ The little man who isn’t there ”Seen as “ The little man who isn’t there ”

Clandestine UserClandestine User

Check for internal auditing mechanisms Check for internal auditing mechanisms Busy idle states of CPUBusy idle states of CPU Memory usage Memory usage Secondary storage etc Secondary storage etc

Counter Measures :Counter Measures : Operating System Code compared to a Operating System Code compared to a

reference versionreference version Provide audit trail to each major component of Provide audit trail to each major component of

machine machine

Tough to detect this pure phantom useTough to detect this pure phantom use

Characterization of Usage Characterization of Usage

In characterizing Computer Usage the main issue In characterizing Computer Usage the main issue to be faced is what unit or units should to be faced is what unit or units should represent system usage.represent system usage.

Notions are :Notions are :

Unit of Computer work – Job / Session Unit of Computer work – Job / Session Time parameters Time parameters Data Set and Program uses Data Set and Program uses Monitoring Files and DevicesMonitoring Files and Devices Group StatisticsGroup Statistics

Characterization of UsageCharacterization of Usage

Job or SessionJob or Session Job is for Batch RunningJob is for Batch Running Session is for interactive working Session is for interactive working

Both of these terms denotes a continuous and single Both of these terms denotes a continuous and single usage of computer working with well defined starting usage of computer working with well defined starting and ending and ending

The parameters that audit trail searches are the user The parameters that audit trail searches are the user identification, program accessed , Data Set accessed.identification, program accessed , Data Set accessed.

Case of Principal Parameter : Case of Principal Parameter :

Characterization of UsageCharacterization of Usage

Time Parameters :Time Parameters : Time of day in sense of date or DOWTime of day in sense of date or DOW

For many jobs the day is almost fairly For many jobs the day is almost fairly narrowed down.narrowed down.

Duration of Length of time the job takes .Duration of Length of time the job takes .

It is expected that users have patterns and It is expected that users have patterns and the intruders would disturb the patterns . the intruders would disturb the patterns .

Characterization of UsageCharacterization of Usage

For purpose of illustration :For purpose of illustration : We assume that ‘ A ’ data is average or We assume that ‘ A ’ data is average or

cumulative experience with the user in question .we cumulative experience with the user in question .we find the variability.find the variability.

Score = 24 Score = 24 Ē ( |A- À|Ē ( |A- À| ) )^̂2 it does show for 2 it does show for several users represented . Whose logon patterns several users represented . Whose logon patterns exhibit greatest variability exhibit greatest variability

USERUSER Score Score DurrettDurrett 00

SunilSunil 1212

Nitin Nitin 88

PradeepPradeep 141141

John John 4141

Not a very elegant method but depending on other measures and tools it can be more accurate

Characterization of UsageCharacterization of Usage

Data Set and Program Usage Data Set and Program Usage

This parameter differs significantly from This parameter differs significantly from system to system .system to system .

Program is significant from security point of Program is significant from security point of view as used in reading and writing. Almost view as used in reading and writing. Almost a clue of penetration activity. a clue of penetration activity.

Characterization of UsageCharacterization of Usage

Monitoring Files and Devices Monitoring Files and Devices Focuses on monitoring of the particular user identifier Focuses on monitoring of the particular user identifier

through a range of actions that he is allowed to through a range of actions that he is allowed to access that include submitting jobs and use of access that include submitting jobs and use of system etcsystem etc

Group Statistics Group Statistics Grouping records into sets having same properties Grouping records into sets having same properties

that refer to same job, session, user ,device, files, that refer to same job, session, user ,device, files, programs Datasets and other resources.programs Datasets and other resources.

Presumption that a session or job referring Presumption that a session or job referring to same file sets can exhibit similar to same file sets can exhibit similar properties from run to run.properties from run to run.

Linux :Syslog Linux :Syslog

Tired and true work horse of UNIX Tired and true work horse of UNIX Logging Utilities Logging Utilities

Accepts log data from Kernel (by way Accepts log data from Kernel (by way of Klogd)of Klogd)

A Preconfigured syslog is part of base A Preconfigured syslog is part of base operation systems of Linux and Unix operation systems of Linux and Unix

Configuring Syslog Configuring Syslog Acts on the Message type or facility and its Acts on the Message type or facility and its

priority priority The mapping of actions to facilities and The mapping of actions to facilities and

priorities are specified in /etc/syslog.confpriorities are specified in /etc/syslog.conf Each line specifies one or more facilities Each line specifies one or more facilities

followed by an actionfollowed by an action A facility or a facility with one priority A facility or a facility with one priority

constitutes a Selector constitutes a Selector

Sample syslog.conf File Sample syslog.conf File

mail.notice /var/log/mailmail.notice /var/log/mail

Syslog Syslog Facilities : Facilities :

Are simply categories they support Are simply categories they support Some of the facilities are Some of the facilities are

AuthAuth Used for many security events Used for many security events

Auth-privAuth-priv Access control messagesAccess control messages

DaemonDaemon System processes and other System processes and other daemons daemons

KernKern Used for Kernel messagesUsed for Kernel messages

MarkMark Messages Generated by syslog Messages Generated by syslog itself itself

UserUser Default facility when none is Default facility when none is specified specified

Local0-7Local0-7 Boot messages etcBoot messages etc

** All FacilityAll Facility

None None No facilityNo facility

Syslog Syslog Priorities :Priorities :

Are priorities assigned to the facilities these Are priorities assigned to the facilities these are hierarchical in increasing order of are hierarchical in increasing order of urgency urgency

Debug, info, notice, warning, err, crit , alert, Debug, info, notice, warning, err, crit , alert, emergemerg

The order of urgency is specified by the The order of urgency is specified by the programmer programmer

* and none can be used like facilities* and none can be used like facilities May be preceded by = or ! May be preceded by = or ! Specify only one priority per facilitySpecify only one priority per facility

SyslogSyslog

Priorities assignment Priorities assignment Mail.notice Mail.notice /var/log/mail1/var/log/mail1 mail.=noticemail.=notice /var/log/mail2/var/log/mail2 mail.!=notice /var/log/mail2mail.!=notice /var/log/mail2

Actions Actions Mostly all log message are written into files Mostly all log message are written into files Full file path must be specified in the lines’s Full file path must be specified in the lines’s

action in syslog.conf file action in syslog.conf file Messages can also be sent to other places . An Messages can also be sent to other places . An

action can be a file , a named pipe printer, action can be a file , a named pipe printer, device , remote host or a user screen .device , remote host or a user screen .

SyslogSyslog

Actions Actions Remote Logging is an important facility in syslog Remote Logging is an important facility in syslog Message that matches that line’s action will be sent Message that matches that line’s action will be sent

though UDP port 514though UDP port 514 Ex: Ex: *.emerg @mothership.mydomain.org*.emerg @mothership.mydomain.org The systems host’s syslog should have started with an –r The systems host’s syslog should have started with an –r

flag flag

You can specify multiple facilities in one line and one You can specify multiple facilities in one line and one action. The priority assignments is one per selector .action. The priority assignments is one per selector .

Can specify multiple selectors separated by ;Can specify multiple selectors separated by ; Ex: Ex: mail,uccp.notice; uucp!=alert /var/log/mailmail,uccp.notice; uucp!=alert /var/log/mail

Syslog Syslog

#sample syslog .conf file that sorts messages by mail kernel and other and #sample syslog .conf file that sorts messages by mail kernel and other and broadcasts emergencies to all logged in userbroadcasts emergencies to all logged in user

#print most sys. events to tty10 ad xconsole pipe #print most sys. events to tty10 ad xconsole pipe

Kern.warn;*.err;authpriv.none |/dev/xconsoleKern.warn;*.err;authpriv.none |/dev/xconsole

*.emerg **.emerg *

# send mail, news, and kernel and firewall msg to respective log files # send mail, news, and kernel and firewall msg to respective log files

Kern.* -/var/log/kernel_n_firewalllogfilesKern.* -/var/log/kernel_n_firewalllogfiles

Mail.* -/var/log/mailMail.* -/var/log/mail

# saves the rest onto one file # saves the rest onto one file

*.*;mail.none -/var/log/messages*.*;mail.none -/var/log/messages

Syslog -ngSyslog -ng

Similar to syslog-ng Similar to syslog-ng An attempt to increase syslog flexibility An attempt to increase syslog flexibility

and adding better filtering, forwarding, and adding better filtering, forwarding, message integrity and encryptionmessage integrity and encryption

Syslog-ng supports remote logging over Syslog-ng supports remote logging over TCP and UDP protocols TCP and UDP protocols

Can be used in conjunction with tunneling Can be used in conjunction with tunneling tools like stunnel and ssh to authenticate tools like stunnel and ssh to authenticate or encrypt log messages sent to remote or encrypt log messages sent to remote host host

Brain child of Balazs Scheidler Brain child of Balazs Scheidler

Syslog-ngSyslog-ng

Bit more involved in configuring Bit more involved in configuring syslog-ngsyslog-ng syslog-ng.conf and resides in /etc/syslog-syslog-ng.conf and resides in /etc/syslog-

ng/.ng/.

Defines sections like the Global Options , Defines sections like the Global Options , message Sources, message destinations message Sources, message destinations and message filters and combine them to and message filters and combine them to create logging rules create logging rules

Syslog-ngSyslog-ng Global Options { }Global Options { }

Are set in Global option{} section Are set in Global option{} section Options dealing with hostnames : chainhotsnames(), Options dealing with hostnames : chainhotsnames(),

Keep_hostnames(), use_fqdn(), use_dns() These deal with Keep_hostnames(), use_fqdn(), use_dns() These deal with soecifically with the hostnames of remote log clients not soecifically with the hostnames of remote log clients not with IP referenced inn the body of the message with IP referenced inn the body of the message

Ex : if syslog-ng-conf containsEx : if syslog-ng-conf contains Options { use_dns(yes) }Options { use_dns(yes) }Joe Bob whose IP address is 10.9.8.7 sends a message :Joe Bob whose IP address is 10.9.8.7 sends a message :

Sep apr 23 12:34:34 s_sys@10.9.8.7 sshd[13037]: Accepted publickey for ROOT from Sep apr 23 12:34:34 s_sys@10.9.8.7 sshd[13037]: Accepted publickey for ROOT from 10.9.8.254 port 1355 ssh2 10.9.8.254 port 1355 ssh2

The log records this message The log records this message

Sep apr 23 12:34:34 s_sys@joeBob sshd[13037]: Accepted publickey for ROOT from Sep apr 23 12:34:34 s_sys@joeBob sshd[13037]: Accepted publickey for ROOT from 10.9.8.254 port 1355 ssh2 10.9.8.254 port 1355 ssh2

Syslog-ng Syslog-ng

Options{ } :Options{ } :

Keep_hostname() : If set to no the syslog does not Keep_hostname() : If set to no the syslog does not take hostname supplied by remote log server at face take hostname supplied by remote log server at face vaslue . It instead resolves the source of IP packets for vaslue . It instead resolves the source of IP packets for itself to decide the hostname itself to decide the hostname

This is in contrast from syslog who accepts This is in contrast from syslog who accepts host name at face value.host name at face value.

Chain_hostname() : determines wether syslog list all Chain_hostname() : determines wether syslog list all hosts through which each message has been relayed – hosts through which each message has been relayed – set to yes set to yes

Syslog-ng Syslog-ng

LOG Message from host 1 to two other hosts LOG Message from host 1 to two other hosts Log entry in Host1:Log entry in Host1:

Apr 19 12:56:56 s_loc@linux syslog-ng[1656]: syslog-ng version 1.4.13 Apr 19 12:56:56 s_loc@linux syslog-ng[1656]: syslog-ng version 1.4.13 startingstarting

Log entry in Host2:Log entry in Host2:Apr 19 12:56:56 s_loc@linux/host1 syslog-ng[1656]: syslog-ng version Apr 19 12:56:56 s_loc@linux/host1 syslog-ng[1656]: syslog-ng version

1.4.13 starting1.4.13 starting Log entry in Host3:Log entry in Host3:

Apr 19 12:56:56 s_loc@linux/host1/host2 syslog-ng[1656]: syslog-ng Apr 19 12:56:56 s_loc@linux/host1/host2 syslog-ng[1656]: syslog-ng version 1.4.13 startingversion 1.4.13 starting

Syslog-ngSyslog-ng

Source { }: contains source() definition Source { }: contains source() definition which contains 2 drivers (message –inputs) , which contains 2 drivers (message –inputs) , may contain more drivers .may contain more drivers .

Source sourcelable { driver1([options]) ; driver2([options ]) }Source sourcelable { driver1([options]) ; driver2([options ]) }

Quite flexible and can accept message from Quite flexible and can accept message from various drivers syslog-ng itself ,UDP streams from various drivers syslog-ng itself ,UDP streams from remote hots , names pipes, TCP connections from remote hots , names pipes, TCP connections from remote hosts, and special files .remote hosts, and special files .

Ex :Ex :Source s_tcpmessage{ tcp( ip(168.197.135.168) port Source s_tcpmessage{ tcp( ip(168.197.135.168) port

(10514) ); };(10514) ); };

Source s_udpmessage{ udp();}Source s_udpmessage{ udp();}

Syslog-ngSyslog-ng

Destination { }: can send messages at the same Destination { }: can send messages at the same places as syslog ASCII files, names pipes, remote places as syslog ASCII files, names pipes, remote hosts via UDP or TCP and to other programs as input.hosts via UDP or TCP and to other programs as input.

Supports File Macros unlike syslog Supports File Macros unlike syslog File ( “filename( [ $MACRO ] “)File ( “filename( [ $MACRO ] “)

Filters { }: Optional . Allows you to route messages Filters { }: Optional . Allows you to route messages based on not only facility and priority as syslog does based on not only facility and priority as syslog does but also on the following but also on the following

the name of the program that sent it the name of the program that sent it The Host that forwarded it over the network The Host that forwarded it over the network A regular expression evaluated on the message A regular expression evaluated on the message

itself itself The name of an other filter The name of an other filter

Syslog-ngSyslog-ng

LOG statement LOG statement

Source s_loc { unix-stream(“/dev/log”); internal(); };Source s_loc { unix-stream(“/dev/log”); internal(); };

Source s_tcpmessages { tcp( ip(192.168.190.190) port(10514););};Source s_tcpmessages { tcp( ip(192.168.190.190) port(10514););};

Destination d_dailylog { file(“var/log/messages.$WEEKDAY”);};Destination d_dailylog { file(“var/log/messages.$WEEKDAY”);};

Destination d_micklog{ file( “/var/log/micklog” owner(mick) perm?(0600); };Destination d_micklog{ file( “/var/log/micklog” owner(mick) perm?(0600); };

Filter f_mail{ facility(mail); };Filter f_mail{ facility(mail); };

Filterf_message{ level(info..warn) and not facility(auth,authpriv, cron,daemon, Filterf_message{ level(info..warn) and not facility(auth,authpriv, cron,daemon, mail, news);};mail, news);};

Log { source(s_tcmessages); destination(d_micklog);};Log { source(s_tcmessages); destination(d_micklog);};

Log {source (s_loc); filter(f_mail); destination(d_micklog);};Log {source (s_loc); filter(f_mail); destination(d_micklog);};

Log { source ( s_loc ); mfilter(f_message); destination(d_dailylog);};Log { source ( s_loc ); mfilter(f_message); destination(d_dailylog);};

Automated Log Monitoring Automated Log Monitoring

You have recorded all log data You have recorded all log data Who read the log messages ?Who read the log messages ?

SWATCH DoesSWATCH Does A free log-monitoring utility written A free log-monitoring utility written

100% in perl .100% in perl . Install from Install from

http://www.stanford.edu/~atkins/swatchhttp://www.stanford.edu/~atkins/swatch

References References

James Andersons on Computer Threats and James Andersons on Computer Threats and security surveillance security surveillance

Hacking Exposed Hacking Exposed Stuart McClure Stuart McClure

Building secure Servers with LinuxBuilding secure Servers with Linux Michael D Bauer Michael D Bauer

Funny things happen !Funny things happen !