1 | P a g e

Peter McEldowney

Mark Waddington | Capstone CSNT 255

Outputting Event Log Events to a Remote

SQL Database Using PowerShell

Objective:

After completing this lab, the Administrator will have the Event Log from the computer of their

choice uploading information to a SQL database. The step-by-step outlines how to create the tables that

will be used to organize the data and will cover how to create a basic SQL script to create these tables.

Within the script to create the tables, indexes will be covered to ensure quick access to the data by a

particular column that is indexed and will also cover items to be aware of. After creating these tables, a

script to upload the event log items to the SQL server will be created so that the script can be run from

any machine to ensure scalability. After the script is created and tested, this step-by-step will go through

how to create a trigger that will run the script whenever a new event log entry is created.

Requirements: PowerShell 3.0, Microsoft SQL Server [w/ Management Tools]

Step 1: Create a diagram of the tables that will be used to store the data

Considerations when designing your tables:

a) What data is available? b) How will you identify the machine once the data is in the database? c) What log did the data come from and how will this be stored in the database? d) Is there an efficient way to organize the data to make retrieving the data quicker? e) What columns will people be searching from? f) What should the Primary Keys for the table(s) be?

g) What should the Foreign Keys be?

What is a Primary Key? A primary key is basically an index number. This is the unique number that identifies a particular row.

For example, if 2 rows have the same information, how would we identify which row is which?

What is a Foreign Key? Foreign Keys refer to data from another table. These are primarily used to link tables. For example,

say you have data in one table and the computer that uploaded the data in another. The foreign key

in the data table would reference an identifying key in another so that the data can be linked to the

associated machine.

2 | P a g e

The tables and relationships that are being made are referred to as an Entity-Relation Diagram. Not only

will creating this make building the SQL script to create the databases easier, but it will also allow for

future database users to access the data efficiently and know what data is available.

a) What data is available? The best way to identify the available data is by looking at the data. For

example, using PowerShell, we can pull event log data and look at what is

available.

Try this command in PowerShell to find the available data:

Ex: Get-EventLog –LogName System –Newest 1 | Format-List

Now we must decide what information we want to log in our database.

[Remember: Databases can grow to be massive so far large environment, it may be more efficient to

only take critical information. In this tutorial, we will be uploading all information because we can.]

3 | P a g e

We can actually call specific values that are defined by incorporating the text before the pipe | within

parenthesis () and then appending a period. to identify we want to call a value.

Ex: (Get-EventLog –LogName System –Newest 1).Index

Now since we have the information available, we can start grouping particular items together and

deciding how we want to arrange everything.

b) Identifying the original machine This can be done within the PowerShell script that will be created in Step 3.

Try this command in PowerShell for a preview on how this will be accomplished.

Ex: [Environment]::MachineName

c) Identifying the originating system log Different logs are kept for different aspects of the system. For example, DNS has

its own log. PowerShell has its own log, and so do System tasks. The available

logs on a system can be obtain by running the following:

Ex: Get-EventLog -LogName *

4 | P a g e

d) Organizing the Data for efficiency.

This step all depends on the volume of data that will be stored within the database. For

centralize Event Log management, it may be beneficial to store only EntryType,

Message, Source, and Username within 1 table so that less data is retrieved when

searching for errors.

An example is portrayed below. Notice how only crucial information is stored in the

primary table. All other data must be referenced by joining tables together by their

similar keys (foreign keys).

e) Frequent columns. Generally people will be searching for information based off specific columns.

These columns (like name of the computer, the type of entry, source, etc) are

columns that should contain indexes. An index is simply a tag put onto a column

so that it is monitored for efficient data retrieval.

One caution of using indexes is that every time new data is entered, the index

must be re-evaluated. As you can imagine, this can cause a great deal of

decreased performance and may even crash systems that are inadequate for

supporting the amount of data being uploaded.

Indexes are not required (and are not covered here) but are an important

aspect to consider if deploying on a large scale as it is much easier to

incorporate when designing than trying to implement during production.

f) Primary Keys

Primary key columns are (as described above) a unique identifying key. No 2

primary keys in a table can be the same or the data would be inconsistent. This

5 | P a g e

is why it is best to create a special column for the primary key and set it to auto-

increment. In the entity-relationship diagram below, you can see how columns

with PK are identified with some sort of text that is followed by_id

g) Foreign Keys FK’s should be easy to identify and associate with other tables. The most

difficult part is coding Foreign Keys.

One nice part about foreign keys is that you can set constraints on the data that

is referenced. For example, if we use the code exemplified below, then when

the referenced data in the other table is deleted, the delete is cascaded that the

table with the foreign key and all associated data is also removed. This helps

with normalization to ensure that all data that is stored is related to other data

and is not just fluff data.

evl_id_fk INT FOREIGN KEY (evl_id_fk) REFERENCES [evl_data] ON DELETE CASCADE

Step 2: Create the Database Script [This script should be based off the information outlined in your Entity-Relationship Diagram]

Considerations when creating the database script:

a) Identify what type of data will be stored in each column. b) Are there any columns where data entry should be mandatory? c) What are the constraints on the table and how are these foreign keys identified in the script?

Will a delete of data from one table cascade to the others?

How to create a SQL database from a script:

[Reference your diagram and use the tables below to help with data types and optional parameters]

6 | P a g e

The first item in order is to create the database. This can be done with CREATE DATABASE [dbname];

You must then USE [dbname]; to work with the database. This is demonstrated on Page 7.

Generic CREATE TABLE structure for Microsoft SQL Server (or T-SQL):

CREATE TABLE [table_name]

(

[column_name] [data type] [optional parameters] [table delimitated by a comma , to indicate next column]

index_number INT PRIMARY KEY IDENTITY,

other_table_index INT FOREIGN KEY (other_column_name) REFERENCES [other_table_name],

timestamp DATETIME NOT NULL,

logname VARCHAR(50) NOT NULL,

username VARCHAR(50)

)

What are Data Types?

A data type is what is held by that particular variable, column, etc. For example, if a column will

always hold a number, it will have INT (or integer) as its data type to limit what the cell can hold.

A field can also have DECIMAL(9,2) (where 9,2 represents up to 9 numbers and 2 decimal

points). Things like DATETIME are data types (DATE only stores the date but DATETIME stores

both depending on your needs). These can be found very easily through a quick search for

almost any programming language. Some of the most common data types for SQL are below:

Syntax Data Type Meaning (remember, variables are a number of your choice)

INT(n) Integer of n length An Integer is just a whole number (not a fraction)

CHAR(n) n Characters This is a fixed length field of length n

VARCHAR(n) Up to n Characters This is a variable length character field with a max length of n

DECIMAL(n,d) Number with a decimal Use this data type to specify a number of length n with the

number of decimal points to store as d

DATE Only stores date This field stores month, day, and year. Nothing more

DATETIME Stores date and time This field can store date and hour, minute, second, & split second

There are many more but these should handle your needs for this task.

What kind of optional parameters are available?

Parameter Meaning

PRIMARY KEY A reference number for a row (there cannot be 2 identical values of a primary key)

IDENTITY(s,i) Indicates that the value should be auto-incremented for each new entry [default (1,1)]

s = seed [or starting number], i = increment [the number the value goes up by]

FOREIGN KEY

(col_name)

Identifies a relationship to another table

(with constraints set, value must exist in another table)

(col_name) references a column in another table

REFERENCES

[table_name]

Comes after a Foreign Key to indicate the table that should be referenced when

associating the row with another table.

NOT NULL Specifies that a value must be entered into this column or an error will return.

CONSTRAINT Indicates that this column is referenced by another table [see below for an example]

I. Look at your Entity Relationship diagram and decide the data type for each field. The Entity Relationship diagram that has been converted into a SQL script to create tables:

7 | P a g e

The code for the servers, evl_primary, evl_data, and evl_timestamp table, would resemble:

Note: These must be executed query by query in SQL Server Management Studio (as an Administrator).

CREATE DATABASE server_event_logs;

USE server_event_logs;

CREATE TABLE [servers] (

pc_id INT CONSTRAINT pc_id_fk PRIMARY KEY IDENTITY,

pc_name VARCHAR(100) NOT NULL

);

CREATE TABLE [evl_primary] (

evl_id INT CONSTRAINT evl_id_fk PRIMARY KEY IDENTITY,

pc_id_fk INT FOREIGN KEY (pc_id_fk) REFERENCES [servers],

entry_type VARCHAR(50) NOT NULL,

message VARCHAR(1000) NOT NULL,

source VARCHAR(100),

username VARCHAR(100)

);

CREATE TABLE [evl_data] (

data_id INT PRIMARY KEY IDENTITY,

evl_id_fk INT FOREIGN KEY (pc_id_fk) REFERENCES [servers] ON DELETE CASCADE,

index_id INT,

category VARCHAR(100),

category_num INT,

replacement_str VARCHAR(100)

);

CREATE TABLE [evl_timest] (

time_id INT PRIMARY KEY IDENTITY,

evl_id_fk INT FOREIGN KEY (evl_id_fk) REFERENCES [evl_data] ON DELETE CASCADE,

instance_id INT,

time_gen DATETIME,

time_written DATETIME

);

8 | P a g e

II. Pay attention to columns that are defined with NOT NULL. These columns must ALWAYS have data inputted or the data entry will fail. This is to make monitoring data entry a

minimal task.

[Note: You can also code a database to upload error that happen in SQL but that is beyond

the scope of this step-by-step.]

III. Notice the placement of the ON DELETE CASCADE parameters. This means that if data is deleted from one of the associated tables, so will the entry in the evl_primary table. This

makes removing an event log entry much easier as the database will do it automatically.

IV. Also, notice the IDENTITY option for the primary keys. This option will auto-increment that column. How to perform this is explained in the table in the beginning of Step 2.

9 | P a g e

Step 3: Importing the proper SQL PowerShell module and testing connectivity

The first component of this step is to understand how to manage SQL through PowerShell. Once SQL has

been installed on a machine, the SQLPS module is made available. Let’s go ahead and import it:

We can check all the available Cmdlet’s and functions that are available with the following cmdlet:

Ex: Get-Command –Module SQLPS

Now since we know what we can do, we must understand what we want to do. This will become

more apparent the more you work with SQL as you will experience more applications for the various

available cmdlets (notice how data backups, replication, and availability groups can all be automated).

For basic queries, let us take a closer look at the Invoke-Sqlcmd cmdlet. Try the following:

10 | P a g e

Ex: Get-Help Invoke-Sqlcmd

We can find more specific information about each parameter with the –detailed switch appended.

Ex: Get-Help Invoke-Sqlcmd –detailed

Let’s try a basic query. We will insert data into the servers table about the computer that we are

sending the information from.

First, we need to define a variable that is a secure string so that it will pass the password

to the database properly. Otherwise, we will receive authentication errors.

We can do this with the ConvertTo-SecureString cmdlet:

Ex: $userpass = ConvertTo-SecureString –String password –AsPlainText –Force

We now have password stored as a secure string in the $userpass variable. Notice how if we

call the variable, we cannot see the password, we only get System.Security.SecureString

11 | P a g e

Next, we will define a variable that holds the Machine Name

Ex: $pc_name = [Environment]::MachineName

Notice how this is the same command we used in step 1, only

we have put the result into the variable $pcname.

Now if we use the Invoke-Sqlcmd cmdlet,

we can pass a SQL query to it with our

variable so that we can query the database.

When calling a cmdlet such as this, it is important to use the proper parameters with the cmdlet.

1. First we call the –Database parameter. This is used to signify what database on our SQL server we want to USE when we are sending queries.

2. The –ServerInstance parameter signifies the machine and Instance Name for our SQL server. This is the name that is

logged into when SQL Server Management Studio is loaded.

Check in the start menu under the SQL Server start menu group.

Start –> All Programs

–> Microsoft SQL Server

–> SQL Server Management Studio

3. Next, we are passing the System.Security.SecureString password that we defined in the variable above.

[Note: We cannot just pass text here or SQL authentication fails]

4. To finish the command, we must then pass a query to the server. For example, we can

“INSERT INTO servers VALUES (‘$pc_name’);”

Note the syntax that we use. We are INSERTing data INTO the table servers and the

VALUES we want to insert should be enclosed within quotation marks “, not

apostrophes ‘. Since the whole statement is in what are some called “double quotes,”

PowerShell interprets the statement as a string so PS will pass single quotes. PowerShell

will still pass variables with double quotes.

This means that we should be passing “INSERT INTO servers VALUES (‘ ‘);”

1 Invoke-SqlCMD -Database server_event_logs `

-ServerInstance SCVMM\ADK `

-Password $userpass `

-Query "INSERT INTO servers VALUES ('$pc_name');"

2

3

4

12 | P a g e

[For more information, check out the following (you must have updated your help

to view this documentation via Update-Help]

Ex: Get-Help about_Quoting_Rules | more

Step 4: Create the script that will INSERT the data into the appropriate tables

One of the most important parts in creating this particular PowerShell script is allowing the

passing of parameters to the script. This eliminates the need to customize multiple scripts for each log

and allows you to pass values to the script that can be used during data entry. For this particular script,

we will be using a [CmdletBinding()] object to pass the name of the log that the event is coming from.

1 [CmdletBinding()]

param(

[Parameter(Mandatory=$true,Position=0)]

[string]$logname = "System"

)

2

3

4

5

1. CmdletBinding indicates that advanced functionality is available within the script or cmdlet. For example, here we are using this to mandate passing a parameter.

2. ‘param(‘ is the opening statement of the beginning part of your script or function being created. 3. [Parameter(Mandatory=$true,Position=0)]

Parameter indicates that this value is passed as a parameter or switch.

Since Mandatory has the Boolean (1 or 0) value or $true (which is 1), this value must be passed

for the script or function to execute.

Position means that the person does not have to specify –logname to pass a value to the

variable $logname. The string that is passed for the variable must be in the absolute first

position, which is position 0 (because binary starts counting at 0).

4. [string]$logname = “System” Here we begin defining variables that are passed to the script or function.

[string] is the data type that the variable is stored as.

$logname is the name of the variable where the value is stored.

For example, if –logname “PowerShell” is passed to the script or function, then if

$logname is called, it will contain the string “PowerShell”.

[Note: We use double quotation marks to include everything included

within so that more than one word can be stored.]

= “System” means that the variable before the = will have “System” stored as the default string.

5. ) is simply indicating that it is the end of the parameters. More parameters can be specified but must be included before this end parenthesis specified here. Parameters must be separated by a ,

In the PowerShell script outlined below, notice how most variables have a specific data type. This helps

when inputting data to the database because then PowerShell will pass the proper data type to the SQL

database. One reason why all of the variables have data types is because I was receiving data type errors

upon entering data into the database.

As well, notice how # comments are added with a # pound sign # to indicate a comment/ignore this line.

Each section of code has explanations as to what is being performed in each section.

13 | P a g e

# set arguments that can be passed to the script #

1 [CmdletBinding()]

2 param(

3 [Parameter(Mandatory=$true,Position=0)]

4 [string]$logname = "System"

5 )

# ensure that SQL powershell module is available #

6 import-module SQLPS

# define variables #

7 $userpass = ConvertTo-SecureString -String 555erv333@ -AsPlainText –Force

# get pc_id for specific machine #

8 [string]$pc_name = [Environment]::MachineName

9 [int]$pc_id_select = (Invoke-SqlCMD -Database server_event_logs -ServerInstance STOUT\ADK -

Password $userpass -Query "SELECT pc_id FROM servers WHERE pc_name = '$pc_name';").pc_id

# check for pc_id, put it into variable if it is 0 (or non-existent) #

10 if ($pc_id_select -eq 0) {

11 Invoke-SqlCMD -Database server_event_logs -ServerInstance STOUT\ADK -Password $userpass -

Query "INSERT INTO servers VALUES ('$pc_name');"

12 [int]$pc_id_select = (Invoke-SqlCMD -Database server_event_logs -ServerInstance STOUT\ADK -

Password $userpass -Query "SELECT pc_id FROM servers WHERE pc_name = '$pc_name';").pc_id

13 }

# get event log into into variables #

#### table = evl_primary ###

14 [string]$ev_ent_type = (Get-EventLog -LogName $logname -Newest 1).EntryType

15 [string]$ev_mess = (Get-EventLog -LogName $logname -Newest 1).Message

16 [string]$ev_src = (Get-EventLog -LogName $logname -Newest 1).Source

17 [string]$ev_usr = (Get-EventLog -LogName $logname -Newest 1).Username

#### table = evl_data ###

18 [int]$ev_ind = (Get-EventLog -LogName $logname -Newest 1).Index

14 | P a g e

19 [string]$ev_cat = (Get-EventLog -LogName $logname -Newest 1).Category

20 [int]$ev_cat_num = (Get-EventLog -LogName $logname -Newest 1).CategoryNumber

21 [string]$ev_repl_str = (Get-EventLog -LogName $logname -Newest 1).ReplacementStrings

# input variables to evl_primary table #

22 Invoke-SqlCMD -Database server_event_logs -ServerInstance STOUT\ADK -Password $userpass `

23 -Query "INSERT INTO evl_primary (pc_id_fk, entry_type, message, source, username) VALUES

24 ($pc_id_select, '$ev_ent_type', '$ev_mess', '$ev_src', '$ev_usr');"

# input variables to evl_data table #

25 [int]$evl_id_sel = (Invoke-Sqlcmd –Database server_event_logs –ServerInstance STOUT\ADK –

Password $userpass –Query “SELECT evl_id FROM evl_primary WHERE pc_id_fk =

$pc_id_select;”).evl_id

26 Invoke-Sqlcmd –Database server_event_logs –ServerInstance STOUT\ADK –Password $userpass -

Query "INSERT INTO evl_data (evl_id_fk, index_id, category, category_num, replacement_str)

VALUES ($evl_id_sel, $ev_ind, '$ev_cat', $ev_cat_num, '$ev_repl_str');"

### table = evl_timest ###

27 [int]$ev_inst_id = (Get-EventLog -LogName $logname -Newest 1).InstanceId

28 [string]$ev_tim_gen = (Get-EventLog -LogName $logname -Newest 1).TimeGenerated

29 [string]$ev_tim_writ = (Get-EventLog -LogName $logname -Newest 1).TimeWritten

# input variables to dbo #

30 Invoke-SqlCMD -Database server_event_logs -ServerInstance STOUT\ADK -Password $userpass `

31 -Query "INSERT INTO evl_timest (evl_id_fk, pc_id, instance_id, time_gen, time_written) VALUES

32 ($evl_id, $pc_id_select, $ev_inst_id, '$ev_tim_gen', '$ev_tim_writ');"

Step 5: Set a trigger on the Event Log to upload the information upon an event

a) Go into Event Viewer This can be accomplish via run eventvwr.msc

or

This can also be accomplish through

�Control Panel

�Administrative Tools

�Event Viewer

15 | P a g e

b) Next, expand Windows Logs (or the log of your choice) so that you can view the log

c) Right click the Event Log of your choice and Select

� Attach a Task To this Log…

d) The Name and Description field are only locally

significant. This means that they are your choice

and have no bearing on the event.

e) Click Next because no information about ‘When an Event

is Logged’ can be changed as this was select

previously in the right click menu.

f) Next, we want to start a program because we have a PowerShell script to execute.

16 | P a g e

g) The next part is the most important part. This is where we choose how to run the script that we have created.

For the Program/script part, we want to call powershell

Next we need to know which arguments (or switches, or parameters) that we want to pass to

powershell.exe so that it will execute properly.

We can find these optional parameters by running powershell.exe /? from the command prompt.

[for the full output, run the command in any command prompt window]

17 | P a g e

As we notice from the previous output, we can bypass the execution of scripts. This is handy

because now we do not have to change the execution policy

-ExecutionPolicy Bypass

Next we notice how we can call a file. We will use this to point the script. In the syntax, it specifies

that we can use [–File ]

The square brackets indicate that it is an optional field.

Here is the syntax for the argument that we are passing.

We are passing “System” as the “Name of the Log” because

that is the log we are uploading information from.

-File C:\scripts\evl2sql.ps1 "Name of the Log"

Another switch to consider is the –WindowStyle argument. This will allow us to hide this task from

displaying on the screen whenever an event occurs.

-WindowStyle Hidden

To summarize, the final result that will go into Add Arguments box is:

-ExecutionPolicy Bypass –WindowStyle Hidden -File C:\scripts\evl2sql.ps1 "Name of the Log"

Pay attention to the order as PowerShell demands a certain order for the arguments passed.

This is found in the powershell /? output.

h) The final part is the click Finish. If there are other configurations you would like to perform, check the check box. If not, the Event Log trigger should be functional.

After Finish is clicked, a window opens that telling that Task Scheduler is where the event can be

viewed and edited in the future. This is covered in Step 6.

18 | P a g e

Step 6: Testing Your Setup to Ensure it is Functioning

Go into Task Scheduler. Here you can view the Event that was

previously created. [Run � taskschd.msc]

In the Task Scheduler, go down the hierarchy to

� Task Scheduler Library

� Event Viewer Tasks

The next part involves testing to

make sure that Events are properly

being uploaded to the Database.

This is done with SELECT queries to

the database.

This is where the foreign keys come

into effect.

If you return no results from your queries, it is time to troubleshoot the script.

Go into PowerShell and run your script.

For the script that I have placed in the location C:\scripts\evl2sql.ps1, I run:

powershell -ExecutionPolicy Bypass -File C:\scripts\evl2sql.ps1 "System"

Notice how you do not include the –WindowStyle Hidden argument. This is so we can see the output of

the script so that we can troubleshoot. The output is shown below.

19 | P a g e

The issue that is happening is on line 58, 59, and 70. We can see that on the 2nd red line of each item

returning an error. If we edit our script in PowerShell ISE, we can look for errors.

As for the general information uploaded to the SQL database, we can access it by using SELECT

statement below.

SELECT *

FROM servers, evl_primary

WHERE servers.pc_id = evl_primary.pc_id_fk;