SECURITY TRAINING AND AWARENESS SURVIVAL GUIDE

2

WHAT’S IN THIS GUIDEWe won’t beat around the bush here. Building a security training and awareness program is difficult.

The myriad moving parts, the fluctuating obstacles, both internal and external, the high stakes; all add up to a great deal of work for someone in your role. Even as technology improves, people are still at the controls, making your job not just important, but vital.

Enter our Security Training and Awareness Survival Guide.

We’ve collected four of our most popular resources into a comprehensive guide for those responsible for running security training and awareness programs.

From making the case with your bosses, to developing a simulated phishing program, building effective training, and tracking your success, this guide covers what you need to know as a training and awareness manager to build an engaging, behavior-changing initiative.

3-9 OVERCOMING 5 COMMON OBJECTIONS TO SECURITY AWARENESS TRAINING

10-19 10 STEPS TO A SUCCESSFUL SIMULATED PHISHING PROGRAM

20-26 5 TRAINING TACTICS FOR ACHIEVING BEHAVIOR CHANGE

27-32 7 METRICS THAT MATTER FOR AWARENESS TRAINING SUCCESS

3

Introduction

Overcoming 5 Common Objections to Security Awareness Training

You live your life immersed in security. You get it, and you have an enthusiasm for it.

This understanding extends to the vulnerabilities employees present as they go about their daily tasks. You know security awareness training is just as vital as any technical safeguard you could name.

Unfortunately, your enthusiasm for promoting security awareness within your organization isn’t matched by the executives signing the checks. To get them to not just consider security awareness, but to invest in it, you need to tie security goals with business goals, and be ready to answer common objections.

We’re here to help you address them.

In the pages that follow we look at five common objections we’ve heard to implementing security awareness training, explain why they’re wrong, and provide some fodder for answering these objections.

4

We Don’t Need Training; We’ve Never Had a Cybersecurity Incident

The Rationale: Our company has gone X amount of years without a data breach of any kind. We must be doing something right. I don’t see why we need something like security awareness training.

Why It’s WrongIf you’ve never had your home broken into, it’s easy to be flippant about remembering to lock your doors at night or to turn on your alarm system. If you’ve never been in a car accident, you might be inclined to drive a little faster than you should down the highway.

And if you’ve never been the victim of a security breach, you may think intrusions only happen to other people, or much larger businesses. But in all these situations, your actions (or inactions) may put you at greater risk for something to happen in the future.

Unfortunately, when it comes to the number of data breaches, trends aren’t on your side (just flip through the pages of the latest Verizon Data Breach Investigations Report). In the face of so many threats, your business must apply its due diligence to ensure that vulnerabilities are not left to fate. The human angle of these threats cannot be ignored.

Suggested ResponseWe’ve been lucky, it’s true, and here’s hoping our luck continues. But no business decisions should rely so heavily on luck.

Industry research like the Verizon Data Breach Investigations Report show that it’s not a matter of if a company experiences a data breach, it’s when.

In the face of so many threats, we must apply all due diligence to ensure that our vulnerabilities are not left to fate and the human side of cybersecurity is accounted for.

Our reputation and revenue could be on the line.

5

It Will Cost Too Much

The Rationale: We don’t have the budget to buy training from a vendor. We already spend enough on software and other tools to keep our data secure.

Why It’s WrongThere is indeed a cost associated with security awareness training. Teaching security-aware behaviors takes a focused training and reinforcement program, which is an investment your company leaders need to sign off on.

However, it’s not a bank-breaking investment. An effective security awareness program is likely to be fully covered for less than 1% of the typical enterprise IT budget.

Do you know what costs more than building a security awareness training program? Recovering from a breach! Industry research consistently sets the cost of just one data breach at around $4 million.

Focusing on the price alone ignores costs that can run many orders of magnitude greater than education. These costs will likely include, but are by no means limited to, loss of revenue, fines, and money spent to recover from the various ways a breach will damage a company.

Suggested ResponseAs with any investment, you must evaluate the risk and the return (as well as possible damages). Sometimes the greatest risk to a company like ours is doing nothing.

The investment is modest: an effective security awareness program is likely to be a fraction of our overall IT budget. We’ve already invested in technology to secure our networks and systems. Security awareness training will help our people use this technology effectively and follow company processes. Training helps ensure that the company gets the most from its technical investments in security.

Conversely, how much would it cost to replace all the customers that leave because we didn’t keep their data secure? Focusing on the price alone ignores costs that can run many orders of magnitude greater than education, like the direct, indirect, and opportunity costs associated with the true cost of data breaches.

6

Training Can’t Really Change Behavior

The Rationale: All sorts of companies who trained their employees still had data breaches or other cyber incidents. Employee training can’t be that effective.

Why It’s Wrong (Kind of)Some security awareness programs aren’t worth your employees’ time. You can’t always tell if the choices of training out there are any good, let alone if you can customize them to suit your needs or if you can reinforce the message on an ongoing basis.

Perhaps the most important element to look for when evaluating training is the vendor’s understanding of the art and science of online training that captures attention, engages the learner, and creates and sustains awareness. Good training keeps things interesting with a variety of media, lively interactions, and relevant content.

Good training also delivers more than just a one-time event. Adult learning experts agree that lessons delivered repeatedly in a variety of formats have a better chance of sticking. Training should also be tailored to a company’s unique needs and risks. The more relevant the training material is, the easier it is to learn.

Suggested ResponseYou’re right: sometimes training isn’t very effective. Many security awareness training initiatives fail to meet expectations.

But we’ve done our research. Just as we evaluate technologies, and appraise other business options, we have investigated and evaluated what has really worked to achieve the best results.

Our solution is designed to deliver behavior change right from the start. We will deliver and reinforce our message throughout the year.

We believe we can provide a best-of-class security awareness program that will help us achieve a security-aware culture, safeguard our customer data, and protect our business assets.

7

Technology is Enough to Keep Our Data and Systems Secure

The Rationale: We have spam filters, antivirus software, and other technical solutions in place. Those are nearly infallible and the best way to keep our data secure.

Why It’s Wrong“Security” to your boss likely means remembering not to disable the antivirus software installed on their computer. Security essentials like firewalls and antivirus software are good at what they do. So good, in fact, that the bad guys are taking the easy way in and focusing on what’s often the weakest link: the human. Phishing emails and other forms of social engineering are the most popular way for cybercriminals to get in for a reason.

You need to help your company’s decision-makers understand that a good security strategy is a multi-layer strategy—one that includes the human defensive layer in the form of an effective security awareness program.

Unlike technology (emerging AI not withstanding), employees can learn and incorporate new information into their daily tasks. Investing in security awareness means investing in them as part of the solution.

Suggested ResponseThese tools are good at what they do, but only at the levels they address: the network/host layers. Such technologies do not provide 100% coverage or any other guarantee.

The ongoing problem is that attackers circumvent these defenses and focus on the much softer targets: people.

A good security strategy is a multi-layer strategy—one that includes the human defensive layer.

Our employees should be thought of as our first and last lines of defense. Investing in them means investing in the security of our entire company.

8

We’re Too Small to Justify a Security Awareness Program

The Rationale: No cybercriminal would ever want to hack into our systems. We’re too small of a company to be a valuable target.

Why It’s WrongNo company is too small to be a target. Attacks are built to target vulnerabilities, not specific businesses or individuals. If you’re a small- or medium-sized business relying on common software, you are equally as (if not more than) at risk as companies five times your size.

The best, and perhaps only, way to effectively address your human-side problems is with training and reinforcement that hits on your most pressing risks. Even a relatively bare bones approach to awareness, with a few general training topics teamed with reinforcement, is better than letting employees fend for themselves. A training initiative can sound like a lot of work for a small business with limited resources, but with the right vendor, it doesn’t have to be. If additional training or learning reinforcement content is needed, a vendor should be able to provide this, too. An all-in-one approach keeps overworked IT staff from having to wrangle multiple vendors, each with a different piece of the awareness puzzle.

Suggested ResponseTargeted attacks are a risk for businesses of all sizes—no one is immune.

Industry research bears this out. Consider the findings from the 2019 Data Breach Investigations Report that found 43% of breaches involved small business victims.

Some industry experts even believe smaller businesses are more of a target because they’re more likely to have over-taxed IT staff guarding the digital gates.

We believe we can provide a cost-effective security awareness program that will help us safeguard our customer data and protect our business assets. By doing so, our company can keep focused on the important tasks of achieving our business goals and meeting our customer commitments.

9

Conclusion

If you made it all the way to the end of this white paper hoping for summary of the whole thing, you’re in luck (though we’d recommend going back and reading for more detail). In brief, the five objections and suggested responses we covered were:

Objection: We don’t need training; we’ve never had a breachAnswer: Never having been in a car accident is no reason not to wear a seatbelt. Leaving the human element of cybersecurity unaddressed is an unnecessary risk.

Objection: Training costs too muchAnswer: A data breach or other cyber incident would cost exponentially more, from fines to lost revenue.

Objection: Training can’t really change behaviorAnswer: The right training can change behavior. Just as we vet any other vendor, the training providers we seek out will be best of breed.

Objection: We have technical safeguards, no training is neededAnswer: The modern cybercriminal is targeting our employees directly to bypass our software safeguards.

Objection: Our company is too small to be a target for cybercriminalsAnswer: No company is too small to be breached, and our employees are as much of a target as anyone working for a company five times our size.

Overall, a key thing to keep in mind when making the case for a security awareness initiative is that security awareness is not a company goal in its own right. Security awareness programs are tactics to help achieve the larger goal of reducing the risk and potential impact of certain types of security breaches. Your bosses are likely not human behavior experts. They are looking for guidance in identifying what types of cyber incidents can be prevented or reduced via training. Keep these larger goals in mind when pitching any security awareness initiative and you’ll stand a better chance of success.

w

10

WHY PHISHING MATTERSSomeday, maybe on a day just like today, your employees will get a phishing email.

Email clients are getting better and better at filtering them out, but they’re not perfect.

Phishing is an attack vector that shows no sign of slowing down. If you’re reading this to get some advice on running a simulated phishing program, you probably don’t need this proven to you. Here are some stats anyway.

28%of U.S. employees

admitted struggling to identify a

phishing email (2020 State of

Privacy and Security Awareness Report)

94%of malware is

delivered by email (CSO Online)

10 STEPS TO A SUCCESSFUL SIMULATED PHISHING PROGRAM

11

SIMULATED PHISHING DEFINED

1 PREP YOUR COMMUNICATIONS

PLAN

2 DEFINE YOUR METRICS

3 INFORM LEADERSHIP

4 SEND YOUR BASELINE PHISH

5 ANNOUNCE THE PROGRAM

6 TELL OTHER DEPARTMENT

HEADS

7 PROGRAM LAUNCH AND ESCALATION

8 SUPPORTING COMMUNICATIONS

9 DIGGING INTO THAT DATA

10 PROFIT

Enter the simulated phishing program, an important way for employers to see how vulnerable their people are to this social engineering attack and train them to do the right thing with the real thing.

The goal of a phishing simulation program is to provide employees with a safe, simulated environment where they can learn about what real phishing attempts look like in the wild.

But what makes a good phishing simulation program? How many emails should you send and how often? How difficult to identify should they be? How do you track improvements?

We’ll walk through the steps to take to establish a simulated phishing program, from developing a communications plan and sending your first baseline phishing test, to building an ongoing program integrated into a larger security training and awareness initiative.

A phishing simulation program shares a goal with your primary training program: to teach.

It shouldn’t feel like a “gotcha” moment, or an attempt to make your employees feel stupid. The point is to make them feel like you’re all working together toward keeping your organization’s digital infrastructure and sensitive data safe.

12

Step 1: Prep Your Communications PlanYou should have a plan for how your simulated phishing program will flow squared away before you dive in. At the very least, this will make it easier to lay out your initiative for your executive team and specific department heads (more on that soon).

Points to cover in your communications plan should include:

The best programs we’ve seen have common branding carried throughout their phishing educational content. Here we mean giving your program a catchy name; one that your people will see and instantly associate with it. We’re fans of plays on words (something like “Phresh Phish of the Day), but the possibilities are wide open.

A catchy name teamed with consistent colors and even font choices for your phishing communications helps engage your employees and makes clear the importance your organization places on this threat and the educational content behind it.

Step 2: Define Your MetricsThe Big M; metrics will tell you and your company if your program is getting results.

Put simply, the most important thing to track is how often your phishing emails get reported: the report rate.

Click rates are often touted as the primary metric of simulated phishing success, but these can be too easily manipulated by tweaking the difficulty of phishing campaigns. If click rates are too low, then you’re not sending tough enough phishing emails.

Report rates help demonstrate your ultimate goal: engagement. You want people to tell you if they think they received a phishing email; simulated or not. The more they report, the more engaged your employees are.

Long story short: click rates are useful, but report rates are vital.

• Frequency of simulated phishing email campaigns

• Supporting educational content you plan to include (articles on the company intranet, supporting graphics, etc.)

• Messaging for announcing the program companywide and employee-facing instructions for how to report a suspicious email

13

Step 3: Inform Leadership Your bosses need to know what you’re doing and why. This is where you’ll be thankful you worked out a communications plan.

If your leadership is still on the fence about initiating a simulated phishing program in the first place, use data to quantify just how big of a problem phishing is and what the risk is to your organization. This can come in the form of suspicious emails blocked by your email client or malicious downloads prevented (IT will be your good friends here, as they should always be).

Step 4: Send Your Baseline PhishBefore you launch your full program, you’ll need to send a campaign without telling the company. Only your IT help desk should know.

Why all this secrecy? Keeping this first campaign under wraps is the best way to gauge your people’s everyday susceptibility to phishing emails. They won’t be expecting a test, meaning they’ll be just as vigilant (or not) as they usually are.

Establishing initial reporting percentages and click through rates is important to show how your primary simulated phishing and training initiative has improved behaviors later on.

The first simulated phishing email should not be too easy, but not too hard either. Consider something like a phony package shipping confirmation or a new voicemail announcement. The link should lead to a simple 404 page.

Again, the point of this first campaign is to simply collect a baseline of clicks and reports.

14

Step 5: Announce the Program Wait, didn’t we just say the simulated phishing campaigns you’re running should be secret?

Well, yes and no.

The baseline phishing email should not be public knowledge to glean as true an assessment as possible of your organization’s susceptibility to phishing.

But after you get a baseline, your full, multi-month program should be formally announced to all employees. In fact, you should over-communicate about the problem to avoid an impression that the program is a test or that you’re trying to trick anyone. Communicate that the program is educational — it’s training.

This announcement should include some key elements:

• The simulated phishing program is part of the company’s ongoing security training and awareness initiative

• Advice on what makes an email suspicious in the first place

• How to report phishing emails (many simulated phishing platforms have reporting buttons that can be integrated into business email clients)

• Where to find additional company resources on phishing (such as your company’s intranet)

15

Step 6: Tell Other Department Heads (and Heads Only) Inform other departments within your organization if you’re planning to spoof emails from them (we’ll talk more about using a variety of spoofed emails later).

Hackers are going to send your employees emails that appear to come from individuals in your organization, like a CEO looking for an urgent wire transfer, or departments, like HR, asking for a quick turnaround on personal information.

Give these teams a heads-up before the phishing simulation campaign, so you can make sure that you’re not interrupting their normal work with a flood of worried emails from your employee population. This should include:

• A brief explanation about why you’re running a simulated phishing program

• Instruction that if they get questions about suspected phishing emails from their own people to tell them to report the emails to IT

16

Step 7: Program Launch and Escalation You’ve stated your intentions companywide, now it’s time to launch your program.

We recommend no more than one campaign per month but at least once per quarter. Your baseline phish should be used to inform the sorts of emails you send out first.

Lots of people caught? Start easy and slowly turn the dial.

Impressive report rates? Praise the people who report, and then make the primary campaigns a little tougher, but keep thanking those who report.

Do a little digging to see if specific departments or locations did better or worse than the average. People like to see how they compare so you might eventually consider reporting on a department basis for department heads only.

Here are some ideas for types of phishing campaigns to run:

Long story short: Get creative! You’ll eventually want to “turn the dial” on your campaigns’ complexity to make sure your employees are continually challenged. Most simulated phishing solution providers will include multiple phishing templates built in, sometimes even with the ability to build your own from scratch.

Important thing to remember: No matter the phishing emails you concoct, make sure you send your IT team screenshots beforehand so they know what’s part of your program. This way if users forward simulated phish to them and ask what to do, they can tell them to report it with the report button (without giving away that it’s a simulated phish).

• Password reset requests

• Shipping notifications around the holidays

• Requests purporting to be from HR (again, inform your HR director before you do this) concerning W2s around tax time

• Spear phishing campaigns targeting specific departments or even positions (wait un-til at least you’re three or four campaigns in, though, as spear phishing is a big jump in complexity)

17

Step 8: Supporting Communications Call It reinforcement or awareness, no simulated phishing program is complete without supporting content outside of the emails themselves.

These can include everything from eye-catching infographics to short articles and videos posted on your company intranet. Occasional reminders to all employees about how to report phishing emails are also useful to send, interspersed with the simulated phishing emails themselves. Last but not least, specific web pages people who click simulated phish get sent to should be educational and supportive. Again, we’re not going for “gotchas” or scolding.

This content should be tied into your larger training and awareness initiative whenever possible. Try to achieve a similar look and feel to help your people mentally connect the varied training content you’ve deployed.

Simulated phishing programs are useful, but you shouldn’t rely on them alone to influence behavior change. Industry analysts at Gartner say as much in their report Innovative Insight for Anti-Phishing Behavior Management:

(For some inspiration, we’ve got a toolkit of free resources all about the dangers of phishing.)

“Anti-phishing behavior management solutions are not a tool for initiating cultural change. Assess your organizational culture first, and deploy anti-phishing as part of a comprehensive program of security behavior management and education.”

18

Step 9: Digging Into that Data So your phishing program is up and running.

Now what? It’s time to look at the data.

First of all, congratulate the people who reported! This can be as simple as a friendly “Thank You” pop-up connected to your email client’s phishing report button.

Also consider providing rewards for those who consistently avoid phishing attempts, such as gift cards, security swag, or a lunch on the company. Everyone loves free food!

If you have a large organization, you can enter all the names of the people who reported phish into a drawing every month. You can also give kudos on your security portal or company newsletter to people who reported REAL phish.

If you have the resources, send them a personal thank you and cc their manager. A little positive recognition can go a long way. You just might find a future security ambassador in the process.

For the people who consistently fell for your spoofed emails, follow up with a short training course and track their completion. In MediaPRO’s own Phishing Simulator, targets who click on phishing lures see a teachable moment on the landing page that’s displayed to them. But you can also configure the campaign to automatically enroll them in a short training course. In the Learning Management System (LMS), set a due date for the training and automate the training reminder emails that are sent to employees until the training is completed. Remember to keep the tone of the reminder email upbeat and helpful.

While exploring the data, look for patterns. Try to find what signals the data might be send-ing. Are repeat clickers more common in a specific department? In a specific geographical region? Bosses or “rank-and-file?” This information can be used inform both additional phishing campaigns and training and awareness materials.

Seeing total clicks go down is a common indicator of improvement, but as we’ve said, user reporting is the most important metric in your program. It’s an indicator of engagement, which is exactly what any training and awareness manager wants.

19

Step 10: Profit Yeah, so we’re referencing an internet meme that dates all the way back to 1998 as our last point. But if you made it this far, we figure you could use a laugh.

In all seriousness, though, a thoughtful simulated phishing program, tied to other security training and awareness elements, will pay dividends.

A program built with engagement in mind is a big step toward establishing a security culture in your organization.

An engaged employee will say something when they see something, will tell their coworkers about it. That’s how culture spreads. And that sort of thing is priceless.

20

Introduction

5 Training Tactics for Achieving Behavior Change

Imagine all your employees in a single room waiting for cybersecurity and privacy training to begin. You’ve called them together because you know the value training can bring.

You’ve done the research and discovered, for example, that “Companies that train their employees in information security best practices spend 76% less on security incidents than their non-training counterparts,” according to a 2014 PWC report. That difference amounted to $521,000 in lost revenue on average.

As you stand before your gathered employees, your mind starts to race. “Is what I’m about to present going to work? Is it going to be enough?” Other research you’ve read on training effectiveness comes to mind. A joint Ponemon Institute/Experian survey found 55% of companies with security and privacy training had suffered a data breach or security incident due to malicious or negligent employees. This same survey found that many respondents felt their organization’s

training lacked the ability to effect real behavioral change. Forty-three percent said, for example, that their organization offered just one basic course meant to apply to all employees. Long story short: if you’re using the wrong training, you won’t get the results you want.

We’re confident we don’t have to extoll the benefits of employee awareness training that teaches cybersecurity and privacy best practices. Indeed, we’re happy to see organizations of all types place a greater focus on the human side of cybersecurity and data privacy over the last few years.

But, as the reports cited above show, not all training is created equal. Organizations both large and small continue to struggle with filling the human-shaped holes in their security and/or privacy strategy. In our previous “Best Practices” white papers, we discussed designing a planning roadmap for your awareness efforts and using data to pinpoint what training you’ll need.

Long story short: if you’re using the wrong training, you won’t get the results you want.

21

Introduction (continued)

Now the rubber needs to meet the road: making sure your existing training is as effective as it can be. But what makes training effective?

We’re well aware that changing employee behavior is no easy task. Fortunately, there are some established ways to get a foothold. In this white paper, we’ll discuss some awareness training best practices we know lead to real behavior change:

• Getting users motivated

• Creating “Social Presence” to heighten engagement

• Providing interactive practice through all stages of training

• Using gamification to increase realism

• Managing training complexity

At MediaPRO, we have 20-plus years of training expertise honed from working with some of the most risk-conscious organizations in the world. Read on for some of the fruits of our experience.

22

Get Users Motivated

You may expect to see an e-Learning best practices white paper on training open with advice on juxtaposing multiple types of media or arranging the look on the page.

But as Ruth Clark and Richard Mayer write in E-Learning and the Science of Instruction, the most fundamental lesson from e-Learning research is that learning is learning no matter the media. And the first job for any kind of training is to get users motivated, to get them to engage.

The most potent motivation comes from factors inside the learner or inherent in the task they’re engaged in. So-called intrinsically motivated learners are more likely to process information in effective ways and achieve at high levels. So, the first step in motivation is getting the users’ attention.

The short path to motivation runs through identification.

In order to hold your learners interest, you need to establish how the training content is relevant to them and how paying attention pays off for them. You can do that by addressing the student directly, or by presenting characters they identify with.

Once you’ve gotten their attention, you’ll want to build their interest. A good way to do that is to show them that their actions at work have consequences, not only for the company, but also for them.

You can do that, for example, by putting the characters they identify with in a situation that presents them with choices that have significant personal consequences.

The first step in motivation is getting the users’ attention.

23

Heighten Engagement by Creating “Social Presence”

Motivation comes not only from the content you present, but also from the way you talk to students.

You’ll see a lot of training adopt a third-person voice (“To increase security, the ‘strict’ option is preferable,”), likely because use of third person in academic circles lends an air of objectivity and authority.

The very objectivity of the voice, though, can waste the motivation we’ve been building. That’s because learning, even e-Learning, operates at a deeper level when the learner experiences it as a social encounter. Students often experience the third-person objective voice as a disembodied voice, leaving your learners less engaged and less likely to identify with characters you present.

The better approach for training is to address the user directly. You’ll find that users parse second-person writing more easily, but that’s not the only reason to prefer it.

Writing in a conversational tone and addressing the reader in second person (“You are our number one defense”) triggers ingrained, unconscious, social conventions that cause learners to invest attention. Essentially, they react to the training as though it was a person talking to them and expecting a response. The result: learners engage at a deeper level.

You’ll augment that sense of social presence by combining a conversational tone with the stories and characters we talked about earlier in this section.

Returning to Clark and Mayer, making the “author” visible through story and images adds additional cues that encourage the learner to deepen their engagement with the content.

Writing in a conversational tone and addressing the reader in second person (“You are our number one defense”) triggers ingrained, unconscious, social conventions that cause learners to invest attention.

24

Provide Practice Throughout the Stages of Training

We all know that if we want users to recognize information, we need to show them examples.

The best way to learn a skill is to practice it, not just to read about it.

Learners learn by doing, so it’s important that their exploration and practice be as true to reality as possible. If we want employees to recognize security features on a bank card, for example, we show them the features on the card. If we want them to tell the difference between sensitive and non-sensitive information, we ask them to separate the two into different folders. But there’s more to practice than that.

In the past, the e-Learning industry has thought of practice as an end point, the last step in a cascade which starts with an introduction and ends with a test. But research shows that practice doesn’t have to work that way. Practice accomplishes varying ends depending on when and how you present it in the course.

Sometimes practice makes sense as part of the initial learning. That can be true when the content is simple. Introducing small challenges can combat “cognitive miserliness,” the tendency for brains to conserve energy by attending less closely or falling back on heuristics to solve problems.

Practice can work well even when it’s staged as a pre-test, literally a quiz students take before they’ve seen the content of the course. You might think it’s unfair or counterproductive to pre-test. In fact, students get three benefits from getting practice before being exposed to the content.

First, as Clark and Mayer point out, students are notoriously poor (OK, terrible) at gauging their own ability. Graded practice before the lesson can alert them to holes in their learning so they pay attention when it makes sense. Second, pre-tests give students who already know the information a chance to test out of sections so they can focus on areas where they need help. Third, pre-testing gives students a preview of the content and vocabulary that will feature in the lesson, so they begin to build cognitive structures to fit the content into when they encounter it.

Learners learn what they do, so it’s important that their exploration and practice be as true to reality as possible.

25

Heighten Realistic Practice with Relevant Gamification

Gamification seems to be all the rage these days.

The term has found its way outside of purely e-Learning circles into the “real” world and is rising in popularity for a variety of uses once not even dreamed of.

Yet, gamification is a double-edged sword. Done well, it can make training much more inviting. Done poorly, it can literally waste your project budget by teaching the wrong skills.

The classic example is the rise and fall of the Oregon Trail, an educational game that featured pioneers moving their families out west. Instead of absorbing lessons about Westward Expansion, many students abandoned the instructional objectives, choosing to load up on virtual bullets and take down as many buffalo as they could.

Good game design that makes a difference in workplace performance compresses realistic job problems into a short timeframe in a safe setting where learners can succeed and fail.

As our friends Clark and Mayer write, learning will get more reinforcement when it becomes essential to progressing through the simulation. You can facilitate this by weaving instructional objectives into the flow

of the simulation.

Consider, for example, our own Catch the Phish game, designed to test a user’s ability to tell a scam email from a legitimate one after receiving phishing training. The game invites users to sort received emails into a “Keep” folder or the trash, based on a number of attributes. If a phishing email is treated as legitimate, a warning window pops up with a brief description of what was suspicious about the email. Too many of these missed phishing emails, and users have to start the game again.

Games and simulations are more effective when they include explanatory feedback rather than a simple correct or incorrect. Explanatory feedback works well as either a hint or as feedback to learner responses.

Good game design that makes a difference in workplace performance compresses realistic job problems into a short timeframe in a safe setting where learners can succeed and fail safely.

26

Manage Complexity

A current fad in instructional design is to create “discovery” spaces that are rich in interactivities with little direction.

The intention here is to create a rich environment. But in practice, complex environments are often inefficient and can contribute to overload.

According to Clark and Mayer, the more effective practice is to sequence content by starting with a simple task that has a low level of challenge and only partial functionality enabled. Then, progress to tasks that have more information and demand more skill or knowledge.

And complexity isn’t just a matter of the number of images on a page, but the detail in those images as well. We often work with subject matter experts whose initial position is that photorealistic images are more appropriate for “serious” content while hand-drawn or simplified images should be reserved for less important topics.

In fact, highly realistic images and sounds are busy and distracting, particularly for

novice learners. Clark and Mayer say the best practice is to minimize realism that isn’t aligned with an instructional objective.

Of course, there’s abundant literature on the way to deliver multimedia content, the number of channels to use simultaneously, and how to manage it. And you’ll find plenty of articles on the web discussing those issues. Before you get into those details, though, consider the following.

Graphical and audio design are important. They underline and clarify. They direct attention. But only after you have the attention first, and only for learners who are interested and motivated.

Shooting phishing email in an arcade is fun. But it alone doesn’t build your employees’ confidence or capability to make your information or your customers more secure.

In awareness training, poor design impacts your bottom line and your reputation in the marketplace. It doesn’t matter how pretty your screens look if your employees aren’t engaged and taking your training to heart. Substance and appearance must walk hand-in-hand to effect real behavioral change.

In awareness training, poor design impacts your bottom line and your reputation in the marketplace.

27

7 METRICS THAT MATTERHOW TO SHOW YOUR AWARENESS INITIATIVE IS WORKINGAs a security professional, proving the worth of the work you’re doing can be a challenge.

Those tasked with managing awareness programs often ask the question: “How can we show that security and privacy awareness training is worth the money?”

Whether you’re just starting your journey toward establishing an awareness initiative or looking to upgrade an existing program, setting measurable goals for behavioral improvements is crucial.

Either way, the stakes are high.

Lack of ROI for awareness training can lead to funding cuts when budget season rolls around.

Fortunately, there are clear ways to track effectiveness and set the stage for a successful awareness training initiative.

Here are seven metrics that help articulate the benefits of training to measure your hard work.

MEASURING AWARENESS TRAINING SUCCESS

11223344556677

TRACKING BEHAVIOR

REPORTED INCIDENTS

PHISHING

REMEDIATION COSTS

EMPLOYEE ASSESSMENT

EMPLOYEE TRAINING

LISTEN TO EMPLOYEES

28

WHAT TO DOWork with your IT team to set a baseline by logging risky events prior to your training event. A few months after your initiative starts, revisit these numbers to see if logged events have decreased.

1. Tracking Risky Employee Behavior (With IT’s Help)Chances are your IT department has systems in place to track employee behavior in the form of network event and data loss prevention logs. There are a variety of software pro-grams that might be running behind the scenes of your corporate network, such as SIEM, DLP, or UEBA, that can aid in monitoring risky behavior.

HELPFUL PROGRAMS FOR TRACKING RISKY EMPLOYEE BEHAVIOR

SIEM

Security Information and Event Management systems collect network event logs, such as unsecure login attempts, virus scans, and other security-related documentation for analysis.

DLP

Data Loss Prevention software monitors the transmission of sensitive information to make sure an employee doesn’t send it to unauthorized destinations.

UEBA or UBA

User and Entity Behavioral Analytics tools are a way to parse information collected by SIEM and DLP systems and provide IT professionals prioritized trend information.

The information these systems collect can serve as a gauge for determining which employee behaviors are putting your organization at risk.

SIEM

DLP

UEBA

29

After your primary training has run its course, review these numbers to see how they changed. The goal is to see an increase in the percentage of phishing emails reported and a decrease in clicked phishing emails (ideally to zero!).

If you’re deploying a simulated phishing tool as part of your awareness efforts, the metrics above still apply. Such a tool should provide a low-lift way of setting a baseline for phishing susceptibility before training by means of a simulated phishing campaign launched before training begins. After training has been delivered, run subsequent campaigns to see how em-ployee behavior around phishing emails has improved.

2. How Often Incidents Are ReportedThough technology plays a key role in security programs, employees are vital to your company’s information security posture. No network monitoring system can spot confidential information left by the printer or deter non-badged visitors from gaining access into a secure area. Your company should have procedures in place that allow em-ployees to report suspicious incidents.

WHAT TO DOIf your company has incident reporting procedures in place, how to report a suspected phishing email is likely one of them. Collect numbers on frequency of reported phishing emails vs. phishing emails not reported to develop a reported percentage. Clicked phishing emails should also be included in this initial data gathering to help set a baseline of your employee’s profi-ciency with recognizing and correctly addressing this threat.

3. Reported Phishing Email Percentage Spotting the dreaded phishing email and knowing who to tell about it is a specific type of incident reporting. But given how frequent these attacks are, this metric is best pulled out and recorded on its own.

WHAT TO DOReview the frequency of reported incidents before training begins. Check if these reports increase as training progresses and in the months following. More reported incidents mean your employees have developed sharper eyes for suspicious activity (not necessarily that more incidents are actually happening!).

Consider combining this information with SIEM or DLP data to identify decreases in how long it takes for a security incident to be detected (called “time to detection”). Also, look for increas-es in the number or percentage of breaches detected and resolved before any harm occurs.

30

5. Direct Assessment of Employee KnowledgeAssessing employee knowledge through surveys is a direct way to measure what they know about security and privacy best practices.

WHAT TO DOIn designing a survey, create questions that address your organization’s most pressing security and privacy risks. Connect with your HR department to help deploy your survey. HR may already use a survey tool for sending out surveys on employee benefits or other company-wide topics. Stick to 15 to 20 questions, with a 10-minute completion time for most employees. Deploy the survey at least twice: once before your initial training event and once after. The responses will tell you if your training stuck with your employees or if reinforcing materials (in the form of short videos, posters, or articles) are needed.

WHAT TO DOIf you have experienced a security incident, capture the cost of remediation and use it as a baseline before you launch your training initiative. Keep these figures in your back pocket in case another incident occurs and determine if training reduced overall incident remediation costs. However, the greatest return will be if you don’t have any need for remediation because you’ve managed to avoid all security incidents!

4. How Much Incident Remediation CostsNo cybersecurity measure is 100% effective.

Chances are your organization has had a run-in with a data breach, malware infection, or another kind of cyber incident. Such an event may even be the reason you’re in the market for security and privacy awareness training in the first place.

Typically, recovering from such an incident isn’t cheap. In fact, the average total cost of a data breach is $3.92 million. Fortunately, independent research suggests an awareness training program significantly impacts the potential cost of recovering from an incident.

A commissioned study analyzing the ROI of MediaPRO’s approach to awareness training found that organizations experienced fewer malware incidents when training employees with MediaPRO, leading to a $58,968 reduction in incident remediation costs. A separate PWC study found that companies that trained their employees spent 76% less on security incidents than companies that offered no training.

31

7. Noting Employee Talk Around the Water CoolerBeyond numbers-based metrics, consider assessing your employees from a “softer” point of view. Take time to understand if employee behavior change is occurring.

WHAT TO DOKeep an ear open to discussions about your training topics before, during, and after your primary training deployment to see what conversations the content is generating.

You may notice that your employees are talking about one of the videos you shared during training. Or maybe they’re discussing that particularly tricky phishing email that made its way to your marketing department.

When your employees happily joke about data classifications, brag about the difficulty of their passwords, or argue about the right answer on the latest quiz you sent out, you will know that you have started to make real progress in creating a risk-aware culture.

This is also an opportunity to gauge the quality of the training. Did that specific attempt at humor completely miss the mark? Was that physical security scenario too hokey? Keeping your ear tuned to discussions around the water cooler is a good way to find out.

6. How Many Employees Complete TrainingTraining completion numbers become vital when compliance requirements come into play.

We’re vehemently against the “check-the-box” approach to awareness training. But if you’re in a specifically regulated industry (healthcare) or under the boot of regulations like the GDPR or CCPA (which require training), showing that your employees have taken training is vital.

WHAT TO DOAny learning management system (LMS) worth its salt will be able to tell you how many em-ployees have completed a given course in a specific timeframe. Though completion rates don’t capture the impact of training, they are necessary to show the minimum goal of a training initiative is met.

32

When evaluating the success of your program, it’s important to remember to look at the metrics comprehensively.

Beware of falling into the trap of hyper-focusing on one metric. For instance, singling in on phishing catch rates will only result in employees who are adept at mitigating one threat. However, this will do little to support a comprehensive understanding of topics like identi-fying personal information or creating physical office security.

A comprehensive awareness program that addresses multiple risk factors deserves a comprehensive approach to tracking success. Fortunately, a multi-topic awareness initia-tive gives you the opportunity to do just that. The more topics you address, the more data points you can collect and analyze to see the impact of your awareness initiative. Addition-ally, continual data gathering will help you continue to evolve your program and address additional risks.

Since training should never be a one-and-done affair, keeping these assessment methods in mind when a training refresh or update is needed will allow you to alter your training program to address emerging risks.

One of the core elements of all of MediaPRO’s TrainingPacks is the ability to monitor training effectiveness and prove compliance requirements.

Learn more about TrainingPacks by connecting with one of our experts.

BRINGING IT ALL TOGETHER

SPEAK TO AN EXPERT