Program Effectiveness Evaluation

Program Effectiveness Evaluation

Due care requires us to provide effective education, training, and awareness to employees; due diligence requires us to evaluate the effectiveness of that training.

This requires that we clearly define the indicators we will measure and how we will use those measurements to determine whether this overall security education, training, and awareness effort is fulfilling the organization's requirements.

The most important place to start is by asking the right questions.

First, remember the purpose of this awareness program: To enable and empower all members of the organization to achieve safer, more secure and compliant actions when using information.

That means you need to ask and answer questions like:

• Do your workers have the tools, processes, and procedures to do this?
• Do they know how to use them and do so effectively?
• Do they have resources to respond to the unexpected?
• How did they learn what actions to take?
• Were they active and engaged participants in the training?
• Does it really help them perform their jobs better?

Learning is most effective when the learner is accountable.

The students-not the teachers-own their own learning. Program evaluation needs to measure this, but at best, it can only be measured indirectly.

There are several approaches to evaluate the training program. Here are some to ways to get started:

User Participation

This can take the form of creating a list of desired training outcomes, then formally assessing participants against those outcomes when the training is complete. User participation can also be measured by using audits, random spot-checks, and questionnaires.

Social Engineering Response

The organization can use social engineering techniques in mock attack attempts and determine whether personnel who have been trained respond appropriately; for example, during phishing simulations.

The behavior of personnel can be assessed by surveying the event logs of users and determining whether their activity is in accordance with policy as conveyed by the training.