I started my career in engineering at a semiconductor company working alongside computer and electrical engineers. In engineering, there is a significant focus on taking effective measurements. An effective measurement must account for validity, reliability, sensitivity, specificity and accuracy. Without effective measurement projects fail.

I was shocked when transitioning into the learning and development (L&D) world and discovering that in a space where over $357.7 billion is now spent globally on corporate training, measuring training effectiveness is usually an afterthought or not done at all. As a result, executives are frustrated with the lack of results from training programs, and chief learning officers struggle to justify their annual budgets. As learning professionals, we must prove that we deserve a seat at the table and that our work directly benefits our organizations.

Many effective evaluation models are available, like Brinkerhoff’s Success Case Method, Kaufman’s Five Levels of Evaluation and the classic Kirkpatrick Four-level Model of Evaluation . While L&D experts like to debate the merits of each model, many organizations get stuck identifying and picking the perfect evaluation approach.

In the end, there is only one metric that really matters. Before we discover that metric, let’s review The Kirkpatrick Model and sample questions for each level that you can take and use in your training evaluation.

Level 1: Reaction

This level is intended to capture your learners’ reactions after a training experience. Measure dimensions like satisfaction, engagement and relevance. These questions should gauge the degree to which the experience was valuable to the employee. For example, you can ask employees:

• How likely would you recommend this training to a colleague?

This question is known as a net promoter score, and the response is on a scale of 0-10. It’s the single best question used to determine a learner’s reaction. If you’d like to dig deeper into why they provided their answer, include a comment box for them to expand on their response.

Level 2: Learning

This level measures your learners’ acquisition of new knowledge, confidence and commitment to apply what’s learned. For training instructors and managers, it helps determine if training objectives are met.

Customize your questions based on the knowledge you want the learner to acquire. Pick the two to three most important pieces of information you want them to learn and ask about it. It can be completed as pre- and post-training evaluation to show how the knowledge changed. Questions to include in this level of the assessment are:

• I am confident applying what I learned in this training.
• I commit to acting based on what I learned in this training.

Use a Likert scale (strongly disagree, disagree, slightly agree, agree, strongly agree) to measure the learner’s agreement with each statement. Likert scales are an industry standard used by research experts like Gallup to quantify and measure responses over time and across target groups.

Level 3 – Behavior

Monitor the change in behavior after the training is complete. Behavior change can be assessed in three ways: self-reported, peer input or manager. Depending on the approach you take, a question can be as simple as:

• How many times a day do you observe Scott practice empathy with colleagues?

Observations are also an effective way to monitor behavior change. Some industries, like public education, have regular cadences to observe staff behaviors and identify opportunities for coaching and development.

Levels 4 and 5 – Results and ROI

This level measures the impact of the training on the business and then translates the impact into monetary terms to determine how the return compares to training’s cost. Business results are the most impactful measurement but also the most difficult to measure. It requires leaders to directly connect the training provided to the business impact.

For some topics, this is easy. For example, if a machine operator goes through training on using a piece of equipment, you can look at the subsequent defect rate and quality levels to determine the impact. However, if your training is on unconscious bias, for example, identifying a direct business impact may be more difficult as the effects are more subtle.

This level is also challenging to measure as it requires the organization to have robust data sets and provide accessibility of this data to the training leaders. Once you identify the business impact, monetize that impact to calculate return on investment (ROI). Remember to consider all training costs like program development, delivery and labor costs for learners to complete the training.

ROI % = ($ Benefit of Training – $ Cost of Training) / $ Cost of Training

The Only Metric That Matters: ROE

Ultimately, all conversations around learning models and different evaluation techniques can be a distraction if you don’t measure the only metric that matters: return on expectations (ROE). Every time you conduct a training, different stakeholders are interested in its success. Proving a ROE, makes sure the stakeholders are satisfied and expectations are met.

Who are your stakeholders? Usually, there are multiple, but they could include the learners, the managers, senior executives, clients or third-party partners. Understanding the needs and expectations of these stakeholders should direct your evaluation and measurement approach.

Consider these examples:

1. An organization may require that its finance team is Sarbanes-Oxley Certified to meet compliance requirements. In this case, the most critical measurement is Level 2 learning, proving that the participants acquired and retained the presented information. 

2. A chief revenue officer may send their sales team through a popular sales training program. They’ve heard from peers that this program makes salespeople three times more effective. In this case, the most critical measurement is ROI and proving a direct impact on the team’s sales effectiveness.

3. A young, aspiring leader may participate in a course on empathy that your L&D team developed. They know this skill is essential for future leadership opportunities. For this stakeholder, the most crucial metric is evaluating Level 1 learner reaction to the time they invested in the course.

How to identify ROE? Once you identify your stakeholders, understand what they care about most. Then, focus your evaluation efforts and energy on the specific metrics and levels of evaluation that prove ROE.

Sit down with each stakeholder and ask questions like:

• Why is this training important to you?
• What do you hope your team gets out of this training?
• What specific impact on your team’s behavior or work do you want to see?
• What metrics and data do you expect to improve as a leader because of this training?

Effective measurement and metrics matter. They will be the difference between you getting the seat at the table and the resources you desire as a training leader or being viewed as a luxury that is optional for the organization to move forward.

However, not any metric will do. The only metrics that matter are the ones your stakeholders care about. Resist the temptation to get distracted by overly complex evaluation models or evaluating for evaluation’s sake. Instead, focus with laser intensity on the measurements that matter to your stakeholders so you can prove ROE!