No matter what type of training programs we deliver within our organization, we can all agree we want them to “work” and positively influence our employees. But we must also agree that when it comes to the impact and outcomes of this training, some programs are certainly more important than others. For instance, if you attend a program that boosts your Excel skills or improves your presentation skills, it’s certainly going to benefit your efficiency and performance, but is it as crucial to your performance as something like safety training?

Some programs should be measured to make sure they’re working, while others must be measured to make sure they’re working. Training that helps drive the proper safety protocols and prevents hazardous incidents is something that definitely needs to be measured for effectiveness. We need to show stakeholders real data that demonstrates improved behaviors on the job and increased key safety indicators. They need to know their investment is worth it.

A Case Study at bp

BP is a multi-national energy company with operations in traditional oil and gas and renewable energy vectors. BP considers safe operations both a moral obligation and critical to their business success. Experts agree that leaders are critical in creating an effective health and safety culture, therefore fostering great leadership behaviors is a key pillar in bp’s safety strategy.

In 2023, bp’s internal L&D team partnered with a best-in-class immersive training vendor to build and deliver a customized experience that would improve safety through better leadership conversations that drive adherence to safety principles. The training targeted employee behaviors like effectively communicating the safety principles, asking for feedback, building trust and having difficult conversations about risks.

The measurement strategy.

Using a representative sample of 82 leaders from across the organization, this study sought to quantify how applying the skills and techniques taught in the safety leadership program could promote key behaviors on the job and drive higher safety performance for the organization. To measure impact, we used a six-level evaluation strategy:

Methods and results.

Level 1: For Level 1, participants rated their training experience immediately after the immersive session on a scale from 1 to 5. Results averaged a 4.8 out of 5.

Level 2: For Level 2, participants were asked about the new knowledge they gained. Results showed 95% of participants acquired new and valuable knowledge that was relevant to their role.

Level 3: For Level 3, participants were assessed for on-the job improvements in six specific behaviors 60+ days after training. The behavioral improvements were captured on a five-point scale ranging from “no improvement” to “exceptional improvement.”

Key findings included:

• 91% of participants improved the effectiveness of their safety conversations.
• 84% of participants built a deeper sense of trust with team members.
• 81% of participants became more skilled at having challenging discussions.

Level 4: For Level 4, to isolate the impact of training on performance, we asked participants to first report their increases in productivity and contribution over the post-training months as a percentage, and then adjusted for error by factoring in only the percent of participants who had significant to exceptional behavior change on the job at Level 3 (and assigned a 0% to all others who didn’t have high application). Then to isolate even further, we also factored in only the percent of their job they would be directly using the trained skills. This brought an original estimate of 39% for everyone in the sample to a very isolated and far more conservative 2% increase in performance. So, our final results here showed an overall increase in productivity per participant of 2% directly attributable to training.

See the diagram below.

Level 5: To develop our return on investment (ROI), we took the benefits (increased performance) identified at Level 4, monetized them and compared them to the fully loaded costs of training. To monetize our performance gain, we first applied an industry standard of pay for performance which translates an employee’s performance at 1 to 2.5 times their salary. That is, employees have to contribute at least the value of their salary, or it would not be profitable to employ them. We used the most conservative version which is a 1:1 ratio. Overall, when the average percent of performance increased across all participants who were analyzed and monetized, we ended with a benefit of $376 per participant. When we compared this to the fully loaded cost of training per participant (including hours away from the job), we ended with a final ROI of 275%. This meant for every $1 dollar invested, bp made back $3.75!

Level 6: For Level 6, the question we wanted to answer was: Did management’s support on the job enhance the impact and ROI of training? To conduct this analysis, we broke our sample into two groups: participants who had “high” manager support and those that had “low” manager support. We found that participants who received high manager support post-training had a seven times greater ROI.

These results clearly suggest that the ROI in training investments can increase exponentially when participants are provided post-training support from their manager.

Conclusion

It’s important to track and evaluate the efficacy for training in all industries, but we cannot overlook the important of safety training, especially when it can reduce risk and possibly save lives. In this real-life example, bp was able to prove that their immersive safety training “worked” by employing a comprehensive measurement strategy and collecting the right data.

They showed the participant journey started with an incredibly engaging experience (Level 1), that inspired more learning (Level 2), which led to more productive behaviors on the job (Level 3), leading to lower risk and higher performance (Level 4), and an impressive ROI (Level 5) which is maximized by ramping up manager support for their participants (Level 6).

This story of impact demonstrates to stakeholders and business leaders just how effective and valuable employee training is to the business. Knowing now, with real evidence, that they invested in training that fueled a safer, healthier environment for all their employees is the exact ROI they were looking for.

Ready to optimize the way you evaluate your training programs and prove the impact of your training initiatives? You can learn more from the author, Dr. Paul Leone, by participating in Training Industry’s Measuring the Impact of L&D Certificate program, or you can start proving your impact today by accessing the evaluation template below.