Ways to Measure Training Effectiveness

/, ROI, Training, Videos/Ways to Measure Training Effectiveness

Ways to Measure Training Effectiveness

By | 2017-05-02T16:08:19+00:00 May 2nd, 2017|Animation, ROI, Training, Videos|0 Comments

According to The Corporate Learning Factbook 2014, U.S. spending on corporate training grew by 15% in 2015 (the highest growth rate in seven years) to over $70 billion in the U.S. and over $130 billion worldwide. 
That’s an astounding amount of money. Is the same trend true for you—are you spending more money on employee training? And if so, how do you evaluate its success? How do you know how much learning sticks, and how much training contributes to your overall bottom line?

More specifically in terms of our series on the effectiveness of using video in training, how can you measure ROI for videos?

There can be no palpable excuse for not measuring the return on training investment. If it’s a black hole in your business then push all your trainers into it and start over. — Anon

If you want to evaluate the success of your sales department, it’s fairly straightforward. A little data and a spreadsheet assures you that your salespeople are hitting the mark (or not). Determining ROI on video training is a little more ambiguous, but there are some clear indicators that can help evaluate the performance of your video.
In the past, it was tougher; metric-gathering systems for video and animation were pretty immature, especially when used on BYOD devices. But recent developments in the quality of learning management systems have made it possible to gather very fined-grained metrics as training is consumed online. These days, there is no reason why you should not know as much about the effect of video training as you would from conventional classroom instruction.

Know what success looks like before you start

Experts agree that to be able to trust metrics after training, you must know what you’ll test for as you create it. First come goals, objectives, and deciding what success will mean. You are providing this training to solve a problem. What do you expect to happen as a result of it, and how will that solution be measured?

For instance, one of our larger corporate customers is very dependent on measuring customer satisfaction as a key performance indicator of their individual locations. Our task was to train their employees in areas that were identified as causing lower customer satisfaction.
We developed a monthly magazine-style training video that was intended to both entertain and educate. The content of the video was directly aimed at addressing the specific customer satisfaction issues that had been identified in the customer satisfaction measurement. As we normally do we carefully measured the take-up rate of the video and the amount of time spent watching as well as geographic locations and time of day. The data measurement was achieved using basic Google analytics and some custom tools.

In parallel with the training, our client remeasured customer satisfaction before its release, during its release and subsequently. It was gratifying to see how customer satisfaction for specific issues improved significantly and in subsequent measurements remained higher than prior training.

On some occasions, the training video had little impact on customer satisfaction. However, because we were able to identify that this was the case we could remake the video taking a different approach for the next video magazine.

Of course not every subject can be adequately addressed by a BYOD training video. What is true is that it’s possible to identify these intransigent or difficult issues as a result of the measurement process. This data allows the training group to develop a different strategy to address the problem.

There is nothing unusual about the approach we took, but what we were able to do was accurately connect the dollars spent on the training materials with the outcome. This is especially exciting as the measurement data is derived from free tools and adds little to the cost of the project.

One place to begin is to listen to the employees you want to train. They, more than anyone, know what they need to learn and how that training will impact the organization and its customers. Interviews and surveys, as well as pre- and post-testing can provide a wealth of information. There are also several software programs designed to track employee learning progress.

Technology has many advantages, but there are also simple ways to observe the effectiveness of training. Can your recently trained employee teach a co-worker what they learned? This helps them retain the instruction and grow more confident in it, as well as spreading the wealth of information.

More technical metrics and how they work

As an understanding of how video works has increased, the use of assessments, quizzes, and interactive measurement points have become more effective.

90% of the data you collect will never be actionable or even helpful. (I made that up because I can’t actually measure it.) — Albert Einstein

When your video training includes a pre-course assessment and a post-course quiz, you can instantly see data concerning the rate of comprehension and knowledge. Another assessment one month later tells you about retention. The variety of methods to measure such criteria are immense and creative. One of my favorite examples is running simple contests and quizzes based on video content. These require the viewer to note certain items or events during the video. Whether or not a prize is offered, the quiz process seems to improve retention significantly. It typically has the benefit of broadening viewership.

Many organizations establish Key Performance Indicators—measurable values that indicate success. A company might develop an online training to address a specific KPI, and then measure the impact post-training. Several of our clients can predict impacts from their training in this way, and see if post-training evaluation—whether online, observed behaviorally on the job, or gathered in other ways such as customer surveys—match their expectations and deliver ROI.

We can help

The days of taking an expensive shot in the dark with your employee education budget are gone! However, deciding on what training success looks like, how to measure it, and how to interpret that data requires technical expertise. We keep our fingers on the pulse of this industry, and we can help you create a video training program that will gather the data you need for clear evaluation.

About the Author:

James is CEO and Chief Story Teller at Real Cool Productions. He has spent the last six years building a production company based on computer science development principles. James is passionate about technology, marketing, the science of learning and the psychology of understanding. He is totally committed to delivering the best possible customer experience to each and everyone of RCP's clients. His favorite quote is "The universe is not made up of atoms, it is made up of individual stories." -- Anon.

Leave A Comment

%d bloggers like this: