What are some ways to demonstrate Effectiveness of Training?

[snoopy2017]

Hi everyone,

What are some ways to demonstrate effectiveness of training? Does read-only training suffice? Or do employees have to take tests/quizzes to demonstrate their understanding? Thank you.

 

[Ronen E]

Hi everyone,

What are some ways to demonstrate effectiveness of training? Does read-only training suffice? Or do employees have to take tests/quizzes to demonstrate their understanding? Thank you.

Hi,

Effectiveness (of anything) means how well its pre-defined objectives have been met / achieved. The better the objectives are defined, the easier it is to measure / demonstrate effectiveness. If the objectives are defined in measurable and objective terms (if possible) it will be even easier.

Start by defining (preferably writing down) the objectives of the specific training. The methods for measuring success (effectiveness) will depend on the nature of each objective and the method & context of the training.

“Understanding” can sometimes be tricky to demonstrate. Sometimes it can be demonstrated through answering non-trivial, carefully constructed questions. Another possible way is to ask the trainee to explain the subject matter back to someone who is already considered knowledgeable / qualified enough in it.

If the objective is “the trainee will have read <X>”, read-only training will suffice, if there’s evidence that the trainee has in fact read <X> (eg a sign-off).

Cheers,
Ronen.

 

[AndyN]

Hi everyone,

What are some ways to demonstrate effectiveness of training? Does read-only training suffice? Or do employees have to take tests/quizzes to demonstrate their understanding? Thank you.

If you have determined the person needs training because there's an issue with being competent in some way, then of the training is effective, they will be able to demonstrate their new competency. Simple.

If you determined they needed training in some other way, you're going to have a bigger problem to solve, because now you have no measurable objective for going through the training.

 

[Edward Reesor]

This speaks to the "Validation" and "Verification" aspects of the quality system seen in all aspects of the standards. As stated above, you should have an effective goal of where you want to be. The training should address that goal and there should be a subsequent measure as to whether the training assisted in reaching that goal. You don't have to reach it in one training exercise but it should get you closer to your stated goal.

The opposite approach is to train the staff, and when asked by the auditors whether it made a difference you state, "I dunno, we just made them read a manual in the hopes that it stopped errors from happening".

Two different approaches to "training"...

 

[Mark Meer]

If the objective is “the trainee will have read <X>”, read-only training will suffice, if there’s evidence that the trainee has in fact read <X> (eg a sign-off).

I don't know that this speaks to effectiveness of the training. It might demonstrate that the training process is carried out according to established plans, but it doesn't say anything about the effectiveness of the training itself.

In my view, "effectiveness" of training can only be evaluated after a period of time following the training, in which the employee's performance is monitored according to some criteria/objective that is impacted by the training.

Simplest example of such an objective might be "there are no non-conforming outputs from processes for which the employee is solely responsible". A check for this can be scheduled following a period of time after the training, and then continually monitored (e.g. in Management Reviews).

 

[Marcelo]

It does depend on the objectives, but I would take care with determining them. Job creation and competency/training evaluation is part of the human factors engineering field (unfortunately most implementers do not know this or the AHFE area), and should be backed by activities such as task analysis and the like.

Then, depending on what you need to train, there's several techniques for evaluation, for example, take a look at the TOC of this book: Amazon.com: Handbook of Task Analysis Procedures (9780275926847): Wallace Hannum, David H Jonassen, Martin Tessmer: Books, it has examples of techniques for learning analysis, job/skill/behavior analysis techniques, and subject matter/content analysis techniques.


You can also see an example of this in my comments here: Update of SOPs for ISO 13485:2016 / MDSAP

 

[Ronen E]

I don't know that this speaks to effectiveness of the training. It might demonstrate that the training process is carried out according to established plans, but it doesn't say anything about the effectiveness of the training itself.

In my view, "effectiveness" of training can only be evaluated after a period of time following the training, in which the employee's performance is monitored according to some criteria/objective that is impacted by the training.

It all depends on what objectives were defined (sadly, many times objectives are not defined at all). You are probably thinking of more high-level objectives, which may add more value to the org; however, technically even achievement of low-level objectives can be considered a demonstration of effectiveness. You do something in order to achieve something; if it was achieved (no matter what it is), your action was effective. I’m not saying that this kind of effectiveness is “good”, just trying to be precise.

Simplest example of such an objective might be "there are no non-conforming outputs from processes for which the employee is solely responsible". A check for this can be scheduled following a period of time after the training, and then continually monitored (e.g. in Management Reviews).

I think this is not a very good example. “No NC outputs” is a result of many contributing factors out of which training may just be one. Both possible outcomes won’t tell you for sure that the training was effective / not effective.

 

[Mark Meer]

I think this is not a very good example. “No NC outputs” is a result of many contributing factors out of which training may just be one. Both possible outcomes won’t tell you for sure that the training was effective / not effective.

Poorly worded, granted. By "solely responsible", was intending to mean the employee was the root cause of the NC.

Perhaps a better statement of the example might be: "No NC's for which cause determination identifies personnel-error as a contributing factor".

MM.

 

[Mark Meer]

Perhaps a better statement of the example might be: "No NC's for which cause determination identifies personnel-error as a contributing factor".

Actually, thinking about this more, this objective still might not reflect on the effectiveness of training.

For example a training program for receiving might just be simply to follow the instruction: "visually inspect the item and confirm it is blue".

The employee, trained on this, might do EXACTLY what the training intended, but pass a damaged item (because the only thing they are checking for is its colour).

In this case, is the training effective?
On the one hand it is, because the employee is doing precisely what the training program prescribes.
But on the other hand, the training program is clearly insufficient to prevent NCs.

 

[normzone]

Question and answer

Observation

Test