Training effectiveness measurement - 6.2.2 Competence, awareness and training

[mdumont]

Hello,

I'm new on the forum and it is my first post. I'm resposible for the development of ISO9001:2000 QMS in a software service company in Europe. We are quite advenced in the develoment and have started deployment but I still have some dark areas to work out.

The current one is the satisfaction of the requiements on training, (§6.2.2 Competence, awareness and training, paragraph c) evaluate the effectiveness of the action taken.) I havent find yet a thread on that subject.

We have in place a Db with people competence, trainings and even description of their assignements. Now the problem is to measure if the result of training is effective. If we send a developer on a Java training for a specific project, how can we evaluate the impact of such training on the development execution? I'm not in favor of peer reviews on each project afetr any training.

Hope to find interresting suggestion.

Thanks in advance.

 

[SteelMaiden]

I maybe make things too simplistic, but if we have to assess the competence of personnel at some planned frequency, and we assess suppliers (training orgs/personnel for outside provided training?) plus the fact that most of the people I've trained give some feedback, either telling me I did a good job, or telling someone else that I suck and then that gets back to me.....shouldn't that be enough? My boss, the top dog, would assess my competence, and if he has sent me to some training and feels I learned what I needed and that I am competent, doesn't that show that training was effective????? On the other hand if he had selected a class for all of us supervisors to go to and we all went back and told him that the class was not good, or the trainer was bad, or whatever, that trainer would never darken our doors again.

Anybody? Has anyone had any kind of feedback from registrars about this?

I almost forgot, welcome to the Cove, mdumont.

 

[howste]

Welcome to the Cove, mdumont!

Here's what I usually do to evaluate the effectiveness of training, in order of easiest to hardest (also in order of least effective to most effective IMO):

1) Testing - written, oral, pretest/posttest, etc. You've got to be careful with this one. The standard requires competence, which is defined as the "demonstrated ability to apply knowledge and skills." Some tests only determine if you have knowledge, others determine if you have the ability to apply the knowledge.

2) Skill/performance evaluation - after training, a knowledgable person watches you do the task/process and determines if you did it correctly (if you can apply the knowledge/skill).

3) Metrics - measure the results of a process to detect an improvement after the training took place. Example: Before training, 5% scrap. After training, 2% scrap.

I should also say that I don't believe that course evaluations by participants are good measurements of effectiveness. If I didn't know how to do something before the training, how can I evaluate if I was properly trained to do it in the class? Usually what you get is an opinion of how the participant thought the class went. If it was a fun class, you get great scores regardless if they learned anything.

 

[db]

I know this has been covered before, but I will try from memory.

Kirkpatrick (I hope that's the correct name) came up with four methods of evaluating training:

1) Participant evaluations: Commonly referred to as "smile sheets". It might show satisfaction with the course, but really does nothing to show training effectiveness. (as howste pointed out)

2) Course evaluations: Better as it measures how much the participant might have learned, but some folks are good test takers, and some are not, so it doesn't give a good picture.

3) Task evaluations: This is the best form of evaluation. IT basically asks if the participant can meet the intent of the course. Can the participant sucessfully now demonstrate ability to perform.

4) Profit increase evaluations: This is the ultimate evaluation, as it asks as a result of the training, have profits increased. The problem is there are too many factors that play on the profit, and it is difficult (if not impossible) to assign what percentage of change to the training.

The requirement is to determine the effectiveness of the action taken. The simplist answer is to ask can the participant now do the job. A one-time, hands-on test is the best method. From there, all you need is regular management observation and you should be set.

 

[Atul Khandekar]

Also See:
http://www.elsmar.com/Forums/showthread.php?t=232
http://www.elsmar.com/Forums/showthread.php?t=4002
http://www.elsmar.com/Forums/showthread.php?t=4577
http://www.elsmar.com/Forums/showthread.php?t=6047
http://www.elsmar.com/Forums/showthread.php?t=6815

 

[Douglas E. Purdy]

FYI for Others following these links.

Also See:
http://www.elsmar.com/Forums/showthread.php?t=232
http://www.elsmar.com/Forums/showthread.php?t=4002
http://www.elsmar.com/Forums/showthread.php?t=4577
http://www.elsmar.com/Forums/showthread.php?t=6047
http://www.elsmar.com/Forums/showthread.php?t=6815

I was following these links in search of a subject I had in mind and found that the last link is this thread.

 

[Sandyr]

I know this has been covered before, but I will try from memory.

Kirkpatrick (I hope that's the correct name) came up with four methods of evaluating training:

1) Participant evaluations: Commonly referred to as "smile sheets". It might show satisfaction with the course, but really does nothing to show training effectiveness. (as howste pointed out)

2) Course evaluations: Better as it measures how much the participant might have learned, but some folks are good test takers, and some are not, so it doesn't give a good picture.

3) Task evaluations: This is the best form of evaluation. IT basically asks if the participant can meet the intent of the course. Can the participant sucessfully now demonstrate ability to perform.

4) Profit increase evaluations: This is the ultimate evaluation, as it asks as a result of the training, have profits increased. The problem is there are too many factors that play on the profit, and it is difficult (if not impossible) to assign what percentage of change to the training.

The requirement is to determine the effectiveness of the action taken. The simplist answer is to ask can the participant now do the job. A one-time, hands-on test is the best method. From there, all you need is regular management observation and you should be set.

 

[Sandyr]

Training Effectiveness

Hi All. This is my first time contributing to the cove. These new requirements for TS can be frustrating.
The way we are going to and have shown training effectiveness in our company is:
eg. an employee is learning a new position. We have an on-the-job training form that lists the persons name, etc.. and the position that they are training for.
Once their training is completed (for some jobs it is a certain amout of time, for others more), the supervisor will visually test the employee for effectiveness of the training taken and the evaluation is recorded on the sheet and a point system is used to evaluate certain aspects of the training.
I feel this is fairly compliant. Not to say that we have gone through registration yet, but... we'll see.

 

[db]

Hi All. This is my first time contributing to the cove. These new requirements for TS can be frustrating.
The way we are going to and have shown training effectiveness in our company is:
eg. an employee is learning a new position. We have an on-the-job training form that lists the persons name, etc.. and the position that they are training for.
Once their training is completed (for some jobs it is a certain amout of time, for others more), the supervisor will visually test the employee for effectiveness of the training taken and the evaluation is recorded on the sheet and a point system is used to evaluate certain aspects of the training.
I feel this is fairly compliant. Not to say that we have gone through registration yet, but... we'll see.

Welcome to the Cove Sandyr!

A quick question. What does the point system do? Other than the points, the system sounds compliant, and sounds like it should work. I would recommend that you couple the testing with post-testing observation (watching the newly trained employee a little closer than normal), but I'm sure you currently do that anyway.

Let us know how things progress.

 

[Sandyr]

I maybe make things too simplistic, but if we have to assess the competence of personnel at some planned frequency, and we assess suppliers (training orgs/personnel for outside provided training?) plus the fact that most of the people I've trained give some feedback, either telling me I did a good job, or telling someone else that I suck and then that gets back to me.....shouldn't that be enough? My boss, the top dog, would assess my competence, and if he has sent me to some training and feels I learned what I needed and that I am competent, doesn't that show that training was effective????? On the other hand if he had selected a class for all of us supervisors to go to and we all went back and told him that the class was not good, or the trainer was bad, or whatever, that trainer would never darken our doors again.

Anybody? Has anyone had any kind of feedback from registrars about this?

I almost forgot, welcome to the Cove, mdumont.