Training effectiveness measurement - 6.2.2 Competence, awareness and training

M

mdumont

#1
Hello,

I'm new on the forum and it is my first post. I'm resposible for the development of ISO9001:2000 QMS in a software service company in Europe. We are quite advenced in the develoment and have started deployment but I still have some dark areas to work out.

The current one is the satisfaction of the requiements on training, (§6.2.2 Competence, awareness and training, paragraph c) evaluate the effectiveness of the action taken.) I havent find yet a thread on that subject.

We have in place a Db with people competence, trainings and even description of their assignements. Now the problem is to measure if the result of training is effective. If we send a developer on a Java training for a specific project, how can we evaluate the impact of such training on the development execution? I'm not in favor of peer reviews on each project afetr any training.

Hope to find interresting suggestion.

Thanks in advance.
 
Elsmar Forum Sponsor

SteelMaiden

Super Moderator
Super Moderator
#2
I maybe make things too simplistic, but if we have to assess the competence of personnel at some planned frequency, and we assess suppliers (training orgs/personnel for outside provided training?) plus the fact that most of the people I've trained give some feedback, either telling me I did a good job, or telling someone else that I suck and then that gets back to me.....shouldn't that be enough? My boss, the top dog, would assess my competence, and if he has sent me to some training and feels I learned what I needed and that I am competent, doesn't that show that training was effective????? On the other hand if he had selected a class for all of us supervisors to go to and we all went back and told him that the class was not good, or the trainer was bad, or whatever, that trainer would never darken our doors again.

Anybody? Has anyone had any kind of feedback from registrars about this?

I almost forgot, welcome to the Cove, mdumont.
 

howste

Thaumaturge
Super Moderator
#3
Welcome to the Cove, mdumont!

Here's what I usually do to evaluate the effectiveness of training, in order of easiest to hardest (also in order of least effective to most effective IMO):

1) Testing - written, oral, pretest/posttest, etc. You've got to be careful with this one. The standard requires competence, which is defined as the "demonstrated ability to apply knowledge and skills." Some tests only determine if you have knowledge, others determine if you have the ability to apply the knowledge.

2) Skill/performance evaluation - after training, a knowledgable person watches you do the task/process and determines if you did it correctly (if you can apply the knowledge/skill).

3) Metrics - measure the results of a process to detect an improvement after the training took place. Example: Before training, 5% scrap. After training, 2% scrap.

I should also say that I don't believe that course evaluations by participants are good measurements of effectiveness. If I didn't know how to do something before the training, how can I evaluate if I was properly trained to do it in the class? Usually what you get is an opinion of how the participant thought the class went. If it was a fun class, you get great scores regardless if they learned anything. :rolleyes:
 
#4
I know this has been covered before, but I will try from memory.

Kirkpatrick (I hope that's the correct name) came up with four methods of evaluating training:

1) Participant evaluations: Commonly referred to as "smile sheets". It might show satisfaction with the course, but really does nothing to show training effectiveness. (as howste pointed out)

2) Course evaluations: Better as it measures how much the participant might have learned, but some folks are good test takers, and some are not, so it doesn't give a good picture.

3) Task evaluations: This is the best form of evaluation. IT basically asks if the participant can meet the intent of the course. Can the participant sucessfully now demonstrate ability to perform.

4) Profit increase evaluations: This is the ultimate evaluation, as it asks as a result of the training, have profits increased. The problem is there are too many factors that play on the profit, and it is difficult (if not impossible) to assign what percentage of change to the training.

The requirement is to determine the effectiveness of the action taken. The simplist answer is to ask can the participant now do the job. A one-time, hands-on test is the best method. From there, all you need is regular management observation and you should be set.
 

Douglas E. Purdy

Quite Involved in Discussions
#6
S

Sandyr

#7
db said:
I know this has been covered before, but I will try from memory.

Kirkpatrick (I hope that's the correct name) came up with four methods of evaluating training:

1) Participant evaluations: Commonly referred to as "smile sheets". It might show satisfaction with the course, but really does nothing to show training effectiveness. (as howste pointed out)

2) Course evaluations: Better as it measures how much the participant might have learned, but some folks are good test takers, and some are not, so it doesn't give a good picture.

3) Task evaluations: This is the best form of evaluation. IT basically asks if the participant can meet the intent of the course. Can the participant sucessfully now demonstrate ability to perform.

4) Profit increase evaluations: This is the ultimate evaluation, as it asks as a result of the training, have profits increased. The problem is there are too many factors that play on the profit, and it is difficult (if not impossible) to assign what percentage of change to the training.

The requirement is to determine the effectiveness of the action taken. The simplist answer is to ask can the participant now do the job. A one-time, hands-on test is the best method. From there, all you need is regular management observation and you should be set.
 
S

Sandyr

#8
Training Effectiveness

:frust: Hi All. This is my first time contributing to the cove. These new requirements for TS can be frustrating.
The way we are going to and have shown training effectiveness in our company is:
eg. an employee is learning a new position. We have an on-the-job training form that lists the persons name, etc.. and the position that they are training for.
Once their training is completed (for some jobs it is a certain amout of time, for others more), the supervisor will visually test the employee for effectiveness of the training taken and the evaluation is recorded on the sheet and a point system is used to evaluate certain aspects of the training.
I feel this is fairly compliant. Not to say that we have gone through registration yet, but... we'll see.
 
#9
Sandyr said:
:frust: Hi All. This is my first time contributing to the cove. These new requirements for TS can be frustrating.
The way we are going to and have shown training effectiveness in our company is:
eg. an employee is learning a new position. We have an on-the-job training form that lists the persons name, etc.. and the position that they are training for.
Once their training is completed (for some jobs it is a certain amout of time, for others more), the supervisor will visually test the employee for effectiveness of the training taken and the evaluation is recorded on the sheet and a point system is used to evaluate certain aspects of the training.
I feel this is fairly compliant. Not to say that we have gone through registration yet, but... we'll see.
Welcome to the Cove Sandyr! :bigwave:

A quick question. What does the point system do? Other than the points, the system sounds compliant, and sounds like it should work. I would recommend that you couple the testing with post-testing observation (watching the newly trained employee a little closer than normal), but I'm sure you currently do that anyway.

Let us know how things progress.
 
S

Sandyr

#10
SteelMaiden said:
I maybe make things too simplistic, but if we have to assess the competence of personnel at some planned frequency, and we assess suppliers (training orgs/personnel for outside provided training?) plus the fact that most of the people I've trained give some feedback, either telling me I did a good job, or telling someone else that I suck and then that gets back to me.....shouldn't that be enough? My boss, the top dog, would assess my competence, and if he has sent me to some training and feels I learned what I needed and that I am competent, doesn't that show that training was effective????? On the other hand if he had selected a class for all of us supervisors to go to and we all went back and told him that the class was not good, or the trainer was bad, or whatever, that trainer would never darken our doors again.

Anybody? Has anyone had any kind of feedback from registrars about this?

I almost forgot, welcome to the Cove, mdumont.
 
Thread starter Similar threads Forum Replies Date
L Verifying training effectiveness for procedure changes ISO 13485:2016 - Medical Device Quality Management Systems 4
S What are some ways to demonstrate Effectiveness of Training? ISO 13485:2016 - Medical Device Quality Management Systems 29
R Documenting Self-Training and Effectiveness - ISO 13485:2016 Training - Internal, External, Online and Distance Learning 4
Colin Effectiveness of IRCA Registered Training ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 4
M Training Effectiveness TS 16949 Clause 6.2.2c Training - Internal, External, Online and Distance Learning 5
S How to Monitor Training Effectiveness Training - Internal, External, Online and Distance Learning 13
Q Third Party Audit Nonconformance for Training effectiveness IATF 16949 - Automotive Quality Systems Standard 20
M Effectiveness of Read-Only Training for Procedures Training - Internal, External, Online and Distance Learning 4
S Training Efficiency Assessment - Evaluate the Effectiveness (Clause 6.2.2 c) Training - Internal, External, Online and Distance Learning 3
M How To Measure Training Effectiveness Quality Tools, Improvement and Analysis 8
D How to Measure QSR Training Effectiveness? Training - Internal, External, Online and Distance Learning 6
somashekar Training Effectiveness across Participants ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
J Effectiveness of training? How to Measure Training Effectiveness Training - Internal, External, Online and Distance Learning 51
S How to show that policies are properly communicated & Training Effectiveness ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
C Education vs. Training - Training effectiveness program ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 15
A Effectiveness of Training... micro or macro? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 29
J Information regarding Competence and Effectiveness of Training Training - Internal, External, Online and Distance Learning 18
S Evaulation of Operator and Staff Training Effectiveness Training - Internal, External, Online and Distance Learning 10
R Training Competency Verification - NCR related to Training Effectiveness Training - Internal, External, Online and Distance Learning 8
S AS9100 - 6.2.2 - Effectiveness of training? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
S Evidence of Effectiveness of OJT (On the Job Training) - ISO 9001 Clause 6.2.2 Training - Internal, External, Online and Distance Learning 24
R Need some information on Training Effectiveness Assessment Form Document Control Systems, Procedures, Forms and Templates 3
R 13485 6.2.2 c) "evaluate the effectiveness of the actions taken" (training) ISO 13485:2016 - Medical Device Quality Management Systems 4
R APQP and Core Tools Training how to Determine the Effectiveness of Training Training - Internal, External, Online and Distance Learning 10
S How do you evaluate effectiveness of training ISO9001 Section 6.2.2.c? Training - Internal, External, Online and Distance Learning 12
B Training Effectiveness Evaluation - Different Styles - In-House and External Document Control Systems, Procedures, Forms and Templates 21
Raffy How can I demonstrate evidence and effectiveness of Corrective Action Training Nonconformance and Corrective Action 7
M ISO 9001:2000 - Evaluating Training Effectiveness and Why ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
N Training Competency and Effectiveness Documentation Training - Internal, External, Online and Distance Learning 27
Raffy How do I measure Measuring Training Effectiveness? Performance evaluation? Other ISO and International Standards and European Regulations 26
SteelMaiden Documenting Competency and Effectiveness of Training Document Control Systems, Procedures, Forms and Templates 7
C Training Effectiveness: Simple format for supervisors to track and evaluate training Document Control Systems, Procedures, Forms and Templates 2
G Evaluating Training Effectiveness - 6.2.2 Competence, Awareness and Training ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 55
S Measuring Training Effectiveness - How do you measure Effectiveness? Training - Internal, External, Online and Distance Learning 50
Anonymous16-2 Labelling Training Training - Internal, External, Online and Distance Learning 3
J Recommendations for online ISO 19011 training? Training - Internal, External, Online and Distance Learning 6
J Looking for a training organization that can provide in house DFMEA training FMEA and Control Plans 4
A AS9100 Training - Where do I start? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 11
R What's the major difference between Green Belt and Black Belt in term of training and project Six Sigma 3
S ISO 13485 Lead Auditor - Debate between our Quality Team and Regulatory Auditor - Internal Auditor Training ISO 13485:2016 - Medical Device Quality Management Systems 17
I Where can I beg, steal (just kidding of course) or borrow good training material on the ISO 9001:2015 standard? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
JAMESH Building a "Lean" Job Shop Training Program Training - Internal, External, Online and Distance Learning 0
I Does training have to be written? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
I Root Cause Analysis as Training ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
Q Regulations for Medical Device Kit for Training Personnel? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
S Can we provide training plan as corrective action for IATF 16949 Non conformity? IATF 16949 - Automotive Quality Systems Standard 9
S IRCA Lead Auditor training and Exam tips Training - Internal, External, Online and Distance Learning 1
N ... and some have greatness thrust upon them - Training for novice QA manager ? Training - Internal, External, Online and Distance Learning 6
C Looking for APQP Training recommendations preferably in the aerospace industry APQP and PPAP 2
B VDA 6.3 Qualification as Process Auditor training course and exam VDA Standards - Germany's Automotive Standards 0

Similar threads

Top Bottom