M
MD
Evaluate effectiveness of training
Why should we evaluate effectiveness of training?
How could we do this?
Why should we evaluate effectiveness of training?
How could we do this?
J
June Ang - 2005
Why?
The purpose of training is to ensure the staff is competence to the responsibilities assigned to him/ her. Accordingly, the purpose to evaluate the effectiveness of the training would be to ensure that the trained staf would be able to carry out the job based on the training he/she has received.
How?
Your probably have objective for your training. What do you want to train? Who should be trained? What to be trained? What is your expectation to them after the training?
Our company practice is the immediate superior has to evaluate the peformance of the satff in that specific trained area. If the staff can perform well according to what has been trained, then the training shall be effective. Having an exam after the training would be one of the methods.
That's my opinions. Hope this help.
The purpose of training is to ensure the staff is competence to the responsibilities assigned to him/ her. Accordingly, the purpose to evaluate the effectiveness of the training would be to ensure that the trained staf would be able to carry out the job based on the training he/she has received.
How?
Your probably have objective for your training. What do you want to train? Who should be trained? What to be trained? What is your expectation to them after the training?
Our company practice is the immediate superior has to evaluate the peformance of the satff in that specific trained area. If the staff can perform well according to what has been trained, then the training shall be effective. Having an exam after the training would be one of the methods.
That's my opinions. Hope this help.
Claes Gefvenberg
Admin
Not so easy... or is it?
I think the first question is easy to answer: We are not spending energy ( No pun intended, Energy
) and money on training just for fun. We do it to be able to fulfil our obligations and improve on the current status. Thus, we want results and value for money just as in any other case.
The second one is a bit more tricky. What I think none of us want is a lot of paperwork and other bumf, so we'll want some simple method(s). One of the main reasons for failing training procedures today is that they very often turn out to be very buerocratic and so timeconsuming that they have to fail. I'll toss out a few hooks here to get the discussion started.
How do we define effective training in the first place? ISO 9004, 6.2.2: It should be "evaluated in terms of expectations and impact on the effectiveness and efficiency of the organisation as a means of improving future training plans"...
One way to get an overall result for all training aimed at a certain process could be to measure the results from that process. Successful training should raise the results. The problem is that a lot of other factors could do that too, so that would be a very blunt instrument, but just maybe it could be used?
A more obvious way would be to check what any given person has learned from his training, maybe with written tests or a rating from a supervisor, but it would be a time consuming cast iron b**** to rate the results, particularly in a big company.
/Claes
Ooops.. Simultaneous posting.. You beat me to it, June...
I've given this some thought too. We have not yet converted fully to ISO9001:2000, and this is an area we are still pondering about.
I think the first question is easy to answer: We are not spending energy ( No pun intended, Energy
) and money on training just for fun. We do it to be able to fulfil our obligations and improve on the current status. Thus, we want results and value for money just as in any other case.The second one is a bit more tricky. What I think none of us want is a lot of paperwork and other bumf, so we'll want some simple method(s). One of the main reasons for failing training procedures today is that they very often turn out to be very buerocratic and so timeconsuming that they have to fail. I'll toss out a few hooks here to get the discussion started.
How do we define effective training in the first place? ISO 9004, 6.2.2: It should be "evaluated in terms of expectations and impact on the effectiveness and efficiency of the organisation as a means of improving future training plans"...
One way to get an overall result for all training aimed at a certain process could be to measure the results from that process. Successful training should raise the results. The problem is that a lot of other factors could do that too, so that would be a very blunt instrument, but just maybe it could be used?
A more obvious way would be to check what any given person has learned from his training, maybe with written tests or a rating from a supervisor, but it would be a time consuming cast iron b**** to rate the results, particularly in a big company.
/Claes
Ooops.. Simultaneous posting.. You beat me to it, June...
Last edited:
E
energy
Training Courses?
Rather than start a new thread, I'll ask this here. Our company has lined up for some State funding for some training. We have had ISO Awareness Training, Internal Auditor Training, Process Mapping(LEAN). Are there any other Training Courses that you can think of that would be valuable in our pursuit of the golden fleece? It can be for individuals or groups. All suggestions welcome. Thanks in advance.
Rather than start a new thread, I'll ask this here. Our company has lined up for some State funding for some training. We have had ISO Awareness Training, Internal Auditor Training, Process Mapping(LEAN). Are there any other Training Courses that you can think of that would be valuable in our pursuit of the golden fleece? It can be for individuals or groups. All suggestions welcome. Thanks in advance.
barb butrym
Quite Involved in Discussions
Energy..........
training...that opens a huge can of worms. Its not whats available, its what do you need.
how are your meetings? are they time wasters? Roberts rules or "holding effective meetings' is a neat topic...just an hour or two.....
the quality tools and Coveys habits are neat lunch time training offerings i get lots of calls for.
What tools do your auditors use in planning and executing internal audits? expand on them..add to them ....... each auditor, once they are past the initial training needs to "grow' what is your plan for that? they each need a tool box......(IMHO>>>>)
Effective corrective and preventive action and root cause analysis.
Continuous Improvement tools
Safety, MSDS, lock out tag out, REDACTED harrassment.....
how about SPC tools so basic that even administrative people like it? charting HR stuff or sales stuff, great tool for reporting...about 1/2 day or less. tie to CI stuff
workmanship standards, especially mil stds or IPC. How about a session that shows people what else is going on besides their operation ...and how they fit into the overall scheme of things...Awareness of opportunities for movement within the company.
computer skills, english as a second language......
all this just off the top of my head...what industry are you in? that would trigger even more.........
training...that opens a huge can of worms. Its not whats available, its what do you need.
how are your meetings? are they time wasters? Roberts rules or "holding effective meetings' is a neat topic...just an hour or two.....
the quality tools and Coveys habits are neat lunch time training offerings i get lots of calls for.
What tools do your auditors use in planning and executing internal audits? expand on them..add to them ....... each auditor, once they are past the initial training needs to "grow' what is your plan for that? they each need a tool box......(IMHO>>>>)
Effective corrective and preventive action and root cause analysis.
Continuous Improvement tools
Safety, MSDS, lock out tag out, REDACTED harrassment.....
how about SPC tools so basic that even administrative people like it? charting HR stuff or sales stuff, great tool for reporting...about 1/2 day or less. tie to CI stuff
workmanship standards, especially mil stds or IPC. How about a session that shows people what else is going on besides their operation ...and how they fit into the overall scheme of things...Awareness of opportunities for movement within the company.
computer skills, english as a second language......
all this just off the top of my head...what industry are you in? that would trigger even more.........
Last edited:
Jim, my organization briefly wrestled with the idea of using performance appraisals/job reviews/job observations/(insert your organization's designation for this process here), but we've opted to search for another method when it comes to addressing competence and competency needs.
I suppose a lot of it depends on the size and structure of your organization. Mine is a lean organization and if our Internal Auditors had access to the appraisals of co-workers (and friends), this could affect how we work with people. We're professionals, like every organization, but employees should not have access to confidential information on other employees (except for those who maintain the information).
A few options have been suggested:
Remove the section(s) on competency from the reviews and keep separate.
On-the-job evaluations from immediate supervisor.
Product quality (i.e., has the root cause of a nonconformance been attributed to a lack of competence?).
Positive/stable trends in the process measurements/monitoring - stable = competent, positive = competent and continually improving.
Tests on selected material with a pass rate of 70%.
Just my $0.02 CDN worth.
I suppose a lot of it depends on the size and structure of your organization. Mine is a lean organization and if our Internal Auditors had access to the appraisals of co-workers (and friends), this could affect how we work with people. We're professionals, like every organization, but employees should not have access to confidential information on other employees (except for those who maintain the information).
A few options have been suggested:
Remove the section(s) on competency from the reviews and keep separate. On-the-job evaluations from immediate supervisor. Product quality (i.e., has the root cause of a nonconformance been attributed to a lack of competence?). Positive/stable trends in the process measurements/monitoring - stable = competent, positive = competent and continually improving. Tests on selected material with a pass rate of 70%.Just my $0.02 CDN worth.
barb butrym
Quite Involved in Discussions
MD back to your question
Proven proficiency an be measured many ways. A test is easy...but IMHO just tests testmanship. But it covers it. Some use a questionnaire at the end. covers it, and can be a useful tool....BUT my favorite is to witness proven proficiency......typically performance reviews. the real test is back at the worksite, so if its not easy to accomplish...then just do it for critical operations. Some really don't require any..as in ISO awareness.....Sometimes classroom participation/feedback is all that is needed. i do a lesson plan for each topic that defines what effectivity is, as well as the equipment, handouts, objectives, standards and whatever else.
re: auditors seeing performance reviews....they should not be permitted read the review, just see that they were done....and the record is there...Absolutely
Proven proficiency an be measured many ways. A test is easy...but IMHO just tests testmanship. But it covers it. Some use a questionnaire at the end. covers it, and can be a useful tool....BUT my favorite is to witness proven proficiency......typically performance reviews. the real test is back at the worksite, so if its not easy to accomplish...then just do it for critical operations. Some really don't require any..as in ISO awareness.....Sometimes classroom participation/feedback is all that is needed. i do a lesson plan for each topic that defines what effectivity is, as well as the equipment, handouts, objectives, standards and whatever else.
re: auditors seeing performance reviews....they should not be permitted read the review, just see that they were done....and the record is there...Absolutely
Last edited:
Re: Training Courses?
Energy,
A couple quick questions and thoughts: Does the training have to be performed by "outsiders" to qualify, or can your own (internal) people (like you) do it and still get the state funding? And does it have to be linked to ISO for the funding?
If you can do it internally, maybe this would be a good time to get some "paid" cross-training done in various areas.
If it has to be "external", maybe some of your folks would like to learn a new software package (I'd like to learn Access database software). Maybe some of the Engineers would like to learn some test automation software (i.e. HPVEE, Labview) or statistics software. Maybe just basic statistics classes period would help some of the line folks. Maybe DOE. Maybe the latest employee retention or salary administration training for your HR people. Train the trainer for the less experienced supervisors?
That's a few quick ones. If I think of others I drop a note later.
Mike S.
_______________________energy said: We have had ISO Awareness Training, Internal Auditor Training, Process Mapping(LEAN). Are there any other Training Courses that you can think of that would be valuable in our pursuit of the golden fleece? It can be for individuals or groups. All suggestions welcome. Thanks in advance.
Energy,
A couple quick questions and thoughts: Does the training have to be performed by "outsiders" to qualify, or can your own (internal) people (like you) do it and still get the state funding? And does it have to be linked to ISO for the funding?
If you can do it internally, maybe this would be a good time to get some "paid" cross-training done in various areas.
If it has to be "external", maybe some of your folks would like to learn a new software package (I'd like to learn Access database software). Maybe some of the Engineers would like to learn some test automation software (i.e. HPVEE, Labview) or statistics software. Maybe just basic statistics classes period would help some of the line folks. Maybe DOE. Maybe the latest employee retention or salary administration training for your HR people. Train the trainer for the less experienced supervisors?
That's a few quick ones. If I think of others I drop a note later.
Mike S.
A
Atul Khandekar
The method we (try to) follow is based on Kirkpatrick's model of measuring taining effectiveness:
1. Ask the trainees to tell us their first impression about the training. Did they like the the way it was presented, was it relevant, how was the course structured/paced?
2. A test (usually written - simple objective type questions) to gauge how much the participants have understood/ grasped.
3. Follow up after a period to check how much is retained. We usually ask the company to conduct another test after say, one or two months (period may vary). If there are exercises given (like conducting SPC on a part ) to be completed in a certain period,we follow up on them. Sometimes there is a small (half-day) refresher associated with this.
We do mostly classroom training. But I believe the method could easily be used for other type of training (OJT) as well.
Kirkpatrick's model suggests a fourth step: Gauging the finacial gains achieved through training.
Also check out these threads:
http://Elsmar.com/Forums/showthread.php?s=&threadid=4002
http://Elsmar.com/Forums/showthread.php?s=&threadid=232
-Atul.
1. Ask the trainees to tell us their first impression about the training. Did they like the the way it was presented, was it relevant, how was the course structured/paced?
2. A test (usually written - simple objective type questions) to gauge how much the participants have understood/ grasped.
3. Follow up after a period to check how much is retained. We usually ask the company to conduct another test after say, one or two months (period may vary). If there are exercises given (like conducting SPC on a part ) to be completed in a certain period,we follow up on them. Sometimes there is a small (half-day) refresher associated with this.
We do mostly classroom training. But I believe the method could easily be used for other type of training (OJT) as well.
Kirkpatrick's model suggests a fourth step: Gauging the finacial gains achieved through training.
Also check out these threads:
http://Elsmar.com/Forums/showthread.php?s=&threadid=4002
http://Elsmar.com/Forums/showthread.php?s=&threadid=232
-Atul.
D
db
apraisals
Appraisals
First, of all, Atul took my post of Kirkpatrick's models. I guess great minds think alike.
As far as using performance evaluations, what I recommend is to use two different forms. One deals only with performance, and the other deals with the personnel stuff. I’m not convinced the auditors should have to read the details of each document either. The internal auditor should know if performance is evaluated because the internal auditor is also evaluated. All he/she would have to do is to verify that the evaluations are not being performed on just the internal auditors.
An alternative method would be to interview managers and employees. If all say the same thing, we could use that as our objective evidence that training effectiveness is being evaluated during the performance review. The registrar would validate this through the surveillance audits.
Appraisals
First, of all, Atul took my post of Kirkpatrick's models. I guess great minds think alike.
As far as using performance evaluations, what I recommend is to use two different forms. One deals only with performance, and the other deals with the personnel stuff. I’m not convinced the auditors should have to read the details of each document either. The internal auditor should know if performance is evaluated because the internal auditor is also evaluated. All he/she would have to do is to verify that the evaluations are not being performed on just the internal auditors.
An alternative method would be to interview managers and employees. If all say the same thing, we could use that as our objective evidence that training effectiveness is being evaluated during the performance review. The registrar would validate this through the surveillance audits.