Good Morning Everyone:
I am trying to compile different styles or forms on how to evaluate the effectiveness of training (in-house or external).
Does anyone out here have info that they use and find to be rather effective???

Thanks

B Gammon said:

Good Morning Everyone:
I am trying to compile different styles or forms on how to evaluate the effectiveness of training (in-house or external).
Does anyone out here have info that they use and find to be rather effective???

Thanks
B.

First-Welcome to the Cove. This subject is covered plenty here. I don't want to brush you off, but please use the search function. Enter training effectiveness, or training and read all the threads. The answer you want is there, somewhere. There are several schools of thought on what constitutes training effectiveness. The simplest one, for me, is job descriptions with a signature stating the employee meets all the requirements for the job and no nonconformances traceable to that individual or department. But, as I said, there are varying opinions on what others think about this subject. Do the reading......come on back here if you don't see something you like.:bigwave: :ko: :smokin:

I am not sure what kind of ideas you are looking for, but I will tell you what I am currently working on. I recently delveloped a customer service training program and will begin to administer this week. Based on the information in the training (Attitude, Telephone etiquitte, tone of voice, etc. I deloped a checklist for each area. I am working on different scripts and will have people call other facilities presenting themselves as a customer, and verify if the people are now following the instructions based on the training.

Read thru 6.2.2 b) and then c), several times. If you provide training, then c) is not required. What is other actions? Pay cut, demotion, lay-off, verbal warning, poka-yoke, etc. I think the key is to TRAIN, TRAIN, TRAIN AND RETRAIN,RETRAIN, RETRAIN.

B Gammon,

Attached is a Training Specification form that was designed for a system that required this form to be completed before any training (internal or external) was to be approved. The aspect of how an evaluation is to be performed is determined up front.

Read thru 6.2.2 b) and then c), several times. If you provide training, then c) is not required. What is other actions? Pay cut, demotion, lay-off, verbal warning, poka-yoke, etc. I think the key is to TRAIN, TRAIN, TRAIN AND RETRAIN,RETRAIN, RETRAIN.

My earlier response in this thread was to the thread starter.

I am now pondering the relationship you stated here. I do not agree that if you provide training that c) then is not required. b) is provide training OR take other actions (training is an action in order for there to be 'other' actions). I believe the 'actions taken' in c) would be either the training or the other actions. So not only do I have to avaluate the effectiveness of training, but also the effectiveness of the hiring, demotion, lay-off, etc. performed in providing the Human Resources needed to implement and maintain the QMS (6.1 a) and meet customer requirements (6.1 b).

This would require that I have a Human Resource Management system with quality objectives to monitor and measure, but according to 6.2.2 e) I only have to maintain records of education, training, skills and experience!

How far off am I in this understanding?

Thanks,
Doug

Post Script: Just read 9004 and it seems to only center on "The education and training provided should be evaluated in terms of expectations and impact on the effectiveness and efficiency of the organization as a means of improving future training plans."

So another example where one standard says one thing and another leads you a different way. [Referring to 9001 7.3 states 'product,' but 9000 defines Design & Development as product, process, or system (3.4.4) and at the same time defines product (3.4.2) as a result of a process. - Quite contradictory in my reading and understanding.]

Training and qualification can only be verified by performing a competence assessment.

Bob A

Really? You mean I can't prove that I have been trained and qualified by showing you a record of it, or with one of these cool plastic cards in my pocket that have a bunch of mumbo jumbo on them, or through a license from a professional organization or regulatory agency on the wall?

How about verifying competency through some type of assessment based upon ones ability to apply the training in such a fashion to achieve the desired outcome from it through action. Through this process we can determine if one is qualified and competent.

One of the easiest ways to indicate that training was effective is by indicating in the management review those training activities that were effective.

If you have a performance appraisal system, there usually is a section for training or personal development. This is an ideal place to indicate that training was effective.

Another method is for the supervisor/manager to initial the training record indicating that it is effective.

In your manual or procedure you can also indicate that training can be determined effective upon receipt of a certificate.

Whatever method you choose, make sure that it is described in your manual or procedures.

I hope this helps.

Danny

I am now pondering the relationship you stated here. I do not agree that if you provide training that c) then is not required. b) is provide training OR take other actions (training is an action in order for there to be 'other' actions). I believe the 'actions taken' in c) would be either the training or the other actions. So not only do I have to avaluate the effectiveness of training, but also the effectiveness of the hiring, demotion, lay-off, etc. performed in providing the Human Resources needed to implement and maintain the QMS (6.1 a) and meet customer requirements (6.1 b).

This would require that I have a Human Resource Management system with quality objectives to monitor and measure, but according to 6.2.2 e) I only have to maintain records of education, training, skills and experience!

How far off am I in this understanding?
I would like to add one additional approaching concept (Formula) which used HR people do.

Kc = Ka/Kb, Sc = Sa/Sb, Pc = Pa/Pb

K = Knowledge,..... a = After Training
S = Skills,................. b = Before Training
P = Performance,.... c = Changes (=Effectiveness) :applause:

Source http://Elsmar.com/Forums/showpost.php?p=71703&postcount=11
My reply was http://Elsmar.com/Forums/showpost.php?p=72257&postcount=17

Introduce my strategy for training, we are develop "Identified Training Needs" separated by each department and modified after review by each team leader as quartely base or add new identified training need as it is required. This will be measured as I wrote above with cumulating some data.But the suitable methods is depend on your concetration on it.
(I believe there is no same way.)

I would like to add one additional approaching concept (Formula) which used HR people do.

Kc = Ka/Kb, Sc = Sa/Sb, Pc = Pa/Pb

K = Knowledge,..... a = After Training
S = Skills,................. b = Before Traing
P = Performance,.... c = Changes (=Effectiveness) :applause:

Source http://Elsmar.com/Forums/showpost.php?p=71703&postcount=11
My reply was http://Elsmar.com/Forums/showpost.php?p=72257&postcount=17


Alex.Jo,

How do you divide knowledge after training by knowledge before training? I understand that I can devise ways to compare (e.g. Pre-test & Post-test), but how do I divide?

Thanks,
Doug

Alex.Jo,

How do you divide knowledge after training by knowledge before training? I understand that I can devise ways to compare (e.g. Pre-test & Post-test), but how do I divide?

Thanks,
Doug

I'm also not sure Douglas how can divide before and after but one thing is very necessity that those training effectiveness must be measured.

I knew one methods, giving CCO (Certification Check Off) to trainee after evaluated his knowledge thru evolvable questions (or training contents).
But case of pre-test, it is little bit different. It is also be connected to competence.

The other useful techniques as you know, use Record.
Deming's emphasized "Recording" because it is starting to improve. Imagine that if certain manufacturing field would going to achieve continuous improve thru overcome current low yield status. Then they have to know first their status with monitoring and second generate records and analyzing those cumulated data(records) and understanding what is wrong. (For do that, QMS will provide statistical approach and help to establish acceptance criterion as well as define specification.)

Training Records, yield trends or rates, dispersion, man-hour control, ........

Our Knowing is two. One is direct/indirect experiencing and the other is recording. Indirect experiencing, I can say it is similar to training. And our text(teaching aids) also came from those cumulated experience by recording.

....
I'm very nervous for my poor english to deliver my idea or concept to foreign communities properly. Feel very bad. :frust:

I'm very nervous for my poor english to deliver my idea or concept to foreign communities properly. Feel very bad. :frust:

Alex.Jo, Stay with it, your English is good enough that I believe that we can understand the point that your illustrating. You are correct that we need to begin by recording both the knowledge base of the student before training, then the knowledge base after training. After which we can then use the formula the Jennifer provided to help us chart any improvement.
:agree1:

I am having a problem in maintaining training records. In particular, evaluations for effectivenss and re-training. We have a form for each position where we evaluate their skill, effectvieness and competancy on a yearly basis. The problem comes in when training is done , its a cumbersone approach tying in the re-training records.
We have a frequent changeover of operators which adds to the problem, and I am the only one doign the training and handling the records.

I am having a problem in maintaining training records. In particular, evaluations for effectivenss and re-training. We have a form for each position where we evaluate their skill, effectvieness and competancy on a yearly basis. The problem comes in when training is done , its a cumbersone approach tying in the re-training records.
We have a frequent changeover of operators which adds to the problem, and I am the only one doign the training and handling the records.Without being too prescriptive (use what works for you), it seems to me you can resolve part of the issue by maintaining a comprehensive folder for each employee with a check sheet in the front listing every possible training form, which you check off with the date when evaluated (additional columns for each subsequent annual competency review.) If it works for space, you might also keep the actual completed forms in the folder as well.

If you have more than a few employees, it makes sense to computerize the records, rather than maintaining hard copy files.

Whether hard copy or computer record, the next logical step is a master cross index file of each function being trained with a column for each employee, checked for the most recent date the competency was OKAYED.

At a glance, you can select candidates to fill a vacancy for a particular function. (Fine tuning the decision can be made by turning to the original competency review forms for those candidates.) You can also review which functions you are short on bench strength and recruit candidates to train for those functions.

If you aren't computer savvy (Excel or Access), a search of Shareware on the internet may turn up a usable program which you can easily adapt to your situation.

I thank you for your response. I don't quite understand why this is giving me so much trouble, but it is. Perhaps the problem is that the employee performance reviews annualy are kept one place and the evaluations for effectiveness and competancy are kept another place by a differant supervisor than myself.
In addition,each can identify differant short comings or training needs.
I need to show the training was done in either case. Employee performace reviews are usually where raises are considered and things like attendance
are carried out.

I thank you for your response. I don't quite understand why this is giving me so much trouble, but it is. Perhaps the problem is that the employee performance reviews annualy are kept one place and the evaluations for effectiveness and competancy are kept another place by a differant supervisor than myself.
In addition,each can identify differant short comings or training needs.
I need to show the training was done in either case. Employee performace reviews are usually where raises are considered and things like attendance
are carried out.Why not change your organization's process to combine the data in one folder? The key is:
the data applies to ONE person - why keep two separate sets of records for ONE person? It's sufficient that each supervisor have a cross-index to retrieve the record held in one location.

I thank you for your response. I don't quite understand why this is giving me so much trouble, but it is. Perhaps the problem is that the employee performance reviews annualy are kept one place and the evaluations for effectiveness and competancy are kept another place by a differant supervisor than myself.
In addition,each can identify differant short comings or training needs.
I need to show the training was done in either case. Employee performace reviews are usually where raises are considered and things like attendance
are carried out.

This is extremely common. Typically yearly employee reviews do not address 6.2.2 c). They are far too general, that is they tend to cover yearly goals. I've yet to find one that addresses specific training (or other action) that is addressed in 6.2.2 b). Is anyone really going to wait up to a year to tell someone that, despite training, they still are not competent?

Leave your yearly performance appraisals to the HR folks, and develop a system to show how training (or other action) met your competency needs. As I've stated before, there is no reason why regular production (or inspection) records cannot meet this requirement. I also submit, that you don't even necessarily have to keep records on this. I want a welder, so I hire a welder (it is far easier to hire a competent one, than to train someone to weld competently). When the new welder begins, I watch the output, and I monitor the welder. Should that person demonstrate competence, then I slack off on my monitoring. I evaluated the action taken by monitoring the person's on-the-job performance. Do I need a special form, or report? NO! What "evidence" do I have that 6.2.2 c) was met? The welder is employed, and is producing satisfactory work. What is the evidence for that? What kind of evidence do you think I could have to show the welding operation is satisfactory?

yes, the evalutions ( form for each position) used to fufill review of competance and training effectivenss will be filed indexed by employee and include the action taken if need be in the same folder.
However, I revised the trianing procedure to state that the "content " f the Enployee performance review is confidential info in regard to internal auditing.
What I hope to accomplish by that is that the dates as we state are auditable but not the content. That for TS 16949 , the training is evaluted by the position form. Any feedback on that one , would de beneficail to me.
I relaize I could make the employee performance review satisfy the TS16949 requirments but our mother company handles content on that form.

We use a Training Matrix. This lists the employees at the top and down the one side the jobs or training. We code the system as follows:
Require Training in red
In training in green
Qualified in black
Not Applicable in blue

It is very quick and easy to see who has had what training and can run what line. It is also very easy to use to show what training is required for future training sessions. Used during Management Reviews.

Each employee also has their own training record which is kept in a book and
signed off by the trainer. Effectiveness is evaluated by the work put out, the time utilized, and by the number of NCRs generated by that employee.

NCRs are a good source of information that will show if the operator has received the training they need and has understood that training.

Jan,

We use exactly the same matrix (table) you describe and it works well for us too! :agree1:

We use a Training Matrix. This lists the employees at the top and down the one side the jobs or training. We code the system as follows:
Require Training in red
In training in green
Qualified in black
Not Applicable in blue

It is very quick and easy to see who has had what training and can run what line. It is also very easy to use to show what training is required for future training sessions. Used during Management Reviews.

Each employee also has their own training record which is kept in a book and
signed off by the trainer. Effectiveness is evaluated by the work put out, the time utilized, and by the number of NCRs generated by that employee.

NCRs are a good source of information that will show if the operator has received the training they need and has understood that training.

US Navy uses these training elements with great success too, with the matrix at the shop level for easy reference and updating. Instead of colors, the matrix blocks (used to, anyway, as I recall them) would be marked diagonally, leaving two places for dates of class and quals due, which would be blackened in as the events are reached. Middle level records included performance on tests, quals and performance observations by the training coordinator (more or less critical skills needed more or less records to keep) and upper level employee documents were the evals, which recorded accomplishments and overall performance writeups in synopsis form. These records transferred with the sailors and impacted advancement.

Jennifer