What are some ways to demonstrate Effectiveness of Training?

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#11
Competence=satisfactory performance. The site determines what is satisfactory and how to record that attainment. Ideally there would be follow up to ensure key skills remain correct and properly implemented, but that does not always happen before a nonconformance reveals a problem.

People have different learning styles. Some do better with books, others with visual displays, but most of us truly "cement" our learning and skills with application (called "tactile learning" in educator's terms). Oversight and a final "walk-through" is therefore very often included in complex, critical skills attainment such as with certified operators in chemical plants.

When determining ineffective training and competency, we must first rule out other reasons for error: incorrect or insufficient inputs, incorrect/insufficient/unavailable reference documentation, insufficient/conflicting directives or objectives from management, incorrect/unreliable materials and/or equipment, etc. must be also considered so as to isolate training and competency as the true cause for error. Without robust process considerations, focusing on competency alone is an unreliable form of corrective action.

Let us also consider that competency can be about a physiological ability to perform. If a color blind person is expected to identify the correct colored wires and install into specified locations, we can expect trouble. I had a client that discovered that very thing, and resolved it though exacting staging of the components so the employee could succeed. Problem solved.
 
Elsmar Forum Sponsor

bobdoering

Stop X-bar/R Madness!!
Trusted Information Resource
#12
US schools have been trying for years to develop a meaningful, effective and accurate method for determining the effectiveness of education. Not only has that effort failed, in many ways the attempt to do it (proficiency testing) has caused an decrease in effectiveness of education! I don't state this as a political statement, but rather as a dire warning that if trained educators have yet to determine a valid methodology for this requirement, don't feel bad if you struggle to do so yourself. And, just because an international standards committee can write it, does not means that it is possible to meet the requirement. This will be a classic part of the standard that will be weaseled in a vast number of ways. When it comes to the three facets of lean - value added, non-value added and value maintained - this will wallow in non-value added for a long time.
 

Ronen E

Problem Solver
Staff member
Moderator
#13
US schools have been trying for years to develop a meaningful, effective and accurate method for determining the effectiveness of education. Not only has that effort failed, in many ways the attempt to do it (proficiency testing) has caused an decrease in effectiveness of education! I don't state this as a political statement, but rather as a dire warning that if trained educators have yet to determine a valid methodology for this requirement, don't feel bad if you struggle to do so yourself. And, just because an international standards committee can write it, does not means that it is possible to meet the requirement. This will be a classic part of the standard that will be weaseled in a vast number of ways. When it comes to the three facets of lean - value added, non-value added and value maintained - this will wallow in non-value added for a long time.
Education and training are not the same thing. Education is (at least) veeeeeery difficult to measure. BTW Education also doesn’t equal learning.

Perhaps the effectiveness of training can’t fully (or can’t efficiently fully) be measured, but for most orgs there is a lot that can be done towards that goal. For instance, how many orgs have you seen that sincerely and formally define measurable (where possible) training objectives for each and every training instance? Without doing that, the rest is irrelevant and non value adding (except for appeasing managers & auditors, maybe).
 
Last edited:

bobdoering

Stop X-bar/R Madness!!
Trusted Information Resource
#14
Education and training are not the same thing. Education is (at least) veeeeeery difficult to measure. BTW Education also doesn’t equal learning.
True the scope of education is much wider, and less specific. HOWEVER, it has the same underlying variables that relate to whether a trainer can train, whether an employee can learn, and there may be some rusultant of those two meta-factors that lies on some continuum from 0% to 100% of both.

Perhaps the effectiveness of training can’t fully (or can’t afficiently fully) be measured, but for most orgs there is a lot that can be done towards that goal. For instance, how many orgs have you seen that sincerely and formally define measurable (where possible) training objectives for each and every training instance? Without doing that, the rest is irrelevant and non value adding (except for appeasing managers & auditors, maybe).
The core issue is the "measureables" and the ability to gage with any accuracy the location on some representative continuum of "knowledge." Even tough when trying to evaluate aggregate success of a group of employees. I would say it is even difficult to get the absolute least resolution of an attribute verification of "training effectiveness" - "yes it is" or "no it is not". Simply attempt to put together the total variance equation of training, and determine the distribution of the individuals on its continuum to determine a "capability". Good luck. That will explain why it is far more difficult than the standard leads you to believe.

The standards do not ask for "a lot can be done' (quality level TCE). They expect auditable objective evidence. It really is a stretch to think it is truly possible within the skill levels of most manufacturing management.
 

Ronen E

Problem Solver
Staff member
Moderator
#15
The core issue is the "measureables" and the ability to gage with any accuracy the location on some representative continuum of "knowledge." Even tough when trying to evaluate aggregate success of a group of employees. I would say it is even difficult to get the absolute least resolution of an attribute verification of "training effectiveness" - "yes it is" or "no it is not". Simply attempt to put together the total variance equation of training, and determine the distribution of the individuals on its continuum to determine a "capability". Good luck. That will explain why it is far more difficult than the standard leads you to believe.

The standards do not ask for "a lot can be done' (quality level TCE). They expect auditable objective evidence. It really is a stretch to think it is truly possible within the skill levels of most manufacturing management.
I intentionally didn’t speak of measuring the effectiveness of training in general, but of that of specific training instances. Every training instance is just an act, it’s not supposed to be an all-encompassing mission. It’s objectives are what you define them to be. If you define easily-measurable objectives, it will be easy to measure and then determine whether the act was effective. If you don’t define objectives at all, or define them very poorly, it becomes mission impossible. If you tie training effectiveness with measuring competence, you’re shooting yourself in the foot.

I was saying “a lot can be done” half-sarcastically. Of course it’s not enough, but it’s at least necessary. What I meant is if you don’t take that first step you are guaranteed to drown in vagueness.
 
#16
It really is fairly simple:

You define required competencies for people to do a job
You have those people demonstrate their competencies
If some areas are lacking, you decide what to do about it, including training
When the training is done, the person demonstrates their (newly found) competency

It's not rocket science, people...
 

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#17
It really is fairly simple:

You define required competencies for people to do a job
You have those people demonstrate their competencies
If some areas are lacking, you decide what to do about it, including training
When the training is done, the person demonstrates their (newly found) competency

It's not rocket science, people...
That's basically it. In many cases the annual employee review is used to determine competency because employees initially have so little time to demonstrate their ability to perform without error. In practice these annual reviews are, however, largely worthless in my view, and we could argue a year is far too long to spot errors and correct the competency issues. This could be why just fixing an issue versus making a corrective action for discovered nonconforming output could be a missed opportunity.
:2cents:
 

Golfman25

Trusted Information Resource
#18
It really is fairly simple:

You define required competencies for people to do a job
You have those people demonstrate their competencies
If some areas are lacking, you decide what to do about it, including training
When the training is done, the person demonstrates their (newly found) competency

It's not rocket science, people...
Yep. Which seems to be to make the whole "training effectiveness" discussion redundant. You can't really look at the training in isolation to determine effectiveness -- ie; a test or something. You should take it one step further and look at the application of that training to the persons job.
 

Golfman25

Trusted Information Resource
#19
That's basically it. In many cases the annual employee review is used to determine competency because employees initially have so little time to demonstrate their ability to perform without error. In practice these annual reviews are, however, largely worthless in my view, and we could argue a year is far too long to spot errors and correct the competency issues. This could be why just fixing an issue versus making a corrective action for discovered nonconforming output could be a missed opportunity.
:2cents:
I don't know. Annual reviews are a convenient time to document competency issues. But in reality, I would hope, that "review" is on going. I mean we generally know who is good at what and who needs improvement in areas as we do our daily job. So if a person is floundering at a particular task, I would hope the evaluation and training (or whatever) would take place before the annual review.
 

Sidney Vianna

Post Responsibly
Staff member
Admin
#20
It really is fairly simple:

You define required competencies for people to do a job
You have those people demonstrate their competencies
If some areas are lacking, you decide what to do about it, including training
When the training is done, the person demonstrates their (newly found) competency

It's not rocket science, people...
I don’t think it is all that simple, in many cases. For example, it is a well known fact that most internal audit programs are poorly planned, executed and reported. One of the primary reasons being the fact that internal auditors are not developed and/or made competent, despite the fact that a good percentage of such internal auditors undergo some type of training. Sometimes, the incompetence is an outcome of an innate drawback, such as selecting someone who does not understand business processes or has poor communication skills, as an internal auditor. There is no amount of training that can overcome the knowledge and skills gap.

People have to realize that, sometimes, the training will not deliver the intended result. How many people became a millionaire after attending Trump University?
 
Last edited:
Thread starter Similar threads Forum Replies Date
S Ways to demonstrate objective evidence that employee is trained and competent ISO 13485:2016 - Medical Device Quality Management Systems 28
John Broomfield Five ways to botch your supplier management program Misc. Quality Assurance and Business Systems Related Topics 7
C Design Control - Ways of capturing inputs from manufacturing Other Medical Device Related Standards 5
S Ways to attract clients - Flyer distribution in Toronto, Canada Service Industry Specific Topics 1
J AS9100D - Good ways to document an internal audit plan / schedule Internal Auditing 25
L Ways to improve testing on our department - Project Kickoff Misc. Quality Assurance and Business Systems Related Topics 4
E Best Ways to Close a Project APQP and PPAP 3
F Supplier Audits - Creative ways to gain confidence in the next Supplier Training - Internal, External, Online and Distance Learning 6
M Can Six sigma support ISO9001? If yes, in what ways? Lean in Manufacturing and Service Industries 5
N Ways to Destroy Scrap Parts AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 10
A Creative ways to access Expert Information when you can't afford a top $ Consultant Misc. Quality Assurance and Business Systems Related Topics 7
C Appropriate ways to address Competency? Generic vs Specific ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
C Ways to check to see if the gage is being used General Measurement Device and Calibration Topics 11
AnaMariaVR2 3 Ways To Deal With People Who Play Office Politics Against You Coffee Break and Water Cooler Discussions 7
Marc Retail stores plan elaborate ways to track you After Work and Weekend Discussion Topics 0
C Which of these 2 ways would you implement a Quality System? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
S Change in Career Path - Logical Steps Up and Ways Forward Career and Occupation Discussions 3
R 6.2.2 - Any other ways to satisfy it if No Performance Evaluation carried out? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
K Recognition of team members in ways other than money? Customer Complaints 5
K Ways to Improve Document Control Document Control Systems, Procedures, Forms and Templates 10
L Creative and powerful ways of Communicating your QMS to your Employees ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
N Different Distribution Parameters when analyzing the same data in different ways Using Minitab Software 8
J What are the best ways to capture Customer Perception? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
bobdoering Dr. Wheeler on "How Measurement Error Affects the Four Ways We Use Data" Measurement Uncertainty (MU) 0
M Ways to encourage participation of other dept in preparing for the surveillance audit ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
D Other ways to Measure Customer Perception apart from Surveys ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
M Plastic Mold Approval - What are the ways to reduce mold approval cycle? Manufacturing and Related Processes 8
V Ways to Recycle Paper Miscellaneous Environmental Standards and EMS Related Discussions 15
P 5 Ways to Ensure Mediocrity in Your Organization Lean in Manufacturing and Service Industries 2
N Ways to avoid injected bugs while Bug fixing (software) Problem Solving, Root Cause Fault and Failure Analysis 5
A Ways of Gathering Customer Feedback ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
A Definition Modify, Magnify, Minify, and Substitute - Seeking practical ways of differentiation Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 17
S Lean - Ways to Measure Equipment Flexibility - Flexible Manufacturing System (FMS) Lean in Manufacturing and Service Industries 2
C 5 Ways to Find Product Standards AQL - Acceptable Quality Level 0
V ISO 9001 Virtual Audits - Ways to audit remote locations such as design centers General Auditing Discussions 6
R CMM Machine upgrade - Ways to reduce CMM inspection cycle time Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
D Ways where a Document Controller's Effectiveness could be enhanced. Document Control Systems, Procedures, Forms and Templates 2
K Creative Ways to Communicate Customer Complaints to Manufacturing. Customer Complaints 10
Y Perform 2 ways ANOVA in minitab with non normal data involved Using Minitab Software 5
T "Creative" ways to handle Early Production Containment Misc. Quality Assurance and Business Systems Related Topics 9
A How many ways do you classify your scrap? Quality Tools, Improvement and Analysis 15
apestate Vibration, Measurement and Isolation - Good ways to isolate M&TE from vibration? General Measurement Device and Calibration Topics 9
A Sampling Plans and Cpk - Ways to Calculate the Sample Quantity Inspection, Prints (Drawings), Testing, Sampling and Related Topics 15
R Looking for Advice on ways to collect Inspection Data Electronically Quality Assurance and Compliance Software Tools and Solutions 7
B 20 Ways To Annoy Your Public Bathroom Stallmate Funny Stuff - Jokes and Humour 7
K Inspection: Looking for ways to check 10 x 20 foot "checking aid" mylars Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
T Starting down the 14001 path - Ways to implement 14001 at my organization ISO 14001:2015 Specific Discussions 1
R Training to ISO 14001 - Effective ways to get the ISO information across to trainees Miscellaneous Environmental Standards and EMS Related Discussions 38
J ROI NPV HRV DIF or payback time? Ways to calculate cost benefit analysis Misc. Quality Assurance and Business Systems Related Topics 11
K Seeking Guidance on the most effective items and ways to Benchmark Preventive Action and Continuous Improvement 4

Similar threads

Top Bottom