How to avoid softgrading in System Audits

P

Popeos

#1
Hi guys ! (and ladies)

I am managing a team of internal auditors and we use a standard that is pretty close to iso ts 16949.

Auditors are trained, have to pass certification, there is a veto power; So basically the system is quite in place.

One of my major concern is softgrading :

- During examination / witness audits, I know that the findings are likely to be compliant with our expectations/the standard. (It is like a well known speed camera).

- Veto power also detects some discrepancies between observation and minor/major findings

But how can we improve our detection of softgrading ?

Any idea welcome

Popeos
 
Elsmar Forum Sponsor

Big Jim

Super Moderator
#2
It would help to define soft grading.

If you mean writing an observation when it is actually a nonconformance, then the cure is to not allow observations to be written if it really is a nonconformance. Save observations for when the practice is weak or awkward and could lead to a nonconformance although one was not found at that time.

If you mean writing a minor nonconformance when it is really a major nonconformance, then yhou need to define those terms. First of all, you do not need to so catergorize nonconformances for internal audits. If you choose (and you procedures are so written) all nonconformances can be simply nonconformances without further designation.

If you do use the terms major and minor when refering to nonconformances, then determine your definitions and stick to them. For example, a common definition of a major is when there is a requirement that you simply are not answering. A minor is commonly defined as the occassional time that you forgot to do something, but you do it most of the time. Sometimes it is called a major when it is likely to lead to nonconforming product escaping into the supply chain. Sometimes a batch of minors around the same topic can lead it being declaired a major under the doctrine of the "trivial many".

Before you start to define your terms, check the standard you are working with to see if it has any such requirements.
 
#3
Hi guys ! (and ladies)

I am managing a team of internal auditors and we use a standard that is pretty close to iso ts 16949.

Auditors are trained, have to pass certification, there is a veto power; So basically the system is quite in place.

One of my major concern is softgrading :

- During examination / witness audits, I know that the findings are likely to be compliant with our expectations/the standard. (It is like a well known speed camera).

- Veto power also detects some discrepancies between observation and minor/major findings

But how can we improve our detection of softgrading ?

Any idea welcome

Popeos
I'm a little confused about the veto power, witness audits etc. but to simply answer your question, the best way - since we are talking about internal audits - is to remove the grading!

Grading of internal audit NCs is, in my experience and from countless discussions here, a monster which has been created and now needs feeding!

The content and context of the audit report is far more important than any arbitrary (even if you do have definitions they will be 'interpreted') grade. or check mark in a box on an NC form.

I would keep away also, until you have highly competent auditors, the use of anything but a factual NC. Observations have always tended to be based on opinion, in my experience... Having an auditor justify a finding with "Well, it could happen..." isn't a great reason for management to do something, IMHO. They need facts and some indication of the likely risks, not 'feelings'...
 

Stijloor

Staff member
Super Moderator
#4
Hi guys ! (and ladies)

I am managing a team of internal auditors and we use a standard that is pretty close to iso ts 16949.

Auditors are trained, have to pass certification, there is a veto power; So basically the system is quite in place.

One of my major concern is softgrading :

- During examination / witness audits, I know that the findings are likely to be compliant with our expectations/the standard. (It is like a well known speed camera).

- Veto power also detects some discrepancies between observation and minor/major findings

But how can we improve our detection of softgrading ?

Any idea welcome

Popeos
Andy is right. No need to grade and/or differentiate between "major" and "minor" nonconformities.
Here is an example I use in my internal audit course:
Whether you have a small hole or a big hole in your tire (tyre) does not matter! Fix the darn thing!! :agree1:

Stijloor.
 

Howard Atkins

Forum Administrator
Staff member
Admin
#5
This is an example, one of many, whereby the Internal Audit system is trying to emulate the 3rd party audit.

The courses spend to much time on the formalistic aspects and not on the real purpose of the internal audit.

The purpose is to find where there are problems and it does not matter what you call them.

Much more important is the problem, worldwide of not performeing corrective actions rather than just corrections and in a large number of cases failure of horizontal deployment.

The internal audit should not be performed as the Lead Auditor for 2nd and 3rd party auditors is taught.
 

Jim Wynne

Staff member
Admin
#6
Hi guys ! (and ladies)

I am managing a team of internal auditors and we use a standard that is pretty close to iso ts 16949.

Auditors are trained, have to pass certification, there is a veto power; So basically the system is quite in place.

One of my major concern is softgrading :

- During examination / witness audits, I know that the findings are likely to be compliant with our expectations/the standard. (It is like a well known speed camera).

- Veto power also detects some discrepancies between observation and minor/major findings

But how can we improve our detection of softgrading ?

Any idea welcome

Popeos
As others have suggested, "grading" of audit results is a questionable practice. If an audit report is properly written, a reviewer in a position of authority should be able to either tell at face value how risky a finding is, or have enough sense to investigate if necessary. IMO, audit results should tell what what was expected and what was found, and in some cases what was found that wasn't expected, no more, no less.
 
P

Popeos

#7
Thanks to all

For Big Jim :

Using better definition for minor or major NC is one of the trail I am working on. Of course, I could remove minor and major, but I would also have to remove "notes/observations", as the risks is otherwise to have minor NC turned into "notes" or "observations".

For AndyN :
As regards "factual remarks", I totally agree, but depending on how you present the fact, it may remain a fact but still look different. Some facts may also be overlooked or forgotten. Analysis of current situations shows thatmy problem is not based on feelings but rather forgotten or wrongly described facts.

For Jym, Stijloor, Howard :
I totally agree that the key item is whether the non-conformity is adressed or not. However once the NC is correctly identified, making sure it is solved is quite easy to follow. But if the auditor considers that the tire just has a scratch whether it is in fact punctured, it is impossible to detect afterwhile as part of a monitoring activity of the quality of the audits.

One final comment is that I made a mistake when I wrote "internal auditor", but I wanted to mean that these auditors were part of the same company I am in. In fact the scope of audit is not internal, but our suppliers. That's why we grade the NC, to determine how risky a finding is.

Anyway thanks for your first comments. Please keep posting to help me avoid softgrading / too "gentle" auditors.

Popeos
 

Jim Wynne

Staff member
Admin
#8
Thanks to all

For Big Jim :

Using better definition for minor or major NC is one of the trail I am working on. Of course, I could remove minor and major, but I would also have to remove "notes/observations", as the risks is otherwise to have minor NC turned into "notes" or "observations".

For AndyN :
As regards "factual remarks", I totally agree, but depending on how you present the fact, it may remain a fact but still look different. Some facts may also be overlooked or forgotten. Analysis of current situations shows thatmy problem is not based on feelings but rather forgotten or wrongly described facts.

For Jym, Stijloor, Howard :
I totally agree that the key item is whether the non-conformity is adressed or not. However once the NC is correctly identified, making sure it is solved is quite easy to follow. But if the auditor considers that the tire just has a scratch whether it is in fact punctured, it is impossible to detect afterwhile as part of a monitoring activity of the quality of the audits.

One final comment is that I made a mistake when I wrote "internal auditor", but I wanted to mean that these auditors were part of the same company I am in. In fact the scope of audit is not internal, but our suppliers. That's why we grade the NC, to determine how risky a finding is.

Anyway thanks for your first comments. Please keep posting to help me avoid softgrading / too "gentle" auditors.

Popeos
If a finding is objectively written, why do you need grading? If findings aren't objectively written, how would you know that? What was it that prompted your concern?
 
#9
As regards "factual remarks", I totally agree, but depending on how you present the fact, it may remain a fact but still look different. Some facts may also be overlooked or forgotten. Analysis of current situations shows that my problem is not based on feelings but rather forgotten or wrongly described facts.
Facts are facts - unaffected by grading.

Forgetting or overlooking issues is nothing to do with grading, either.

The accuracy of a statement of Non-conformity has nothing to do with grading, but is a case of accuracy of note taking, analysis and reporting by the auditor. If you have an issue with auditor competency, I'd suggest you address that, again, grading or writing definitions is not going to help one iota.
 
P

Popeos

#10
First reply to your comments :

I know that findings are usually written as facts, because we have a monitoring of the content of the report, and we struggle to avoid feelings and poor description of facts.

Thanks to this activity, we detect part of the softgrading, when the grading of the NC is not consistent with the facts, for instance.

However some items can be described as facts, but may forget part of what has been seen in the field, or the description may state a different fact than what has been seen.

Regarding the training, I agree, and we are working on it, but this is not my point.

In fact, I should have asked HOW TO DETECT softgrading instead of HOW TO AVOID, because we already have defined many actions on occurence (and I will keep working on these with your comments in mind).

Besides, HOW TO DETECT may enable me to track drifts on the mid term range or after some evaluations.

I was thinking of statistical analysis of results, or double audits. Any clue about detection ?

Popeos
 
Thread starter Similar threads Forum Replies Date
K Is it possible to avoid the OQ? ISO 13485:2016 - Medical Device Quality Management Systems 20
D Using "Particle Size Standard" templates as gauges - How to avoid giving a gauge # while using for process control? General Measurement Device and Calibration Topics 2
M Informational How To Avoid Compliance & Timeline Risks When Selecting A Medical Device Supplier Medical Device and FDA Regulations and Standards News 0
supadrai Countries where you should (absolutely?) avoid appointing a distributor ... Other Medical Device Regulations World-Wide 6
J Draft Guidance document compliance mandatory to avoid a RTA? Other US Medical Device Regulations 2
A Supplier trying to avoid RMA's Supplier Quality Assurance and other Supplier Issues 11
L How to avoid Nonconformance Reoccurrence problems IATF 16949 - Automotive Quality Systems Standard 20
M Device Identification (UDI): How to avoid redundant info? Other US Medical Device Regulations 5
Wes Bucey How would you avoid this? Dust storm on I-80 in northern Nevada Coffee Break and Water Cooler Discussions 4
D Small Company wanting to avoid certain 21 CFR 11 Requirements US Food and Drug Administration (FDA) 8
D Help for BS EN 13980 to avoid recurring audits or testing for the ATEX certification Various Other Specifications, Standards, and related Requirements 3
C How to avoid ISO 13485 certification ISO 13485:2016 - Medical Device Quality Management Systems 9
L FDA GMP requirement: Avoid Non contact with floor? 21 CFR Part 110 US Food and Drug Administration (FDA) 1
N Ways to avoid injected bugs while Bug fixing (software) Problem Solving, Root Cause Fault and Failure Analysis 5
L How to Avoid Missed Welds - Job shop and weld several small quantity jobs Manufacturing and Related Processes 11
J Can any organisation avoid Identification of Key Characteristics AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 6
Jim Wynne Worm Alert - How to avoid computer worms and viruses After Work and Weekend Discussion Topics 4
M Complete revamp of quality systems required - Major pitfalls I should try to avoid Misc. Quality Assurance and Business Systems Related Topics 12
Wes Bucey Bosses from he!! [Heck...] - Identify and Avoid Career and Occupation Discussions 18
R The Legal Aspects of Auditing - Risks that auditors may face and practices to avoid Registered Visitor Articles Archive 2
S GM/VP softgrading internal audit finding - need feedback from an audit guru! General Auditing Discussions 11
M Go Live With New ERP System before Recertification Audit General Auditing Discussions 6
John C. Abnet Terms- Different Items in a system ISO 26262 - Road vehicles – Functional safety 0
T Controlling Expandable Forms in Paper-Based Document Control System Document Control Systems, Procedures, Forms and Templates 10
G Management Review (integrated system) Management Review Meetings and related Processes 17
M Unique Quality Management System for 2 sites ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
S Orthopedic Implants (Knee system) Sterilization Indicators Manufacturing and Related Processes 6
C Projects in the CAPA system Preventive Action and Continuous Improvement 6
E Insulation diagram for ECG monitoring and diagnosis system. IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
R Quality System Functional Safety Checklist / Guidance IATF 16949 - Automotive Quality Systems Standard 0
J Mastercontrol or other system ISO 13485:2016 - Medical Device Quality Management Systems 0
O Any info on release date of FDA “Computer Software Assurance for Manufacturing and Quality System Software” document? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 0
P MSA - what exactly mean "system" and master sample Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
P MSA study for visual system with artifical inteligence Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
D Big companies suffer from quality management system? ISO 13485:2016 - Medical Device Quality Management Systems 1
qualprod Assign a name to a home-made ERP system? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
H IEC 60601-1 ME equipment or ME system IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
K Biological indicator system classification under MDR EU Medical Device Regulations 2
N Chemical Testing on Medical Devices - Solutions in a container closure system (bag) EU Medical Device Regulations 1
J Audit Checklist for Integrated Management System for ISO 9001:2015, ISO 14001 & OHSAS18001 (IMS) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
R How would you work without a quality management system? IATF 16949 - Automotive Quality Systems Standard 3
J ISO 13485 System 'soft start' - How to best reflect this in initial audits, management review minutes and other records? ISO 13485:2016 - Medical Device Quality Management Systems 3
P Conformity assessment based on a quality management system or production quality assurance EU Medical Device Regulations 3
D EQMS - When the internet or system might not be available ISO 13485:2016 - Medical Device Quality Management Systems 7
Ron Rompen MSA on automated measurement system - Multiple Step Vision System Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
S Quality management system (Well head installation & maintenance) Quality Management System (QMS) Manuals 5
DuncanGibbons Model-Based procedures and Architecting the QMS as a System Document Control Systems, Procedures, Forms and Templates 2
P How to develop executable quality management system for rookies? IATF 16949 - Automotive Quality Systems Standard 9
Anonymous16-2 21 CFR Part 11 - Steps to take if we want to validate an electronic system Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 2
M Requirement to manufacture under a quality management system EU Medical Device Regulations 4

Similar threads

Top Bottom