Company Conducted Internal Audit Offsite using a Document Review Process

Jim Wynne

Staff member
Admin
#51
<snip>
If not, I dare challenge anyone here to explain in a very objective manner how they would assess the effectiveness of an internal audit process. It is a complex, multi-faceted assessment which escapes scrutiny in most third-party audits, with VERY FEW exceptions.
As I suggested in an earlier post, the null hypothesis should be that the process is effective. It's a relatively simple thing to review the process as it's documented and look at the way the whole thing flows, which is what most auditors do. If that standard approach doesn't result in evidence against effectiveness, the null hypothesis is accepted, at least until evidence to the contrary is found. This same approach applies, or should apply, to the auditing any process. It's innocent until proven guilty.

What the OP is proposing, however, is the opposite--a decision has been made that the process is ineffective without any real evidence that it is in fact ineffective. The audited company is put in the position of having the burden of proof of innocence, when the burden should always be on the auditor to provide convincing evidence of guilt.
 
Elsmar Forum Sponsor

Sidney Vianna

Post Responsibly
Staff member
Admin
#52
As I suggested in an earlier post, the null hypothesis should be that the process is effective. It's a relatively simple thing to review the process as it's documented and look at the way the whole thing flows, which is what most auditors do. If that standard approach doesn't result in evidence against effectiveness, the null hypothesis is accepted, at least until evidence to the contrary is found. This same approach applies, or should apply, to the auditing any process. It's innocent until proven guilty.

What the OP is proposing, however, is the opposite--a decision has been made that the process is ineffective without any real evidence that it is in fact ineffective. The audited company is put in the position of having the burden of proof of innocence, when the burden should always be on the auditor to provide convincing evidence of guilt.
Well, Jim. In post #36 of this thread, Toni has provided objective evidence of a AS9100 requirement being clearly violated and not being reported in the internal audit results. How many more instances of glaring omissions, must a CB auditor collect, before deeming the virtual/remote approach to internal audits not effective? It is a subjective call and assessment.

ISO 19011:2011 (not a normative document) discusses the importance of balancing off and on-site activities when of internal audits performance.

While generalizations are always risky, it seems to me that the company in question wants to do the minimum possible in order to "pass" the external audit. It seems that previous auditors did not question the practice and Toni decided to.
 

Jim Wynne

Staff member
Admin
#53
Well, Jim. In post #36 of this thread, Toni has provided objective evidence of a AS9100 requirement being clearly violated and not being reported in the internal audit results. How many more instances of glaring omissions, must a CB auditor collect, before deeming the virtual/remote approach to internal audits not effective? It is a subjective call and assessment. <snip>
The OP states that the condition in question had been present "...for at least the last six months..." Are we now saying that all internal audits must account for all nonconforming conditions? Had there been an audit of handling of nonconforming material that could be shown to have missed the problem in question? If so, I think there's reason to question the effectiveness of the practice.
 

Sidney Vianna

Post Responsibly
Staff member
Admin
#54
Had there been an audit of handling of nonconforming material that could be shown to have missed the problem in question? If so, I think there's reason to question the effectiveness of the practice.
But that is exactly what I understand Toni is saying. A RECENT internal audit done via this non orthodox method was not able to identify the problem described in post # 36. Which exemplifies Toni's assertion that this type of internal audit is ineffective.

Anyone working for the CB community who thinks virtual/remote internal audits can be effective should be willing to do the same and audit registrants by phone, email, telepathy, tweeter, etc. The CAAT protocol is of LIMITED application and does not replace physical onsite audits.
 

Jim Wynne

Staff member
Admin
#55
But that is exactly what I understand Toni is saying. A RECENT internal audit done via this non orthodox method was not able to identify the problem described in post # 36. Which exemplifies Toni's assertion that this type of internal audit is ineffective.

Anyone working for the CB community who thinks virtual/remote internal audits can be effective should be willing to do the same and audit registrants by phone, email, telepathy, tweeter, etc. The CAAT protocol is of LIMITED application and does not replace physical onsite audits.
What was not said was whether the condition in question should have been considered to be within the scope of the audit in question. I think it is a good example of something that would not likely to be caught from a distance, and I think it's evidence that the effectiveness of the system is rightly questionable. That's what should happen--the process should be questioned and investigated--no argument there. What I am saying is that the OP has also clearly stated that he thinks it's a good idea to write nonconformities on ambiguous grounds--"grey areas"--and leave the organization to try to separate the grey into its black and white constituent parts. Not good.
 

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#56
I want to begin by reiterating my inherent suspicion of desk audits, even though I do perform them regularly. It's difficult enough to get it right in-person!

That said, I have asked for the specific write-up to understand precisely how the OP defined the nonconformance.

I also asked if a nonconformance to traceability requirements was also recorded - that would be two nonconformances for the same issue.

If there were two NCs written up for the same issue (5x5x5 box of mystery parts) is that acceptable as per registration guidelines?

Should audits drive conformity to the standard, in effect serving as an inspection process? Or should the organization's QMS be handling the lack of traceability issue?

Besides the comments I have seen almost zero as evidence of an audit's failure to physically observe this box, because I haven't seen the audit notes or spoken with the person who did the audits. But as an internal auditor I could have held an email conversation with responsible people and cited this lack of traceability based on their interviews. So is the remote aspect the true problem? I am not convinced. I am furthermore not convinced that these NCs will solve the true problem, which was that not only did this box of mystery parts exist but X number of people seemed to believe it was okay because they kept adding to it. That is a system problem, not an audit problem.
:2cents:
 

Sidney Vianna

Post Responsibly
Staff member
Admin
#57
What I am saying is that the OP has also clearly stated that he thinks it's a good idea to write nonconformities on ambiguous grounds--"grey areas"--and leave the organization to try to separate the grey into its black and white constituent parts. Not good.
I agree with that.

The "nonconforming until proven conforming" approach goes against the principles of auditing.
 

phxsun2001

Involved - Posts
#58
What was not said was whether the condition in question should have been considered to be within the scope of the audit in question. I think it is a good example of something that would not likely to be caught from a distance, and I think it's evidence that the effectiveness of the system is rightly questionable. That's what should happen--the process should be questioned and investigated--no argument there. What I am saying is that the OP has also clearly stated that he thinks it's a good idea to write nonconformities on ambiguous grounds--"grey areas"--and leave the organization to try to separate the grey into its black and white constituent parts. Not good.
The Project LA and CB reviewed the NCR and decided that it was a finding. Thanks for all the input.

Tony
 

Big Jim

Super Moderator
#59
The Project LA and CB reviewed the NCR and decided that it was a finding. Thanks for all the input.

Tony
Thanks for reporting the outcome. It is the response that I expected, including conferring with the CB.

That said, this does not mean that such a plan could not be part of a viable internal audit program, but, as you, I feel that there are some things that cannot be accomplished from a distance.
 

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#60
I still wonder if two NCs were issued for the same observation of the 5x5x5 box of mystery parts. Is the true issue of traceability going to be addressed?
 
Thread starter Similar threads Forum Replies Date
M Transferring ISO 17025 from one company to another ISO 17025 related Discussions 1
P Audit check for IT company (ISO 9001) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
lanley liao Does all of the suppliers need to integrated into the supplier list qualified of the company? Oil and Gas Industry Standards and Regulations 2
F IVD registration in EU - Northern Ireland based company EU Medical Device Regulations 0
W Where does a coatings and paint company fall in IATF? IATF 16949 - Automotive Quality Systems Standard 5
A AS9100D - Clause 8.1 Operation - Coating service company AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
O Informational Ford Motor Company Customer Specific Requirements for IATF 16949:2016 - 08 Jan 2021 Customer and Company Specific Requirements 0
L Have been purchased by a corporate company ISO 13485:2016 - Medical Device Quality Management Systems 7
R Advice needed: Shall I report my not complying company to NB / competent Authority (Europe) EU Medical Device Regulations 6
lanley liao What shoud i do if our company top management has been changed. Oil and Gas Industry Standards and Regulations 8
S Malcolm Baldrige Company Dashboard Quality Tools, Improvement and Analysis 3
T IATF Rules for sharing production space with another company IATF 16949 - Automotive Quality Systems Standard 10
J Leveraging another company's ISO 13485:2016 ISO 13485:2016 - Medical Device Quality Management Systems 5
R MDEL and company affiliate Canada Medical Device Regulations 0
E Our company is planning to file MDD not MDR next month. Do we require to show chemical characterization report ? CE Marking (Conformité Européene) / CB Scheme 2
S Is QMS like a set of rules and regulations that a company follows? ISO 13485:2016 - Medical Device Quality Management Systems 10
E Contract manufacturer FDA requirements foreign company US Food and Drug Administration (FDA) 6
J How much to charge for helping a startup company with initial ISO 13485 certification? Consultants and Consulting 3
J Sister-company providing parts is only ISO 9001 registered IATF 16949 - Automotive Quality Systems Standard 7
D IATF 16949 Requirement for CMMI in a Global Company Elsmar Cove Forum Suggestions, Complaints, Problems and Bug Reports 0
T Help to Suggest name for a new certification and inspection company Coffee Break and Water Cooler Discussions 7
M Address change for a company with CE/ISO13485 EU Medical Device Regulations 2
M IT validation for a paper based MD repair company QMS ISO 13485:2016 - Medical Device Quality Management Systems 6
M QMS for a repair/servicing company ISO 13485:2016 - Medical Device Quality Management Systems 2
C Internal Audits in a tiny Dx Company Internal Auditing 33
T ISO 13485 - 5.5.1 Responsibility and authority - Small Company Independence ISO 13485:2016 - Medical Device Quality Management Systems 13
F Quality manual for trading company ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
L Implementing the PRRC role in a company EU Medical Device Regulations 7
BeaBea ISO 9001 Customer Feedback Methods - What has worked for your company? Service Industry Specific Topics 17
M Customers Request AS9100 certification - Small Company (less than 20 employees) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
U Document Approval - Software company ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
T EQMS for small medical device company ISO 13485:2016 - Medical Device Quality Management Systems 18
qualprod Corona virus Contingency plan - What have you done in your company? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 25
Q Must product name be listed the same name in FURLS, UDI, GUDID and Company Website? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
E Sharepoint for ISO 13485 QMS for small IVD company ISO 13485:2016 - Medical Device Quality Management Systems 11
N Small Company - Internal audit process - Who does the audit? Internal Auditing 16
Z 510(k) usage - Company has 2 physically similar products Medical Device and FDA Regulations and Standards News 2
Q Company Ownership Change ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 13
L Contracted Manufacture Company wanting to be able to design and manufacture own product. 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 7
D ISO9001 for one man company ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
T Client Communication - SaaMD company Misc. Quality Assurance and Business Systems Related Topics 3
R Notified Body for MDD 1Q20 - Florida Company Registrars and Notified Bodies 4
E In need of a new TGA sponsor - Small software company Other Medical Device Regulations World-Wide 4
F ISO 17025 8.8 Internal Audits in a segmented company ISO 17025 related Discussions 5
E Company A supplies pharmaceuticals to the MOI - Who is responsible? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
J Quality Assurance in China - Developing a quality management system for a California company Misc. Quality Assurance and Business Systems Related Topics 9
S How to determine & document Organizational Knowledge of a company Document Control Systems, Procedures, Forms and Templates 4
U IT Process is Taken From company and Added to Corporate Structure ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
G Is ISO 9001:2015 certification worth it for a company that does only contract manufacturing? Quality Management System (QMS) Manuals 14
N Non traumatic edge - Remark in some of my company drawings EU Medical Device Regulations 1

Similar threads

Top Bottom