Process based Audit Approach vs. Clause based Audit Approach

RoxaneB

Super Moderator
Super Moderator
#21
Re: Process Audit Approach vs. Clause Audit Approach

The better KPI would be performance to plan...
That was one of the KPIs I had in place "Audits Conducted" - which allowed for a planned forecast and then I would add in the acutal results. I also had "Assessments Completed". But both KPIs were merely a way of tracking what was done and not the overall results.

One way that we've started to work in the health of the system is through "Repeat Nonconformances". The issue that has come up is when is a Nonconformance a repeat issue? When the WHAT is the same or the Root Cause. But that's probably a separate thread, right? ;)

David DeLong said:
I would suggest using the turtle diagram to develop a process audit but that is about it. Of course I would keep the turtle diagram for "show and tell" when 3rd party auditors come it.
If you have anything in your management system simply for the sake of pleasing the auditor then there is a bigger problem to resolve. If turtle diagrams don't work, find something that does. If your auditor says no, ask them to call out the need for turtle diagrams.

More often than not, a standard tells us WHAT needs to be done and not HOW.

Classic point - Document Control. No where does it state that we need to use numbers to uniquely identify our documents. But the vast majority have gone that route. If they're all titled differently, then they are unique. And if you only have a handful of documents, titles may be good enough for you.

This is your organization's system. Make it work for your stakeholders...that includes your organization.
 
Elsmar Forum Sponsor

michellemmm

Quest For Quality
#22
Re: Process Audit Approach vs. Clause Audit Approach

My own opinion is that registrars have dropped the ball by not issuing major findings when it's clear that the process approach exists only in the turtle diagrams. It's no wonder that people are confused about what to audit, and often default to just making sure that the "shall" clauses are satisfied. There is no process approach until companies actually adopt it.
Jim,
To be fair to registrars, they cannot or should not issue a nonconformity....

DIS9001 states:

0.2 Process approach

This International Standard promotes the adoption of a process approach when developing, implementing and improving the effectiveness of a quality management system, to enhance customer satisfaction by meeting​
customer requirements..
Also,

4.1 General requirements
The organization shall establish, document, implement and maintain a quality management system and continually improve its effectiveness in accordance with the requirements of this International Standard.

The organization shall
a) determine the processes needed for the quality management system and their application throughout the organization (see 1.2),
b) determine the sequence and interaction of these processes,
c) determine criteria and methods needed to ensure that both the operation and control of these processes are effective,
d) ensure the availability of resources and information necessary to support the operation and monitoring of these processes,
e) monitor, measure (where applicable), and analyse these processes, and
f) implement actions necessary to achieve planned results and continual improvement of these processes.
These processes shall be managed by the organization in accordance with the requirements of this International Standard...

The use of words "PROMOTE" and "DETERMINE" is confusing to me... Shouldn't they say...."Require" instead of "Promote"?

:bonk:

"Determine" used to be "DEFINE" in ISO9001.... The impact of determination is not reflected in any other process.... Does determine equates to shall?

Maybe you can help me put things in proper prospective...
 

michellemmm

Quest For Quality
#23
Re: Process Audit Approach vs. Clause Audit Approach

That was one of the KPIs I had in place "Audits Conducted" - which allowed for a planned forecast and then I would add in the acutal results. I also had "Assessments Completed". But both KPIs were merely a way of tracking what was done and not the overall results.
Great Post!!!:applause:

One of the KPIs I tracked depicted the correlation between the result of self assessment score by process owners Vs. the result IA score. This what I call a reality check!
 

RoxaneB

Super Moderator
Super Moderator
#24
Re: Process Audit Approach vs. Clause Audit Approach

Clause 4.1 in ISO 9001 is, in my eyes, the executive summary of all the other clauses.

4.1 General requirements said:
a) determine the processes needed for the quality management system and their application throughout the organization (see 1.2),
By developing a QMS that complies and conforms to ISO 9001, you have determined the processes (i.e., document control, managmet review, operational control, etc.).

That and I would hope that an organization knows what processes are needed to make the product/service that the Stakeholders desire.

4.1 General requirements said:
b) determine the sequence and interaction of these processes,
There's a picture in ISO 9001 that shows the sequence and interaction of the clauses.

Surely an organization knows the flow of their own internal processes that make the product/service. Mapping it out always makes for a nice addtion to the manual and is a great training tool for new employees.

4.1 General requirements said:
c) determine criteria and methods needed to ensure that both the operation and control of these processes are effective,
Welcome to KPIs, monitoring and measuring practices, quality control, data analysis and then responding to any results that aren't what they should be.

4.1 General requirements said:
d) ensure the availability of resources and information necessary to support the operation and monitoring of these processes,
This is good business sense. A system is destined to fail if the committment is not there. Knowing the necessary resources is vital before you can ensure that they are available...this is where planning comes into play.

4.1 General requirements said:
e) monitor, measure (where applicable), and analyse these processes, and
Addressed by a clause later on in the standard.

4.1 General requiremens said:
f) implement actions necessary to achieve planned results and continual improvement of these processes.
Ditto.

4.1 General requirements said:
These processes shall be managed by the organization in accordance with the requirements of this International Standard...
In other words, check out 4.2 through 8.5.3. Essentially, if a system meets all the requirements everywhere else, then 4.1 has been met. I always save 4.1 for last when I do an audit. It's like that final checklist of the system's health.

And findings are assigned against the specific clause number unless there is a complete absence of the clause, then I find the corresponding 4.1 bullet.
 
P

pinpin - 2009

#25
Re: Process Audit Approach vs. Clause Audit Approach

Audit is also a process.

The purposes of audit are to find out whether:
1) we comply with policies, procedures, rules, contract requirements and TS requirements; and
2) the systems are effective.

As such, if "internal audit" is done well, it should be able to find nonconformities and areas that require improvement, unless the systems are too good that they are not any to find or the auditors are not competent enough. Of course, auditor shall find value-added things to be issued as a NC or to highlight areas that require improvement to strengthen the systems.

As such, I think the "number and types of NC found by "external" auditors" (CB or customer) could be use as an indication to show whether "internal audits" are well done and effective. These should then be used as KPI if it is deemed that KPI must be set for "internal audit".

If external auditors found NC on areas where the internal auditors have covered before, these could mean the internal audit was not effective enough, which could be due to internal auditors are not competent enough or the audit sampling was not good enough, or...etc.

So when we audit our own "internal audit process", we ourselves shall be competent enough to be able to look at whether the area that the NC issued by external auditors have been covered by our internal auditors. And if it is, we shall explore and identify whether our internal auditors are competent (i.e. they know what important things to check, the turtle diagram is properly fill out, evidence of conformance and effectiveness are recorded, not just for show only,...) and the audit sampling is proper (sample those important things and number of samples checked are commensurate with the importance and risks concerned), and etc so that they can find NC or areas for improvement. Then we shall "evaluate" or "assess" to determine the "effectiveness" of our internal audits base on our findings.

External auditors should be looking at whether our internal auditors are competent as mentioned above by looking at whether those NC they (the external auditors) found are due to inadequacy of our internal auditors or internal audit process, and issue findings or NC for things such as lack of competence and inadequacy of the internal audits; instead of merely look at whether we have performed numbers of audits as planned. I do not think the numbers of audits done or not done are the key thing we should focus and use as KPI as the "K" here refers to "Key".

Please correct me and give me your views. :thanks::thanx:
 

RoxaneB

Super Moderator
Super Moderator
#26
Re: Process Audit Approach vs. Clause Audit Approach

As such, I think the "number and types of NC found by "external" auditors" (CB or customer) could be use as an indication to show whether "internal audits" are well done and effective. These should then be used as KPI if it is deemed that KPI must be set for "internal audit".
I disagree with a "# of nonconformances" KPI, as well. It is subject to too many variables...not to mention I would hope that the goal is 0. If the result is greater than zero, action must be taken, but we're already taking action on the nonconformances (I hope) so now we're simply repeating ourselves.

If the goal is not zero, I really would question the expectation to receive nonconformances. There was once a joke about a company who said they would accept 3 defective parts out of 10,000 (or something to the effect). So, the supplier shipped 3 defective parts for every 10,000 shipped. If we expect 5 nonconformances, we'll get them.

For mature systems, the KPI of "# of major nonconformances issued by external" can be help as it's potentially indicative of a system becoming lax or complacent in its maturity.

Too often we see little nit-pit stuff issued as a minor and I certainly would not want to have a skewed KPI because someone was in a bad mood.

Personally, looking for the repetitive issues holds more value to me as it means we have not truly addressed the root cause.

pinpin said:
If external auditors found NC on areas where the internal auditors have covered before, these could mean the internal audit was not effective enough, which could be due to internal auditors are not competent enough or the audit sampling was not good enough, or...etc.
I understand what you are saying and, to some degree, agreee with you. But does this mean that every nonconformance detected by the external auditor, you also issue one internally to find out why the auditors did not locate it? Sounds like extra work and some serious finger-pointing.

Auditing is sampling. We know this. We can not be expected to see everything when we audit. This is the nature of auditing. This is the nature of the process.

pinpin said:
External auditors should be looking at whether our internal auditors are competent as mentioned above by looking at whether those NC they (the external auditors) found are due to inadequacy of our internal auditors or internal audit process, and issue findings or NC for things such as lack of competence and inadequacy of the internal audits; instead of merely look at whether we have performed numbers of audits as planned.
Again, I think you're almost right. External Auditors should be evaluating our system's requirements and determining if we are meeting the standard's requirements and actually doing what we say we will do. Of course there is the element of subjectivity where they must ascertain if our internal audit programme is effective, but again, the nature of auditing is sampling and we can not expect our internal auditors to see everything that the external will.
 

Jim Wynne

Staff member
Admin
#27
Re: Process Audit Approach vs. Clause Audit Approach

Jim,
To be fair to registrars, they cannot or should not issue a nonconformity....

DIS9001 states:
0.2 Process approach

This International Standard promotes the adoption of a process approach when developing, implementing and improving the effectiveness of a quality management system, to enhance customer satisfaction by meeting​
customer requirements..
It doesn't make any difference what the standard, or normative references, say if the company itself claims to be following the process approach.
 
C

CliffK

#28
Re: Process Audit Approach vs. Clause Audit Approach

What a pleasure to read this thread--so many thoughtful answers.:applause:

Like a previous poster, I like the matrix approach to audit planning. But I think you really need two matrices. One maps processes to departments (thus showing the application of processes throughout the organization, by the way.) The other maps elements of the standard to the processes. Using this approach, you can plan for appropriate levels of audit attention in three dimensions: organizational, elemental and process.

I'm a firm believer in the process approach. For too many years I was a recipient of "over the transom" design materials not to be. That said, I think the process approach can get a bit overblown.

For example, in some cases the process and element are nearly congruent. Purchasing and Product Design/Development are good examples. Also, most of the activity for these processes takes place in specific departments. So I'm not sure the distinction between process, element and departmental audits is meaningful in these cases.

Not all the third-party auditors get it when it comes to process audits. Now they use a blank turtle diagram instead of a checklist to write their notes on, but the questions are no more process-sensitive than before. Their reports use the word process a lot, but don't exhibit much awareness of inputs, outputs, transformations, controls, resources, interactions with other processes, feedback loops and all the other stuff that goes into truly understanding a process.

On the other hand it, does not seem to be a good use of time to jump from department to department looking for compliance to the document control procedures, unless there is reason to believe a problem exists with document control.
 
C

CliffK

#29
Re: Process Audit Approach vs. Clause Audit Approach

Afterthought. It seems like this process versus element versus department auditing question is one we can discuss for a long time.

But I'm not sure it's the right question to ask about any organization's internal audit program.

I think the right question is this: does the internal audit program yield useful information that the organization can use to improve itself in a meaningful way?

And maybe a follow up question: is there any way the organization can improve its current audit methods to yield better information?
 
2

20110108 Request

#30
Re: Process Audit Approach vs. Clause Audit Approach

Please read ISO 9001:2000, 0.2 Process approach, "This International Standard promotes the adoption of a process approach when developing, implementing and improving the effectiveness of a quality management system, to enhance customer satisfaction by meeting customer requirements." Please note the highlighted word: "promotes." There is a hugh difference between that word and "requires."
 
Thread starter Similar threads Forum Replies Date
V Internal Audit - Process based vs. Procedure based AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 11
RoxaneB Process-based auditing in 14001 - Seeking Reviews: Audit Schedule Content Feedback General Auditing Discussions 9
I ISO 13485 Practical Internal Audit Checklist - Standard vs. Process Based Internal Auditing 26
M ISO 9001:2000 Process Based Internal Quality Audit Plan General Auditing Discussions 11
A API Spec Q1 Purchasing Process - Supplier Reevaluation based on Supplier Risks 5.6.1.4 Oil and Gas Industry Standards and Regulations 17
S How to start with ISO process in project based team - Documentation Document Control Systems, Procedures, Forms and Templates 4
M Informational Critical Thinking and the Process of Evidence-Based Practice Medical Device and FDA Regulations and Standards News 0
T GRR based on part tolerance or process variation. Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 7
M APQC PCF (Process Classification Framework) and ISO 9001 - Processes Based Approach ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
K Changing from Procedure based to Process based Competency Evaluations General Auditing Discussions 5
A Suggestions for a UK based course in Process Auditing wanted Training - Internal, External, Online and Distance Learning 2
W AS9100C Process based manual Design and Development of Products and Processes 4
T Where in the ISO 9001 standard it asks for process based auditing? Process Audits and Layered Process Audits 6
G Validating a Process Based on Lot Release Criteria Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
T Are "process based" and craftsmanship mutually exclusive? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
Q Introducing Process based approach on Production Floor Manufacturing and Related Processes 6
M Seeking input for a process-based quality management system training course ISO 13485:2016 - Medical Device Quality Management Systems 1
B Sample Size based on Process Capability Indices (Pp/Ppk)? Capability, Accuracy and Stability - Processes, Machines, etc. 20
V Relevance of Process Based Auditing (PEAR) to regulated industry. US Food and Drug Administration (FDA) 3
M Changing PFMEA (Process FMEA) RPN Values based on a Rejection FMEA and Control Plans 5
J Integrating a Project Model in a Process Based Organization ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
Z Reduced Sampling based on Process Capability (AS9100 Standard) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
D Process4.biz and other Visio based Process Management Tools Human Factors and Ergonomics in Engineering 2
T Are MSA?s (Measurement System Analysis) Process or Gage Based? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 20
P Definition Dock-to-Stock Process based on Historical Quality Performance of a Part and Supplier Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 8
5 Process based Internal Audits, instead of auditing to the elements of the standard Internal Auditing 21
N I'm still not grasping the idea of Process Based Internal Audits Process Audits and Layered Process Audits 19
V Process Based Organization Chart that lists processes instead of departments? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 25
M PFMEA Severity - What is Process FMEA Severity estimation based on? FMEA and Control Plans 77
S Getting started with statistically based process improvement The Reading Room 12
Manix What provision is made for the Process of Implementing Project based work in TS16949? IATF 16949 - Automotive Quality Systems Standard 3
T Implementing the Process Based Approach - Reality check needed ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 15
F How to use Total Variation based on Process or Tolerance? Page 60 of the MSA manual Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
V Process approach based internal auditing Internal Auditing 1
A Where's the value add in product audits when your process is inspection based? IATF 16949 - Automotive Quality Systems Standard 9
C Simple four-box process model - Referenced in ASQ Process-based auditing course General Auditing Discussions 5
L 20 Factors based QMS to TS 16949 Process based Quality Management System IATF 16949 - Automotive Quality Systems Standard 1
Gman2 MY thoughts on 'process based' auditing ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 24
D Feature-Based Processing Recommended Business Process Guideline (THE-5) Various Other Specifications, Standards, and related Requirements 0
Marc Functional vs Process Based Documentation Document Control Systems, Procedures, Forms and Templates 13
J Need Help with FPY Data in Assembly Process Manufacturing and Related Processes 7
A When someone refuses to follow a process.... Misc. Quality Assurance and Business Systems Related Topics 21
E Software maintenance Process Software maintenance Process to IEC 6204? IEC 62304 - Medical Device Software Life Cycle Processes 3
R AS5553 Clause 3.1.7 f - "Implement a returns process....." AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
normhowe "The Problem with Quality Management: Process orientation, controllability and zero-defect processes as modern myths" Book, Video, Blog and Web Site Reviews and Recommendations 2
Judy Abbott General temperature used in the blasting process and laser process Manufacturing and Related Processes 2
B SOP for CNC turret punching machine for sheet metal process Manufacturing and Related Processes 0
A API Monogram audit review process Oil and Gas Industry Standards and Regulations 4
R AS9102 FAI Change in Material / Process Supplier AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 4
A Process mapping Process Maps, Process Mapping and Turtle Diagrams 1

Similar threads

Top Bottom