AS9100:C PEARs (Process Effectiveness Assessment Report)

[John Broomfield]

Re: AS9100:C PEAR's (Process Effectiveness Assessment Report)

So the operators, supervisors, and managers have a "gut feeling" that their processes are effective? I'm curious how this is reported in management reviews, where management is required to review to ensure "suitability adequacy and effectiveness."



That's not what the auditor should have written. AS9100 (8.2.3) requires "These methods shall demonstrate the ability of the processes to achieve planned results." With no evidence, the auditor's finding should be "The organizations methods do not demonstrate effectiveness of the processes."

howste,

The auditor has the auditee's statement as evidence and AS9100C requires no documented record of process monitoring for effectiveness.

What evidence does the auditor have for making this your statement: "The organizations methods do not demonstrate effectiveness of the processes"?

Does this statement say the auditee is lying or do you have something more substantial for making this accusation?

As I have said AS9101D is ahead of AS9100C is specifying requirements. For the PEAR to work, AS9100C must be updated to first require records of the assessments of processes for their effectiveness.

John

 

[Joy]

Re: AS9100:C PEAR's (Process Effectiveness Assessment Report)

Clause 4.2.1 d may be a consideration for this.Measurement is also a process (refer clause 8.1).Also see clause 8.2.3 and 8.4.To meet all these requirements,organisations cannot help but have to establish systems to record the performance data.

Just check, whether not having established systems to record performance data is a violation of 4.2.1 d or not .

However I would appreciate Broomfield's point as there may be instances where auditee can say that it is not a requirement in the standard to maintain record of performance data but there is way to handle that

 

[howste]

Re: AS9100:C PEAR's (Process Effectiveness Assessment Report)

howste,

The auditor has the auditee's statement as evidence and AS9100C requires no documented record of process monitoring for effectiveness.

What evidence does the auditor have for making this your statement: "The organizations methods do not demonstrate effectiveness of the processes"?

This is all you gave me:

"...we monitor our processes for effectiveness but have no records of this"

Where is the evidence of process effectiveness in this statement?

 

[John Broomfield]

Re: AS9100:C PEAR's (Process Effectiveness Assessment Report)

This is all you gave me:

Where is the evidence of process effectiveness in this statement?

howste and Joy,

"I realize that you the auditor may want documented evidence but the system standard does not require it.

So, you will have to be satisfied with my statement (as corroborated by my colleagues) until you find evidence to the contrary."

"Also I am intrigued as to Joy's line of questioning, I am referring to 4.2.1d, 7.1d, 8.2.3 and 8.4c and we have found that exception reporting is all we need to maintain effective processes."

Next question please.

John

 

[howste]

Re: AS9100:C PEAR's (Process Effectiveness Assessment Report)

howste and Joy,

"I realize that you the auditor may want documented evidence but the system standard does not require it.

So, you will have to be satisfied with my statement (as corroborated by my colleagues) until you find evidence to the contrary."

I recognize that verbal evidence can be accepted in audits in the absence of something more verifiable. But a statement like the one you gave me ("we monitor our processes for effectiveness") states the method but does not demonstrate that the process is effective. There has to be more evidence.

Here are a few related requirements. The management review process requires inputs of process performance, and records of management reviews are required. 8.2.3 requires monitoring and measurement methods to demonstrate process effectiveness. 8.4 requires the organization to determine, collect and analyze data to demonstrate effectiveness, including trends of processes.

So, I can expect to find records of process performance at some frequency. I can expect to find data demonstrating effectiveness, including process trends. How would an organization do this without documented evidence?

"Also I am intrigued as to Joy's line of questioning, I am referring to 4.2.1d, 7.1d, 8.2.3 and 8.4c and we have found that exception reporting is all we need to maintain effective processes."

Who is "we" you're referring to? With "exception reporting" are you suggesting that as long as there are no problems that the processes are effective? How do we analyze trends? How do we demonstrate continual improvement? "No news is good news" is not the message that the writers of the standard intended. Look, for example, at Note 1 in the definition of customer satisfaction: "Customer complaints are a common indicator of low customer satisfaction but their absence does not necessarily imply high customer satisfaction."

By the way, what are the criteria that the organization is using for monitoring and measurement?

Next question please.

John

 

[Joy]

Re: AS9100:C PEAR's (Process Effectiveness Assessment Report)

In many cases the customers conducts audits of their suppliers using AS9100 as audit criteria during supplier selection and re-evaluation.My humble question,

Can the organisations (which believe that keeping records of process performance data is not a requirement of AS9100)state that the data is not available, as it is not a requirement?

Business exists if customers exist.If, business exists then third party audits are conducted.If, third party audit is conducted,then auditor may face the challenge of such question.So,I strongly believe the auditor would not face this challenge from organisations which are really in aerospace business and tier suppliers to big aerospace companies (Disclamer-purely personal belief).Such question may arise from organisation without aerospace customers or business;the standard can be used by any organisation.

However,it was good learning.

 

[John Broomfield]

Re: AS9100:C PEAR's (Process Effectiveness Assessment Report)

I recognize that verbal evidence can be accepted in audits in the absence of something more verifiable. But a statement like the one you gave me ("we monitor our processes for effectiveness") states the method but does not demonstrate that the process is effective. There has to be more evidence.

Here are a few related requirements. The management review process requires inputs of process performance, and records of management reviews are required. 8.2.3 requires monitoring and measurement methods to demonstrate process effectiveness. 8.4 requires the organization to determine, collect and analyze data to demonstrate effectiveness, including trends of processes.

So, I can expect to find records of process performance at some frequency. I can expect to find data demonstrating effectiveness, including process trends. How would an organization do this without documented evidence?


Who is "we" you're referring to? With "exception reporting" are you suggesting that as long as there are no problems that the processes are effective? How do we analyze trends? How do we demonstrate continual improvement? "No news is good news" is not the message that the writers of the standard intended. Look, for example, at Note 1 in the definition of customer satisfaction: "Customer complaints are a common indicator of low customer satisfaction but their absence does not necessarily imply high customer satisfaction."

By the way, what are the criteria that the organization is using for monitoring and measurement?

howste and Joy,

For the sake of our discussion between you two auditors and the auditees here is the "we" defined from above: "as operators, supervisors and managers we monitor our processes for effectiveness but have no records of this".

We continue with our answers in response to your demands for more evidence:

"We are glad you think this is funny", said the manager.

"I appreciate that as auditors you are struggling to support your demand (from a AS9100C requirement) for records of our assessments of process effectiveness but we have none."

"Instead, our process monitoring assesses the effectiveness of our processes in accordance with 8.2.3. If we find a process that needs correcting then we take care of the corrections and record any procedural corrections required on the relevant process document (router or instruction)."

"Occasionally we find a need for corrective action and we use our judgement for that. Overdue corrective action would end up with top management review as part of our process 'Investing in continual improvement'."

"So, perhaps the 'must be more evidence' you seek is in the form of annotated routers and instructions?

"Please do not accuse of us of being content with no problems. We actively seek opportunities to improve our processes with our multiple layers of monitoring and assessment."

"Can we go back to work now?"

John

 

[Joy]

Re: AS9100:C PEAR's (Process Effectiveness Assessment Report)

Thanks John.Sure we can start our works.You have missed my last question and I was expecting some great answer from you on that.

 

[howste]

Re: AS9100:C PEAR's (Process Effectiveness Assessment Report)

"We are glad you think this is funny", said the manager.

The laughing emoticon was a response to what I thought was an attempt on your part to be dismissive. I read your "next question please" as "I've said the final word on this, now we're moving on." I'm glad to see that wasn't actually the case.

"I appreciate that as auditors you are struggling to support your demand (from a AS9100C requirement) for records of our assessments of process effectiveness but we have none."

I believe if you reread all of my posts, you'll see that I've never asked your fictitious auditees for records. What I have asked for (repeatedly) is demonstration of process effectiveness.

"Can we go back to work now?"

Sure, when you've answered the questions I've asked instead of the ones you thought I asked...

 

[John Broomfield]

Re: AS9100:C PEAR's (Process Effectiveness Assessment Report)

What I have asked for... is a demonstration of process effectiveness.

howste and Joy,

"Sorry, I thought you had asked how we did that for ourselves not for you as auditors."

"OK, you'll see that each of our management system's key processes has a documented process objective to inform anyone who is competent to monitor the process of what to expect. Our routers also specify process control and verification criteria for effective monitoring and measurement."

"You are quite welcome to accompany any one of us (operator, supervisor, manager) in monitoring our processes for effectiveness."

"I understand that you as AS auditors have to complete process effectiveness assessment reports. I am glad that we do not have to keep such records. But we would appreciate a copy of each PEAR you complete and any recommendations that you may have."

John