Do We Have to Comply with Our Own Procedures?

[Jim Wynne]

Boris,

From today's news. What can happen when good people "following their procedures" for reporting child abuse allowed these procedures to trump the diagnosis of why the babies bones were so fragile.

http://www.lccsa.org.uk/news.asp?mid=71&ItemID=22690

Auditors rarely know ahead of the audit if the procedures are effective in avoiding adverse outcomes. This is why we should be careful to audit the monitoring of the processes against their written and unwritten procedures instead.

John

Several years ago at the same Chicago hospital where both of my children were born, a man died from gunshot wounds while just outside the door of the emergency room. Hospital employees were notified, but refused to attend to the dying man because to do so outside the building was prohibited by hospital rules. This, and the story you link to above, have nothing to do with the subject at hand.

I personally think it's OK for auditors to use a little discretion in these things, and to suggest, perhaps by way of documented OFI, that if it's permissible for a process to operated contrary to what the documentation says, the documentation should be updated to allow for such contingencies. This doesn't alter the fact that the organization's documents represent QMS requirements.

 

[Jim Wynne]

Jim,

Of course the procedures are audit criteria but first they are criteria for monitoring processes.

Some represent monitoring criteria, but others and perhaps most others, have nothing to do with monitoring, per se. They describe the requirements for operation of processes,.

As independent auditors we should not assume that the procedures are right.

That should be the null hypothesis, if there's going to be one, but an independent auditor should also not assume that the procedures are wrong. She should independently evaluate the processes in question using the documented requirements as criteria. What else is there to go on?

Therefore we involve members of the process team in showing us how they use their procedures to monitor the processes for conformity and effectiveness. We also examine evidence of corrections and corrective actions resulting from process monitoring.

Again there's a misbegotten assumption that this is all about monitoring, and it's not. If the documented instructions say, for example, that an assembly operation is to follow certain steps A, B and C in that order or something bad will happen, and the auditor observes the process being operated A, C, B, there is a nonconformity. There might be instances where the order actually makes no difference, in which case the documentation needs to be improved. In any event, if the documented requirements are not being fulfilled, look up the definition of "nonconformity" in ISO 9000.

That way we learn more about how well the management system helps employees to determine and fulfill requirements.

Command and control auditing to enforce procedures that are possibly ineffective or incorrect makes no sense to me.

It's not about "command and control auditing"; it's about objectively evaluating whether or not the documented requirements are fulfilled.

 

[Big Jim]

Auditors aren't there to make judgements but to report facts.

I hope you really don't mean that.

Isn't good judgement a critical element of being a good auditor?

How do they determine if a nonconformance is present if they don't make judgements?

Isn't judgement needed in determining the facts?

 

[Big Jim]

Again there's a misbegotten assumption that this is all about monitoring, and it's not.

It's not about "command and control auditing"; it's about objectively evaluating whether or not the documented requirements are fulfilled.

I agree it is not all about monitoring, but monitoring can't be ignored either. Determining if effective monitoring is taking place should be part of a balanced approach to auditing. After all, isn't monitoring part of the requirements?

 

[Jim Wynne]

You might want to do a hard refresh on your browser John (F5) as I have reverted to my given name.


I haven't seen it but will gladly have a look.
Auditors aren't there to make judgements but to report facts. I replied to your post because you made a sweeping and incorrect statement. You have gone on to insult me so I suggest you ignore my posts and I will ignore yours - except to say I disagree. Please feel free to do the same

I agree it is not all about monitoring, but monitoring can't be ignored either. Determining if effective monitoring is taking place should be part of a balanced approach to auditing. After all, isn't monitoring part of the requirements?

I agree 100%. My point was that Mr. Broomfield's focus seemed to be on monitoring only.

 

[John Broomfield]

I agree 100%. My point was that Mr. Broomfield's focus seemed to be on monitoring only.

Jim,

I am sorry for giving you that impression.

My focus is on auditors not appearing to enforce the procedures without any knowledge of their correctness or effectiveness.

This is why I suggest we should audit the process via the auditee's monitoring of that process against its procedure.

Of course we audit lots of other aspects of the management system as necessary to fulfill the audit objective.

John

 

[Jim Wynne]

Jim,

I am sorry for giving you that impression.

My focus is on auditors not appearing to enforce the procedures without any knowledge of their correctness or effectiveness.

This is why I suggest we should audit the process via the auditee's monitoring of that process against its procedure.

Of course we audit lots of other aspects of the management system as necessary to fulfill the audit objective.

John

I think you've created a false dichotomy. There's no reason that auditors can't determine whether or not the processes are being operated as designed and documented and evaluate effectiveness.

 

[John Broomfield]

I think you've created a false dichotomy. There's no reason that auditors can't determine whether or not the processes are being operated as designed and documented and evaluate effectiveness.

Jim,

I agree, but let us also acknowledge the fact that the auditee is meant already to have done this with their process monitoring.

An auditor reporting an employee for failing to follow an ineffective procedure helps no one and loses the auditor credibility.

I'd rather report the failure to monitor and correct the process against its procedure so either the process or the procedure or both and the process monitoring receive the corrective action.

Improved process monitoring reduces dependence on the auditor.

John

 

[Paul Simpson]

Jim. Yet again you misrepresent me by selective quoting. I appreciate we see eye to eye on next to nothing but you do realize that if you do this I will challenge you. So if we want to follow the whole thread of this particular argument, John said:

<snip>
Auditors rarely know ahead of the audit if the procedures are effective in avoiding adverse outcomes. This is why we should be careful to audit the monitoring of the processes against their written and unwritten procedures instead.</snip>

.. and I replied:

<snip>Auditors aren't there to make judgements but to report facts. </snip>

I hope you really don't mean that.

Isn't good judgement a critical element of being a good auditor?

How do they determine if a nonconformance is present if they don't make judgements?

Isn't judgement needed in determining the facts?

So when I said that auditors aren't there to judge it is in the context of John's post about the effectiveness of processes. Auditors have to evaluate impartially whether practice follows procedure (documented or otherwise). That includes many elements including achievement of process measures and compliance.

I took issue with John's sweeping statement.

Again the context of my reply is important. The thread title is 'Do We Have to Comply with Our Own Procedures?' - my answer is 'Yes - always.' This post was reinvigorated with a question from Steve about internal audit and I took issue (in particular) with John's statement:

Management may say "ISO 9001 says that you must obey our procedures" but that has been untrue since the year 2000.

This is just plain wrong.

So back to your post - even though it was based on an erroneous summary of what I have posted:

• An auditor makes a series of judgements based only on the facts presented.
  
  [*]Does the practice comply with procedure (documented or not)
  [*]Are objectives and process measures being met?[​

So whatever way you try to twist it.
I stand by my original post: If a procedure is judged to be important enough to document then any auditor (1st, 2nd, 3rd party) should be planning to evaluate whether people are working to it.

If John had started his response to Steve with 'Yes, but ...' I could have let it go but with the statement quoted above I had to exercise my right to reply.

 

[Big Jim]

Jim. Yet again you misrepresent me by selective quoting. I appreciate we see eye to eye on next to nothing but you do realize that if you do this I will challenge you. So if we want to follow the whole thread of this particular argument, John said:

.. and I replied:


So when I said that auditors aren't there to judge it is in the context of John's post about the effectiveness of processes. Auditors have to evaluate impartially whether practice follows procedure (documented or otherwise). That includes many elements including achievement of process measures and compliance.

I took issue with John's sweeping statement.

Again the context of my reply is important. The thread title is 'Do We Have to Comply with Our Own Procedures?' - my answer is 'Yes - always.' This post was reinvigorated with a question from Steve about internal audit and I took issue (in particular) with John's statement:This is just plain wrong.

So back to your post - even though it was based on an erroneous summary of what I have posted:

• An auditor makes a series of judgements based only on the facts presented.
  
  [*]Does the practice comply with procedure (documented or not)
  [*]Are objectives and process measures being met?[​

So whatever way you try to twist it.
I stand by my original post: If a procedure is judged to be important enough to document then any auditor (1st, 2nd, 3rd party) should be planning to evaluate whether people are working to it.

If John had started his response to Steve with 'Yes, but ...' I could have let it go but with the statement quoted above I had to exercise my right to reply.

I'm not twisting anything but I'm glad you cleared up your intention. Please note that I started off saying I hope you didn't really mean this.