Driving Objectivity of Review Mechanism during Internal Audits

v9991

Trusted Information Resource
Observations & Recommendations, depend upon the maturity of "auditor & systems" and "state of focus on compliance". Situation is that, we are in multi-facility environment, with personnel involved at respective locations.

How do you drive objectivity of review mechanism during the internal audit?

What are the appropriate (leading and lagging) targets to be assigned for the internal audit process?


1) Drive objectivity we want to list-out a questionnaire for each procedure & process ( questions grouped in to "Correctness" - "Completeness" - "Compliance" (similar lines of adequacy and accuracy)...this way we could drive the next level of objectivity into the audittee/auditor-interactions.

2) Leading indicator Is "Average & repeat" "observations & recommendations" per each internal audit an valid approach to monitor the effectiveness of internal audit process.

3) Lagging Indicator : conventionally we, take the point that no-critical/major observations noted during external/regulatory audits; Can we taken...No-"RCA of incidents&Complaints" to be traced to the Internal audited-processes!!!.
 

qusys

Trusted Information Resource
Re: objectivity of internal audit mechanism.

Very difficult!
Impartiality and objectivity should be assured by the competence of the auditors ( as per ISO19011), even though some general common guidelines can help. Clearly there could be some differences related to QMS processeses in the multiple facilities environment, so on this case the guideline cannot be of help, in my opinion.
An alternative could be to perform crossed internal audit using a composite team of auditors that audit a different QMS of another facility they come from. It is called Corporate audit and you could have a procedure to manage this process. As to KPI, it is up to the organization . This is a suggestion of mine: if you want to evaluate the effectiveness and the efficiency of the internal audit activity, you can monitot the adherence to the established internal audit programme ( % of performed audit vs planned ones) as well as the number and the ranking of ncn coming from 3rd and 2nd party audits. If the internal audit is effective, you reasonally should have a reduction over the time of ncn coming from external parties. Hope this helps:bigwave:
 

AndyN

Moved On
Observations&Recommendations, depend upon the maturity of "auditor & systems" and "state of focus on compliance". Situation is that, we are in multi-facility environment, with personnel involved at respective locations.

How do you drive objectivity of review mechanism during the internal audit?

What are the appropriate (leading and lagging) targets to be assigned for the internal audit process?

1) Drive objectivity we want to list-out a questionnaire for each procedure & process ( questions grouped in to "Correctness" - "Completeness" - "Compliance" (similar lines of adequacy and accuracy)...this way we could drive the next level of objectivity into the audittee/auditor-interactions.

2) Leading indicator Is "Average & repeat" "observations&recommendations" per each internal audit an valid approach to monitor the effectiveness of internal audit process.

3) Lagging Indicator : conventionally we, take the point that no-critical/major observations noted during external/regulatory audits; Can we taken...No-"RCA of incidents&Complaints" to be traced to the Internal audited-processes!!!.

I'm not sure I understand the question here - do you want to ensure that your internal auditors are maintaining objectivity?
 

v9991

Trusted Information Resource
AndyN, Yes, that is correct...

probably things got mixed up:eek: because i listed few things I am planning to do..

1. questionnaire/checklist would provide guidance on minimal path.
2. leading would drive towards effectiveness of execution of checklist/reviews.
3. lagging indicators would improvise the system/implementation.
 

somashekar

Leader
Admin
Observations & Recommendations, depend upon the maturity of "auditor & systems" and "state of focus on compliance". Situation is that, we are in multi-facility environment, with personnel involved at respective locations.

How do you drive objectivity of review mechanism during the internal audit?

What are the appropriate (leading and lagging) targets to be assigned for the internal audit process?


1) Drive objectivity we want to list-out a questionnaire for each procedure & process ( questions grouped in to "Correctness" - "Completeness" - "Compliance" (similar lines of adequacy and accuracy)...this way we could drive the next level of objectivity into the audittee/auditor-interactions.

2) Leading indicator Is "Average & repeat" "observations & recommendations" per each internal audit an valid approach to monitor the effectiveness of internal audit process.

3) Lagging Indicator : conventionally we, take the point that no-critical/major observations noted during external/regulatory audits; Can we taken...No-"RCA of incidents&Complaints" to be traced to the Internal audited-processes!!!.
Seems that you are keen to get the internal auditors competency to some bench marks. How good are your internal auditors ? How much they understand about the purpose and depth of internal audits within the QMS functions?
While a grading of Internal audits is not necessary and we have had long threads about this topic here, what you mention is more perhaps the key role an MR plays in the post audit analysis, and taking inputs for next audit plans.
If the internal audits are performed in its true sense and all internal auditors are seasoned enough to show consistency of audit depth., your said points are well addressed. An other aspect of internal audit is the audit of previous audit report and findings, which at most places slips away or is slidelined by internal auditors. If this is peformed with deeper questioning, your indicators question gets well addressed.
we want to list-out a questionnaire for each procedure & process ( questions grouped in to "Correctness" - "Completeness" - "Compliance"
Internal auditors will be too happy with this and turn out similar reports audit after audit and also get into offline desktop audit reporting. The objective of assessing the PDCA in every process that is audited seems to get curbed and soon the auditors are handicapped when no questionnaire is made available.
 

AndyN

Moved On
AndyN, Yes, that is correct...

probably things got mixed up:eek: because i listed few things I am planning to do..

1. questionnaire/checklist would provide guidance on minimal path.
2. leading would drive towards effectiveness of execution of checklist/reviews.
3. lagging indicators would improvise the system/implementation.

I don't believe that what you are thinking about will be any indication of objectivity. A checklist doesn't ensure that what the auditor is looking at, as audit evidence, is objective!
I'm not sure what you mean about the 'lead' and 'lagging' indicators.

IMHO, 'objectivity' is a human trait which can be determined best by observing an auditor in action:
Their line of inquiry and questioning
Their gathering of facts and evidence
Their analysis of these facts and evidence
Their reporting of the conclusions drawn

Anything else isn't going to be much good and may be after the fact...
 

v9991

Trusted Information Resource
Its like frustating... Being aware of the situation and able to do nothing about it...
I agreee, No amount of procedures/checklist/metrics could replace the individual's objectivity/sense-of-purpose to any activity.

But this is a small effort from my side to nudge the team towards right direction. Moreover, management is going to ask, what have you done in-house before recommending people for audit-course; + how do you ensure that after training& learnings (people might move on...), are internalized & sustained & measured. Above efforts are in that direction. Of course they will need to evolved specific to situations evolving over time.

I'm not sure what you mean about the 'lead' and 'lagging' indicators.

They are supposed to be providing, an indicators of effectiveness/progress of the initiatives. leading indicators are something like, knowing how much of desired progress is made (assessment before writing an exam!!!)... lagging indicators are measuring the effect/impact due to initiatives.
 

Jen Kirley

Quality and Auditing Expert
Leader
Admin
Objectivity is, in my view, not met by understanding technical terms (I barely comprehended what you listed and described :eek:) but by a heart felt, intelligent and disciplined philosophy to building/facilitating organizational excellence.

Skill is a different subject. I do my auditing on three factors: Definition, implementation and effectiveness. This can be accomplished with checklists, but after auditors get much experience these checklists may offer little more benefit than ensuring nothing important gets left out.
:2cents:
 

Colin

Quite Involved in Discussions
I agree with the previous posts, I think you are trying to make a 'mechanical formula' for what is a human skill. I would prefer to spend my time developing the internal audit team to look for what you really want out of them. Compare findings, discuss problems and look for improvements in the audit process - it is a 'people based' skill.
 

AndyN

Moved On
Its like frustrating... Being aware of the situation and able to do nothing about it...
I agree, No amount of procedures/checklist/metrics could replace the individual's objectivity/sense-of-purpose to any activity.

But this is a small effort from my side to nudge the team towards right direction. Moreover, management is going to ask, what have you done in-house before recommending people for audit-course; + how do you ensure that after training & learning (people might move on...), are internalized & sustained & measured. Above efforts are in that direction. Of course they will need to evolved specific to situations evolving over time.

So help us out...what is your objective? It will, IMHO, be no 'small effort' but will make a 'rod for your back' to try to develop such indicators. It sounds to me as if your management don't want to invest in internal audits. Have the auditors already had class time? What are you seeking to accomplish with more class time? If you find the root of the issue it'll be better to address that - if management don't support IA, then THAT's what you must address - and we can help you!!
 
Top Bottom