Internal Audit Questionaires - Checklist that is outside the box

[Randy]

My only reason to post a list is remember when you first began, any list was welcomed, at least to me it was. I was only trying to help.

Keep on keepin on!

 

[AndyN]

A lot of people have mentioned process based audits. This was the sexy add on to the last version of ISO. It's OK I guess, but to just find out if the process is working, the old ways are still sometimes the best.

Just take the documented procedure as written and turn statements in to questions. "How do you review contracts? Can you please show me an example? How about another example?"

Your procedure says there is an approved supplier list, can you show it to me? Have you taken on any new suppliers lately? How were they evaluated? Can you please show me?

You can write answers right on the procedure and keep the forms and marked up procedure as your audit working papers.

If you have read their procedure before starting the audit, a simple "can you please tell me how purchasing works" will pretty much reveal if the system has been implemented as documented.

Caster:

Two comments on your points of view, if I may:

Process audits aren't an 'add-on' to the latest (last) version of ISO - I was required to do them as far back as the early 90's by the CB I was working for!

I'm not sure where this idea/practice of an 'element' approach audit came from but, like a lot of ISO related stuff, I guess it's just another example of folk-lore come true.

I understand your comments about the approach of taking the document and just 'follow your nose'! However, for many readers here that may present a challenge, since there may not be documentation for a process (don't forget ISO 9001 only requires 6), the qms processes may only be in the form of a simple 'map' of sequence and interaction......

I do agree and it's very true to say that an auditor should sit down and study before the audit, however, I feel you've overly simplified the approach and could mislead someone into forgetting that it's more than just compliance to a procedure we're required to audit.

 

[4Compliance]

Our company developed a very detailed audit checklist (a morph between FDA's QSIT and ISO 13485), not for the auditor (who is highly skilled and trained by FDA experts), but for the auditees.

It consisted of the usual questions on a typical auditor checklist, with specific examples of what the auditor would be looking for when any particular question was asked (in language the auditees could understand). We then distributed these checklists to the deparments that would be audited.

This approach appears to be working well with the auditee's in my company, as they are now aware of the expectations from their specific departments and how their department contributes to processes being audited. The auditee departments use these checklists to self-audit, find their own deficiencies, and correct these issues prior to their audits. They are more proactive as they can find deficiencies they weren't aware of before and actively try to show corrective action to the auditors when their scheduled audit is performed.

 

[Stijloor]

Our company developed a very detailed audit checklist (a morph between FDA's QSIT and ISO 13485), not for the auditor (who is highly skilled and trained by FDA experts), but for the auditees.

It consisted of the usual questions on a typical auditor checklist, with specific examples of what the auditor would be looking for when any particular question was asked (in language the auditees could understand). We then distributed these checklists to the deparments that would be audited.

This approach appears to be working well with the auditee's in my company, as they are now aware of the expectations from their specific departments and how their department contributes to processes being audited. The auditee departments use these checklists to self-audit, find their own deficiencies, and correct these issues prior to their audits. They are more proactive as they can find deficiencies they weren't aware of before and actively try to show corrective action to the auditors when their scheduled audit is performed.

That should not be necessary. If folks are competent and are complying to the "rules and regulations" established for their respective departments/processes, no checklists should be needed. The employees should be motivated by understanding what's good for the customer and the organization, not driven by looming audits and how to "pass" them.

Stijloor.

 

[Weiner Dog]

I truly agree- I've been auditing for over 25 years- be it with FDA, as an internal auditor, and now as a medical device consultant. One doesn't learn auditing overnight or by taking a 1 day course. It is learned through doing over and over- and fumbling at the beginning. I still learn when conducting audits, even though I've conducted over 2500 to date.

Checklists should be used as reference tools, especially when conducting process audits. If one only relies on a checklist (as a crutch), then he/she is not an effective auditor. FDA's QSIT is a good audit tool for medical devices. Since FDA's QSR was developed from ISO EN 13485 and ISO EN 13485 was developed from ISO 9001, QSIT can be modified for non device audits too. If QSIT is good for FDA, then it should be good for the medical device industry too as an internal auditing tool.

 

[Randy]

Our company developed a very detailed audit checklist (a morph between FDA's QSIT and ISO 13485), not for the auditor (who is highly skilled and trained by FDA experts), but for the auditees.

It consisted of the usual questions on a typical auditor checklist, with specific examples of what the auditor would be looking for when any particular question was asked (in language the auditees could understand). We then distributed these checklists to the deparments that would be audited.

This approach appears to be working well with the auditee's in my company, as they are now aware of the expectations from their specific departments and how their department contributes to processes being audited. The auditee departments use these checklists to self-audit, find their own deficiencies, and correct these issues prior to their audits. They are more proactive as they can find deficiencies they weren't aware of before and actively try to show corrective action to the auditors when their scheduled audit is performed.

Pardner, you're backwards in this approach. Essentially you're saying that you guys are promoting a negative based, reactive, corrective, wait until its broke system, instead of a positive, proactive, preventive one.

If things were running like they are intended to run you should be focusing on where conformance is being met and preventive action/improvement could be identifed. (Remember the phrase "implement & maintain"?) In fact if I remember correctly the internal audit is to be performed to determine the extent to which the system conforms to planned arrangements and is effectively implemented and maintained....Yes, I think I'm correct in this.

As for the "typical auditor" checklist/questions........I'm about as typical and common as they come and I don't even know what I'm gonna ask most of the time until I do.

Instead of all the shenanigans and foo-for-all why not just focus on relevant employee competency, which is all that's required anyway?

 

[Desara01]

Other thoughts on checklists:

I am a big fan of process-based audits (REAL process-based audits) and so I always point my auditors to the process flows. With a process flow they can see the inputs and outputs of each step, as well as who's responsible for perfomring that step. Since we all know that the handoff points are where we run into trouble, they can test these as well. Turtle diagrams are useful for examining the different components of a process but don't clue you in on the logical flow. Ideally, I like to have both.

In any case, encourage them to follow the logical flow of the process so that they can connect the dots. Cheers - hope this helps

 

[emendation]

You guys are brutal. The guy was just looking for some help, none of your comments are helpful if your just getting started. From now on I'll keep all my comment to myself as the pundits on this forum are well in control. Jeezzze.

I'll continue to lurk as there is a vast amount of knowledge here. Thank you.

All of their comments are helpful. I wish I had received some of their advice when I started seven years ago.

 

[RoxaneB]

When I trained internal auditors on how to audit our site's processes, part of the training was the audit preparation aspect. However, please allow me to take a step back and explain the foundation of our audit checklists.

Within our system, I had developed an audit checklist that addressed the requirements of ISO 9001 and ISO 14001. It was divided into the clauses and set-up as 'yes/no' questions that were, essentially, the requirements or " shall's " written in plain english.

Beneath each question, in italicized font, were sample evidences that could be found within our system. These samples were not all inclusive, but were to serve as guidance to the auditors on what to look for and, in a way, help them to frame their questions.

There were also checkboxes to be completed indicating if the processes (and, let's face it, the auditees' responses) were Satisfactory, Opportunity for Improvement, Unsatisfactory or Not applicable. A result of O or U would indicate that an Opportunity for Improvement Report or Nonconformance Report would be an output of the audit.

On the far right, however, was a column for comments and any additional prep work that the auditors wished to do.

And this brings us back to their training...

To ensure that our internal audit (which, while process based, was conducted annually as one large system audit for both ISO 9001 and ISO 14001) addressed ALL requirements, it was required that this checklist be used by all auditors for all of their work.

A matrix - developed by me as part of our management system development - showed how processes and clauses were applicable. So, let's say and auditor was assigned to audit the Human Resources process. The matrix would show which sections of the checklist (and thus the standards) were applicable to this process.

The auditor would review the requirements and, in the comments section, write questions that were specific to HR and in common english and meaningful to the auditor...if the auditor felt this was needed.

Some of the auditors had enough experience that they can simply look at the requirements and frame "tea time" questions like:

• What are your responsibilities?
• How do you know that?
• What do you do if....?

The newer auditors, as part of their training, would take these checklists and write out their questions in the comment section leaving room for answers to be recorded, as well.

This process not only helped the auditors to prepare but also demonstrated that our audit covered all processes and all requirements of the standards.

 

[BeeSting]

In September I started with a company here in Chicago and I used a similiar checklist to do some required "catching up" on audits that had not been performed by my predecessors. Although our TS auditor recognized the element-based checklists as a remedial response to a problem (prior to my tenure, the company received a minor for not completing internal audits), she did make a note that our organization required a better internal auditor training program.

In fact, I wanted her to make that note - It's me versus the world here so I had her mention and document a couple specific items that I wanted for back up.

I've been trying to get the idea across to my fellow management members that they've already proven that having a SCHEDULE means nothing if #1 no one is accountable to make it happen and #2 no one knows how to conduct the audits in the first place.

Taking the mystique out of the auditing process and making it something that ANYONE can do will probably be one of the most important things you can do.

If you don't have good process maps yet, you need to get management on board with writing them out. You will have actions (operator gets die from rack), inputs (operator looks at maintenance log to see which dies are in good order), Associated processes (tool room worked on that die and approved it) and outputs (a good part or a successful operation).

After you have everything laid out by "what is actually done", turn it into an easy-to-follow flow chart. Then go through the procedures and work instructions to make sure everything jives. If "what you do" is positive, effective, and well-controlled, make sure your procedures and work instructions reflect that.

Now you have a very simple flow chart layout that anyone can take and run with. As far as process auditing - it's all laid out right there. Your team brainstormed all the associated links. All you need to give the auditor is a flow-chart, a note pad, a pen, and a "starting point". A starting point is like "operator receives work router" or "QC dude finds a problem during last piece inspection" or "Sales rep gets a call about 47,000 bent parts".

------

"You don't pay me enough for me to sit here, day after day, listening to what you WON'T do." - BeeSting