Benefits of an Out of Box Audit on High End IT Products

[wallen - 2008]

Hello all, my company as a process step carries out final / out of box audits on high end IT products and I have a number of concerns regarding the benefit of the existing methodology. The current process takes 3 units per week regardless of overall volume (statistically representative? – I think not), each unit produced can be differently configured, so all sub units and processing steps may not be the same. The out of box audits are actually suspended during the quarterly peak periods when 65%+ of product is shipped.
I am suggesting that the audits adds very little value, only show the quality of the units inspected and no real inferences can be made of the overall population or the implied quality of the process. Out of box audits is a little old school trying to detect a problem at the end of the process. I am trying to sell a more proactive methodology which focuses on a cross functional knowledgeable team, walking the process and use quality tools such as FMEA etc to identify weaknesses, implement improvements or at minimum identify detection methods to catch the defect as it occurs.
Just throwing my ideas out their for comment or possible suggestions of alternative solutions?
P.S there is a lot of inhouse political support for keeping the existing so I need a flame proof,water tight,anti venom coated, bullet proof alternative process before Ienter the arena.

 

[AndyN]

Re: out of box audit

Wallen:
In my experience, you're on the right course. And, yes, it's going to be a battle trying to convince people who are vested in old fashioned detection techniques. There are two items of armour that you can use - data and cost.

The data should show that doing these audits didn't detect any of the kinds of things that actually made it to the customer and the cost can be shown in terms of prevention (doing the FMEA, for example) to the cost of detecting problems which, even though you do these o-o-b audits, don't catch everything and finally, the costs of warranty, customer dissatisfaction/complaint handling etc. This is just an idea and there's probably much more that you know goes on in dealing with the day to day dross that the proactive approach would reduce.

Another consideration is finding a champion in management who either has a vision for another way, or would make a willing disciple.......

Good luck!

 

[Helmut Jilling]

Re: out of box audit

Hello all, my company as a process step carries out final / out of box audits on high end IT products and I have a number of concerns regarding the benefit of the existing methodology. The current process takes 3 units per week regardless of overall volume (statistically representative? – I think not), each unit produced can be differently configured, so all sub units and processing steps may not be the same. The out of box audits are actually suspended during the quarterly peak periods when 65%+ of product is shipped.
I am suggesting that the audits adds very little value, only show the quality of the units inspected and no real inferences can be made of the overall population or the implied quality of the process. Out of box audits is a little old school trying to detect a problem at the end of the process. I am trying to sell a more proactive methodology which focuses on a cross functional knowledgeable team, walking the process and use quality tools such as FMEA etc to identify weaknesses, implement improvements or at minimum identify detection methods to catch the defect as it occurs.
Just throwing my ideas out their for comment or possible suggestions of alternative solutions?
P.S there is a lot of inhouse political support for keeping the existing so I need a flame proof,water tight,anti venom coated, bullet proof alternative process before Ienter the arena.

I would agree with your premise and most of your points. It has some value, but limited. After the fact inspections are certainly reactive.

Walking the line with a notepad and pen is always useful. By all means, try to get a small group together and try it. It is a little bit of Kaizen and Lean Mfg.

Doing FMEAs might be useful, depending on the current defect levels. If already extremely low, a FMEA may not add much more benefit.

I would not try to debate something like this, or win an argument. Let the data speak. Analyse the current results. Are they finding any problems? If yes, then they are useful, and you have process problems (root causes) which must be fixed. If the dtaa shows no findings, then apprently you have outgrown this method and it no longer provides much benefit.

Don't debate the method, discuss the outcome you want to achieve, and develop a method that will achieve it.

 

[qualityboi]

Ok, here is a biased opinion of a former OBQ supervisor. It really depends if you use your OBQ as just an inspection point or do you use it as a customer experience. We made high end computers for the likes of univerisites and research facilities such as Jet propulsion laboratories. We were computer geeks that loved computers, we checked everything from computer setup, documentation and even used the computers with various programs. Information was fed back to many areas. We even had the packaging design changed because of the difficulty of possible injury getting the computer out of the box. We won many awards and had great profit sharing checks during that time you may remember Edge / Micron Electronics computers in the early - mid 90s. New management came in, had us build cookie cutter PCs and removed final test inspection (with no replacement to find errors, they said exactly what you said "can't inspect quality in" and limited the OBQ. The entire operation went south and when the CEO announced the death of the PC, the company almost went under. Building quality in and robust design are really great, but you can only do that with data, good data from a pure source, customer data can vary depending on how it is collected and often too late. The human element is the variable and depending on what you are making depends on if you need inspection or not. Remember if you move your inspection from the quality guys and have manufacturing do it, it is still inspection! Round up everything that the group finds, then figure out how much it will cost in returns and customer loyalty. Better yet tell your customer, the one that audits you the most whaty they think about getting rid of the group and see how they react. I wouldn't get rid of it until you find an effective process that will replace the positive impact the OBQ process has on the product.

 

[Helmut Jilling]

Ok, here is a biased opinion of a former OBQ supervisor. It really depends if you use your OBQ as just an inspection point or do you use it as a customer experience. We made high end computers for the likes of univerisites and research facilities such as Jet propulsion laboratories. We were computer geeks that loved computers, we checked everything from computer setup, documentation and even used the computers with various programs. Information was fed back to many areas. We even had the packaging design changed because of the difficulty of possible injury getting the computer out of the box. We won many awards and had great profit sharing checks during that time you may remember Edge / Micron Electronics computers in the early - mid 90s. New management came in, had us build cookie cutter PCs and removed final test inspection (with no replacement to find errors, they said exactly what you said "can't inspect quality in" and limited the OBQ. The entire operation went south and when the CEO announced the death of the PC, the company almost went under. Building quality in and robust design are really great, but you can only do that with data, good data from a pure source, customer data can vary depending on how it is collected and often too late. The human element is the variable and depending on what you are making depends on if you need inspection or not. Remember if you move your inspection from the quality guys and have manufacturing do it, it is still inspection! Round up everything that the group finds, then figure out how much it will cost in returns and customer loyalty. Better yet tell your customer, the one that audits you the most whaty they think about getting rid of the group and see how they react. I wouldn't get rid of it until you find an effective process that will replace the positive impact the OBQ process has on the product.

I think your story has good, interesting points. But Edge/Micron did not fail because OBQ inspections were stopped. Perhaps it played a role, but if that led to its demise, there had to be much more going on.

 

[wallen - 2008]

Thanks for your comments. To summarize it looks as if I’ve lost the battle and OOB audits are to remain as part of the process. Although I have not given up on the war just yet, I will continue to gather the data and beef up my argument while still looking for a management champion willing to climb above the parapet and fly the quality flag. As a consolation I have at least got a review of the audit check list to validate that product characteristics inspected are still relevant to the customer.