Hi,

I have just received a process capability document from a new supplier based in another country. At first look it seems fine, no problems capability ok for all dimensions etc. I decided to cut and paste the data into minitab just to check the run charts etc.

To my shock I discovered that the charts showed that the data for most but not all of the dimensions had been cut and pasted a few times so the chart just repeats itself.

Now this type of cheating can be discovered, but it got me thinking if there is anyway of confirming if the data is real if they had used a random number generator as is availble in Minitab.

All trust is now lost in this supplier and I am curently deciding what to do with them.

The easiest way to tell whether the data were randomly generated in Minitab is by the number of decimal places. It is easy to generate random data to a ridiculous number of decimal places. However, it is more difficult to generate random data to a specified number of decimal places equal to the resolution of the gage.

If the data are copied and pasted from Minitab to Excel and the decimal places set to match the gage resolution, the full value from Mintab is still in the cell. Just reset the number of decimal places shown to a large number to view them.

Note: this method is not foolproof, but few people know how to get past this, or bother to make the effort.

Hi,

I have just received a process capability document from a new supplier based in another country. At first look it seems fine, no problems capability ok for all dimensions etc. I decided to cut and paste the data into minitab just to check the run charts etc.

To my shock I discovered that the charts showed that the data for most but not all of the dimensions had been cut and pasted a few times so the chart just repeats itself.

Now this type of cheating can be discovered, but it got me thinking if there is anyway of confirming if the data is real if they had used a random number generator as is availble in Minitab.

All trust is now lost in this supplier and I am curently deciding what to do with them.

Our friend and Moderator Mr. Sidney Vianna has the answer in this post + attachment (http://elsmar.com/Forums/showpost.php?p=337073&postcount=5). ;)

Stijloor.

Hi,

I have just received a process capability document from a new supplier based in another country. At first look it seems fine, no problems capability ok for all dimensions etc. I decided to cut and paste the data into minitab just to check the run charts etc.

To my shock I discovered that the charts showed that the data for most but not all of the dimensions had been cut and pasted a few times so the chart just repeats itself.

Now this type of cheating can be discovered, but it got me thinking if there is anyway of confirming if the data is real if they had used a random number generator as is availble in Minitab.

All trust is now lost in this supplier and I am curently deciding what to do with them.

As a respect to the folks from the other country I would had avoided saying the country name from where you received the report, at the end that is irrelevant. This in my opinion avoid a possible missinterpreation, but I am sure that was not intentional. In some cultures we could be more sensitive than others (I am speaking for myself) so I believe it would had been nicer to write the post a little differently with a more impersonal touch.

If indeed the PPAP data was a "cut and paste" it could even happen as a result of a mistake. Please if possible ask your supplier why this happened.

Hi,

I have just received a process capability document from a new supplier based in India. At first look it seems fine, no problems capability ok for all dimensions etc. I decided to cut and paste the data into minitab just to check the run charts etc.

To my shock I discovered that the charts showed that the data for most but not all of the dimensions had been cut and pasted a few times so the chart just repeats itself.

Now this type of cheating can be discovered, but it got me thinking if there is anyway of confirming if the data is real if they had used a random number generator as is availble in Minitab.

All trust is now lost in this supplier and I am curently deciding what to do with them.
The presense of falsified data is real in many reports.An experienced auditor can detect them.But he cannot charge the organization unless he has solid evidense.There should be provision in the standard to get the parts inspected in the presense of the auditor.
I think the process audit requirement should be made more stringent.
V.J.Brahmaiah

Hi,

I have just received a process capability document from a new supplier based in India. At first look it seems fine, no problems capability ok for all dimensions etc. I decided to cut and paste the data into minitab just to check the run charts etc.

To my shock I discovered that the charts showed that the data for most but not all of the dimensions had been cut and pasted a few times so the chart just repeats itself.

Now this type of cheating can be discovered, but it got me thinking if there is anyway of confirming if the data is real if they had used a random number generator as is availble in Minitab.

All trust is now lost in this supplier and I am curently deciding what to do with them.
In three of the parameters Specification lower limit is not mentioned.Then how was CP/PPK derived for these dimensions?
100% inspection is the only solution for this problem.
Call the supplier to your plant and make him carry out 100% inspection.
V.J.Brahmaiah

The capability studies for a PPAP come pretty darn close to the least reliable data one can find - even when attempted honestly. It has no between-lot information (raw material or process), little if any set-up information. Others simply analyze their process completely wrong. It is a lousy tool to try to describe a process with one number. You can't even describe a characteristic with one number - how in the world can you describe the output of a process over time with one number? Although they may not have developed a keen understanding of the absurdity of this value - they may sense its absurdity enough to provide contemptuous data. Does it absolve them? No - but it does not absolve those that request it, either. It puts people in a position to do some very unintelligent things.

The capability studies for a PPAP come pretty darn close to the least reliable data one can find - even when attempted honestly. It has no between-lot information (raw material or process), little if any set-up information. Others simply analyze their process completely wrong. It is a lousy tool to try to describe a process with one number. You can't even describe a characteristic with one number - how in the world can you describe the output of a process over time with one number? Although they may not have developed a keen understanding of the absurdity of this value - they may sense its absurdity enough to provide contemptuous data. Does it absolve them? No - but it does not absolve those that request it, either. It puts people in a position to do some very unintelligent things.
What's your reasonable suggestion for a better solution? It's always easier to tear down an outhouse than it is to build a new one...

What's your reasonable suggestion for a better solution? It's always easier to tear down an outhouse than it is to build a new one...

Not requiring a capability value at all is a reasonable suggestion. My analogy would be not jumping off the cliff is better than jumping with everyone else.

I think that Steve is asking more as to what sort of information (data, datum, whatever) should be requested to prove process capability in lieu of a capability index number?

I agree for the most part that these indices are probably not nearly as telling as we have always been led to believe in the past. Forgive me for my ignorance, statistics has never been my strong point, being self taught and fed only what others would have me know.:mg:

I think that Steve is asking more as to what sort of information (data, datum, whatever) should be requested to prove process capability in lieu of a capability index number?

I agree for the most part that these indices are probably not nearly as telling as we have always been led to believe in the past. Forgive me for my ignorance, statistics has never been my strong point, being self taught and fed only what others would have me know.:mg:

A run chart or SPC chart (if applicable) would be the only anecdotal evidence of what occurred during that run. And, it is not likely to provide indication of future performance.

All trust is now lost in this supplier and I am curently deciding what to do with them.In my opinion, the moment you confirm the data was manipulated/falsified, you should start a process to drop the supplier from your list. Dealing with a supplier that can't be trusted is too much of a risk. They are no longer an asset, but purely a liability. Failure to take swift action increases your risk exposure tremendously.

A run chart or SPC chart (if applicable) would be the only anecdotal evidence of what occurred during that run. And, it is not likely to provide indication of future performance.

Agin, Bob, I am definitely not saying you are wrong, but what, as quality professionals should we be doing? What evidence should be asked for, and what can we do to change the perceptions currently in use? I ask this as someone who wants to learn, not as an argumentative question.:)

Not requiring a capability value at all is a reasonable suggestion. My analogy would be not jumping off the cliff is better than jumping with everyone else.
It sounds like the OP was not just relying on a single capability value.

I fail to see how your analogy applies to this situation. Are you saying that requesting data is the same as suicide? :confused: It's clear that you aren't a fan of preliminary process capability studies. It seems that you're implying that they shouldn't try to characterize the process in any way. The alternatives are to do nothing or to do something better. What do you suggest?

Again, Bob, I am definitely not saying you are wrong, but what, as quality professionals should we be doing? What evidence should be asked for, and what can we do to change the perceptions currently in use? I ask this as someone who wants to learn, not as an argumentative question.:)

I expect to see a run chart - or SPC chart if applicable - as an indication of that particular run - with any adjustments necessary noted. Most data taken during an honest run @ rate would help paint the picture as to the current success of the process.

It would be nice if their was an appraisal of the expected distributions of the variables involved - as the full distribution may not be realized during the PPAP run.

I am tossing these out as off the cuff observations. More deliberate pondering may provide even more ideas. But - bottom line -we need to put on our reality pants on as far as how much predictability such a small amount of data can provide. Without a USB crystal ball, it will not likely be accurately reported.

And I agree with Sidney - if the vendor is neither honest with their data nor comfortable enough with you as a customer to work through data issues (yes, it can very readily be a two-way street!), it is time to part ways.

I fail to see how your analogy applies to this situation. Are you saying that requesting data is the same as suicide? :confused:

Sorry you didn't get it. It is about doing something just because everyone else is. :rolleyes:

Although, committing to high, incorrectly evaluated capability values can be professional suicide...:notme:

It sounds like the OP was not just relying on a single capability value.

I fail to see how your analogy applies to this situation. Are you saying that requesting data is the same as suicide? :confused: It's clear that you aren't a fan of preliminary process capability studies. It seems that you're implying that they shouldn't try to characterize the process in any way. The alternatives are to do nothing or to do something better. What do you suggest?

Part of the problem lies in the fact that OEMs put requirements in place without having any idea whether their suppliers are capable of fulfilling them. This is especially true of SPC and statistics in general. I recall a supplier submitting a PPAP package that included a capability report that had obviously been fudged. He was amazed (and chagrined) to find out that it was possible to tell conclusively (in many cases) that the data he presented wasn't the result of a randomly-operating process.

This is a problem that doesn't lend itself to simple answers, and I don't think it's reasonable to expect a person who recognizes the problem to either offer a solution or be quiet about it.

Part of the problem lies in the fact that OEMs put requirements in place without having any idea whether their suppliers are capable of fulfilling them. This is especially true of SPC and statistics in general.

This has absolutely been true, with the greatest disservice starting back at the introduction of QS9000, where people were forced to use statistics that they had no clue how to use - and do it NOW! It is so pervasive now that it will be equally as difficult to straighten it out.

I know I have been fighting it tooth and nail for over 15 years.....some of the time I have spent learning it better myself.

I expect to see a run chart - or SPC chart if applicable - as an indication of that particular run - with any adjustments necessary noted. Most data taken during an honest run @ rate would help paint the picture as to the current success of the process.

It would be nice if their was an appraisal of the expected distributions of the variables involved - as the full distribution may not be realized during the PPAP run.

I am tossing these out as off the cuff observations. More deliberate pondering may provide even more ideas. But - bottom line -we need to put on our reality pants on as far as how much predictability such a small amount of data can provide. Without a USB crystal ball, it will not likely be accurately reported.

And I agree with Sidney - if the vendor is neither honest with their data nor comfortable enough with you as a customer to work through data issues (yes, it can very readily be a two-way street!), it is time to part ways.

Thanks Bob, these observations were what I was looking for....some of us don't know enough to even know what we don't know. It is always nice to learn some possibilities, and what better place to ask questions than someplace where you know there are folks who have more knowledge.

This has absolutely been true, with the greatest disservice starting back at the introduction of QS9000, where people were forced to use statistics that they had no clue how to use - and do it NOW! It is so pervasive now that it will be equally as difficult to straighten it out.

I know I have been fighting it tooth and nail for over 15 years.....some of the time I have spent learning it better myself.

This situation that Bob describes pretty much sums up all my knowledge of statistics. QS9000. Do it and do it now. Even though our processes were were set to run at the low end of a spec (in order to save those dollars that were a requirement in the lower costs drive) and the statistics were all set to run dead center in the tolerance range. It never made much sense to me that we could produce everything in spec (no complaint for out of tolerance product for size in the 4 years I was there doing QS) yet the indices were all wack because they were offset from center. I have a real fear of statistics, all brought on during these years.

FWIW:
I always found a straightforward approach to be the most efficient.

If it were my supplier, I'd just say, "This chart looks like it was generated by a random number program. Do you have some samples we can examine to see if we can come close to replicating your results? The ideal, of course, would be if you had retained marked samples from your original data measurements."

Four years ago, I wrote (in http://elsmar.com/Forums/showpost.php?p=105469&postcount=18)We had no quality inspectors (we did have quality trainers and guys who acted as "court of last resort" when a question would arise.) Operators did own first article inspections, based on control plan/inspection plan agreed with customer as part of contract review. Another operator would perform a redundant first article inspection with different inspection instruments. Marked sample with BOTH inspection reports was sent to customer for confirmation before production began.

In-process inspection, SPC, etc. was performed by operator in real time. If nonconformance was discovered, production would halt - all operators would collaborate on finding and curing cause, only calling in outside help if solution eluded them. Inspection records, charts, etc. went right to computer where they were available in real time to in-house folk and customers.Our measurements were with instruments hard-wired (today I'd use wireless) directly to the computers where the measurements were entered into the software. They could have been fudged, but fudging was more difficult than just doing it legitimately.

We made a big deal out of having customers approve our production and inspection process as part of their contract review process. If you had done a thorough approval process of the supplier BEFORE granting a contract, you'd have more confidence in the quality data supplied with your products.

Under no circumstance would I fire the supplier without more proof of top management at the supplier instigating and approving data falsification.

Under no circumstance would I fire the supplier without more proof of top management at the supplier instigating and approving data falsification.

Good point, and add that to:

Even though our processes were were set to run at the low end of a spec (in order to save those dollars that were a requirement in the lower costs drive) and the statistics were all set to run dead center in the tolerance range. It never made much sense to me that we could produce everything in spec (no complaint for out of tolerance product for size in the 4 years I was there doing QS) yet the indices were all whack because they were offset from center.

....and you may have a vendor with a perfect process, but their data set does not "generate" an acceptable Cpk (far any number of statistically valid reasons). This - and their weak knowledge of statistics - could have painted them into a desperate corner, in which this data was an attempt to get the PPAP approved.

I can see this as a scenario - as I have witnessed it before.

FWIW:
I always found a straightforward approach to be the most efficient.

If it were my supplier, I'd just say, "This chart looks like it was generated by a random number program. Do you have some samples we can examine to see if we can come close to replicating your results? The ideal, of course, would be if you had retained marked samples from your original data measurements."
If the data had been the output of a "random number program," how would you be able to tell that the data was non-random? The fact is that people who are ignorant of the power of statistical analysis probably aren't even aware of random-number generators, and wouldn't know what to do with one if they were aware. Not only that, but people who are aware can probably fudge data without a random-number generator and present a report that probably won't arouse suspicion.


We made a big deal out of having customers approve our production and inspection process as part of their contract review process. If you had done a thorough approval process of the supplier BEFORE granting a contract, you'd have more confidence in the quality data supplied with your products.
To extend the tautology, if you do a good enough job of qualifying your suppliers, you won't need to have them submit data at all. The sad fact is that there are few people who are fortunate enough to work in a perfect shop with omniscient leadership. Things are a lot more difficult for mere mortals.

Under no circumstance would I fire the supplier without more proof of top management at the supplier instigating and approving data falsification.
"Instigating" usually isn't an overt or direct act; all it takes is for "leadership" to require people to defy the laws of physics in order to meet all of the requirements and then claim ignorance when the inevitable happens.

Another thing about PPAP submissions and data integrity--the PPAP elements should never be viewed in isolation (from one another). If you're suspicious of the data in a capability report, look at the GR&R and dimensional reports for the same dimension(s). Another way to tell that something might not be right is when, for example, the sample mean in the GR&R is significantly different from that in the capability report.

Under no circumstance would I fire the supplier without more proof of top management at the supplier instigating and approving data falsification.And how, do you propose, such proof would be gathered? Do you envision hiring a private investigator? Or calling the supplier top management and asking, point blank: Did you tell your employees to falsify data in my submission package?

Allowing, fostering and/or creating a corporate culture where an employee feels empowered and/or compelled to falsify submission data is as bad as directing a subordinate to do it.

To this date, former top brass of defunct Enron and Arthur Andersen claim ignorance of the wrong doings being perpetrated at lower levels. Do you know what? It does not matter if they instigated it or not. If your governance model and controls are not robust enough to prevent fraud, you, as a supplier is not deserving of doing business with reputable customers.

If you're suspicious of the data in a capability report, look at the GR&R and dimensional reports for the same dimension(s). Another way to tell that something might not be right is when, for example, the sample mean in the GR&R is significantly different from that in the capability report.

I'd be cautious about using this as a flag for something wrong with the GR&R. I've had numerous cases in the past where we performed GR&R studies, but then had to tweak the tool to improve capability, but we do not re-do the GR&R study because the measurement system has not changed.

On other occasions our customers have accepted GR&R studies that cover multiple parts where the design is the same but key dimensions were slightly different size. The measurement system is the same, but the results don't necessarily match up with the capability results.

I'd be cautious about using this as a flag for something wrong with the GR&R. I've had numerous cases in the past where we performed GR&R studies, but then had to tweak the tool to improve capability, but we do not re-do the GR&R study because the measurement system has not changed.

On other occasions our customers have accepted GR&R studies that cover multiple parts where the design is the same but key dimensions were slightly different size. The measurement system is the same, but the results don't necessarily match up with the capability results.

Lack of congruence between PPAP elements is a "flag," as you put it, and it highlights a need for further investigation. No one should jump to conclusions without understanding what's actually going on. It's possible to infer from the dates of the GR&R and capability reports that something happened in the interim that would explain any disparity that might be there, but the inference should be confirmed.

Equally heinous is allowing, fostering and/or creating a supplier relationship where an supplier feels compelled to falsify submission data because your interpretation of data is so faulty you can not accept valid results - albeit not 1.33.

All,

Very interesting varied discussion on the issue.

1. The use of SPC, Run Charts PPAP information is to provide an insight to the capability, stability of the process / processes at the time they where taken. They are an estimate of the performance of a process / system.
2. The use of numerical goals (1.33) or stability rules (1 point outside of 3 sigma) etc is right - as long as you know what you are looking at, understand the data and the processes that make that data. Then you can make an informed decision on any actions needed or not.
3. Therefore the purpose of the data is to enable informed decisions to be made.
4. The main issue with PPAP, SPC, Run Charts, GR&R etc is that the results are not understood by the people who need to understand them either at your supplier, within your own company or at your customer.

5. Falsified data shows a company out of control not just a process.

Reply to my e-mail questioning the data is below.

-Dear Mr. Simon,

We also agree with you that this type of data is not possible in production process.

We analyzed at our end & found that Operators are measuring the parts but not entering the data into the run charts at same time. This is the reason to filled up the wrong data in run charts., because operators are not so much aware about the importance of the run charts & also there in no verification by the supervisor on regular basis.

We have trained our line operators about the importance of the run charts & educate them how important are these run charts & now they are filling the data on daily basis.& our line supervisors also started to verify that run chart data is filled up on daily basis .

We have enclosed the run chart data for your analysis for the month of Sep 09.

Thanks & Regards

I'll add my two cents. If data is inaccurate due to ignorance and not understanding how to do something right, then the motive may not be suspect and there is room to correct and improve.

If however, top management promotes a culture to willfully decieve and falsify data, just to get and keep business, that is MUCH more difficult to correct and will likely lead to continuing problems. This is one of the big issues with dealing with some overseas suppliers and is rarely calculated into the "savings analysis" that accountants do when they are discussing outsourcing. It is why much of the overseas outsourcing does not generate the "savings" they were expected to generate.