Fellow Covers

What I am looking for are some different points of view on a topic close to my heart at the moment. Assume a customer requirement of capability 1.67 min. Also assume you have a customer who is just a “tick the box” guy and is not an engineer nor has any understanding of processes or even the product, but wants you to wrap the parts in pound notes for no cost.

The part is produced by a multistage process, some of which can be controlled by SPC e.g. machining, forging etc. using consumable tools and others are “blanket” processes such as heat treatment, barrelling etc. which combine sub-batches from previous process into a much larger quantity. A simplification I admit, but assume the second part of the process does not affect the dimension reported.

Here is the question. Should the 1.67 capability apply to the SPC controlled process, or to the overall population generated by the complete overall process?

Before answering that, consider the following situation –

Producing a part dimensioned 1.15-1.65 which is set up by a consumable tool which is only adjustable in significantly large steps but yields a very capable process Pp 5 plus.

You produce 5800 parts at: mean 1.255 (Ppk 2.469).
You produce 5800 parts at: mean 1.555 (Ppk 2.234).

Both batches are acceptable at above the 1.67 level and are functional at these sizes.

Stage two Heat Treat and Barrel Clean etc in batches of 11600 where the two batches are thoroughly mixed because of economy, capacity and process control at this stage.

Now the sample at final product stage, you get Mean 1.405 but a Ppk at 0.5397. They are still the same parts and they are still functional.

Comments please
:argue:

That is a challenging situation. The basic capability indices weren't designed for bimodal (or trimodal...) distributions like this. The calculations assume the process is stable, but here you have a very clear "special cause" of variation - the different sub-batches that are combined before the final process.

You probably have the simplest solution, and one that lives up to the spirit of the capability calculations. If you report the two separate capability results, you are giving an accurate representation of the ability of the parts to meet the customer requirements.

This is a case where you hope for a customer who is either very ignorant or very knowledgeable about capability. If they are very ignorant, they won't recognize that there might be a problem with the way you are doing things. If they are very knowledgeable, they will be able to understand the
nuances of what you are doing. It is the person with a moderate understanding that you would have to look out for.

Now this assumes that the final steps really DON'T have an affect on these dimensions. I expect you have adequately confirmed that.

A few other random brainstorms.
is there any simple way to mark the parts so that you know which sub-batch they came from?
if you produce many sub-batches and have them sitting around waiting for the final processing, you could sort these by size. Then when you go to the final processing, always try to match sub-batches with similar sizes. The Ppk still won't be as good as the individual sub-batches, but it will be much closer.
if the different populations really are that different, after the final processing, draw a slightly larger sample than you need and measure the parts. From the results, you can sort these parts back into the correct original sub-batches. Then find Ppk for each sub-batch after this final processing.Ultimately, I think the best option is just reporting the technically incorrect value (the average of the different sub-batches). This is a better indicator of the effective capability of the product than the value calculated after mixing the various sub-batches.


This kind of goes along with the quote I am using at the moment. If your customer hasn't grasped the obvious about the meaning and intent of capability, then these sorts of subtleties won't help them (or hurt them, for that matter).


Tim F

I just remembered a good article in Quality Progress that addressed this same general topic. It is written in a story format and is an easy read. In this case, they were mixing parts molded in different cavities, but the idea is the same.

It was from May of 2003. If you are a ASQ member, you can see it here:
http://www.asq.org/pub/qualityprogress/past/0503/qp0503pylipow.pdf
Since it is copyrighted, I can't post a copy here. :(


This might be something to show your customer if you need to explain your calculations. It always helps to cite an article published in an "official" source. :)


Tim F

Capability analysis NOT a batch index, it is a process index. So, if those two lots were to never combined, but rather just control charted, you would have an overall mean that is 1.405 with a Pp based on S/c4. The process is not capable of consistently meeting the customer requirement. On a lot by lot basis it might, but over all it cannot.

The process is not capable of consistently meeting the customer requirement. On a lot by lot basis it might, but over all it cannot.

not necessarily. we don't know the standard deviations of each lot. when the variaiton from lot to lot is larger than within lot we will have an overinflated estimate of the total standard deviation when calculating it from the individual values and it will be underinflated if calculated usign only the within lot standard deviations.

The dilemma we have with lot to lot variabilty beign greater than within lot is that the overall standard deviation has very heavy tails. It is NOT Normally distributed and therefore the intent of the Ppk value is not met. this may be the case here. (As always without the individual data it is difficult to assess the OP's situation) He did provide the averages and Ppks but it's the standard deviation that kills you...

If he does have this heavy tail situation he may have good capability (no defects and not 'close' to the spec limits) but the traditional calculations can't take him there because they don't apply

The dilemma we have with lot to lot variabilty beign greater than within lot is that the overall standard deviation has very heavy tails. It is NOT Normally distributed and therefore the intent of the Ppk value is not met. this may be the case here. (As always without the individual data it is difficult to assess the OP's situation) He did provide the averages and Ppks but it's the standard deviation that kills you...

If he does have this heavy tail situation he may have good capability (no defects and not 'close' to the spec limits) but the traditional calculations can't take him there because they don't apply

Exactly my problem in a nutshell, actually the standard deviation for the process is circa 0.014 to yield Ppks at 2.469 and 2.234 respectiveley but at well spread means. On machining etc I always try to center the process, but cold forging is not so easily targeted it just goes in steps. If the two batches were not mixed for economic final processes, there would be no reason for none acceptance.

My customer accepts that the parts are "good" and "functional" but can't get his head round an apparently "poor" Ppk (he needs to tick the box). I could always take away the problem by processing in small batches but there is no technical benifit for the cost incurred.

I wondered how others got arround this issue.

What I have done in the past - depending on the sophistication of the Custoemr and the intent on requiring Ppk values - is to plot the individual values for each individual lot against the spec limits so that the custoemr can see the within lot and lot to lot variation (graphical method of choice is the multi-vari)

then I either determine the best fit distribution (most commonly it is close to Uniform) and calculate the Ppk value given the distribution statistics, OR I just use the categorical defect rate and back into the Ppk value from the Z table, OR I have been known to calculate the upper and lower Ppk values using only the lots that are closest to the lower and upper spec limit respectively. the latter is hokey but does take into account incursion on the spec limits by the tails of the extreme lots.

If the Ppk is just to predict defect rates, then the second approach typically works without question (if questioned the multi-vari provides the backup rationale)

If the Ppk is there to ensure reduced variability within the tolerances the multi-vari chart coupled with some form of statistical calculation (the first or third method) convinces my customers that the distribution WIDTH is narrow enough for their purposes within the tolerance range. If all else fails, I elevate within the organization to a knowledgable statistician, supplier engineer, or design engineer to gain understanding of intent and statistical reality.

(By the way as I've said many times in this forum, this is only one reason why I belive that capablity indices are an obomination on the face of the earth and shoudl be abolihsed - but that doesn't help you with your customer...)

not necessarily. we don't know the standard deviations of each lot. when the variaiton from lot to lot is larger than within lot we will have an overinflated estimate of the total standard deviation when calculating it from the individual values and it will be underinflated if calculated usign only the within lot standard deviations.

Based on the data provided, we can estimate the overall S used to calculate the Ppk based on the specifications. When we combine the data from the two lots, we the overall S and specification does not change, just the mean value. When the mean changed, and the S and specification does not change, then the fact is the overall process is not capable.


You produce 5800 parts at: mean 1.255 (Ppk 2.469).
You produce 5800 parts at: mean 1.555 (Ppk 2.234).

Based on the data provided, we can estimate the overall S used to calculate the Ppk based on the specifications. When we combine the data from the two lots, we the overall S and specification does not change, just the mean value. When the mean changed, and the S and specification does not change, then the fact is the overall process is not capable.


You produce 5800 parts at: mean 1.255 (Ppk 2.469).
You produce 5800 parts at: mean 1.555 (Ppk 2.234).

maybe, maybe not. The OP is indicating that he is making all conforming parts (The customer agrees that the parts are "good" and "functional")

The capability of the entire process cannot be assessed (note assessed does not equal the mathematical calculation of Ppk...)

If the OP would post his raw data we could assess this and provide more accurate answers...

Fellow Covers

What I am looking for are some different points of view on a topic close to my heart at the moment. Assume a customer requirement of capability 1.67 min. Also assume you have a customer who is just a “tick the box” guy and is not an engineer nor has any understanding of processes or even the product, but wants you to wrap the parts in pound notes for no cost.

The part is produced by a multistage process, some of which can be controlled by SPC e.g. machining, forging etc. using consumable tools and others are “blanket” processes such as heat treatment, barrelling etc. which combine sub-batches from previous process into a much larger quantity. A simplification I admit, but assume the second part of the process does not affect the dimension reported.

Here is the question. Should the 1.67 capability apply to the SPC controlled process, or to the overall population generated by the complete overall process?

Before answering that, consider the following situation –

Producing a part dimensioned 1.15-1.65 which is set up by a consumable tool which is only adjustable in significantly large steps but yields a very capable process Pp 5 plus.

You produce 5800 parts at: mean 1.255 (Ppk 2.469).
You produce 5800 parts at: mean 1.555 (Ppk 2.234).

Both batches are acceptable at above the 1.67 level and are functional at these sizes.

Stage two Heat Treat and Barrel Clean etc in batches of 11600 where the two batches are thoroughly mixed because of economy, capacity and process control at this stage.

Now the sample at final product stage, you get Mean 1.405 but a Ppk at 0.5397. They are still the same parts and they are still functional.

Comments please
:argue:
Michael,

If your assumption that the last steps of your process have no effect on the dimensions of your product is correct, here is what I think is happening to you and what you should do about it.

Let the lot with the mean 1.255 and Ppk 2.469 = Lot 1
Let the lot with the mean 1.555 and Ppk 2.234 = Lot 2
Let the combined lot (after treatment) mean 1.405 and Ppk 0.5397 = Lot 3

Assuming you are using the capability analysis for a normal distribution,

Sigma Lot 1 = 0.014176
Sigma Lot 2 = 0.015667
Sigma Lot 3 = 0.472485

Using the equation

Sigma avg = SQRT((sigma 1^2 + sigma 2^2)/2), the average sigma for Lot and Lot 2 is 0.010564.

Deriving the distance between mean 1 and mean 2 in terms of sigma

Sigma distance = (mean 2 – mean 1)/sigma average, the sigma distance is 28.398.

Conclusion, you have two separate peaks that only join at the 14.2 sigma points, essentially they never touch.

The capability analysis cannot operate with the above distribution even using the non-normal process and is not providing you with valid answers for the combined lot.

Some suggestions for correcting this problem:

1. You need to prove your assumption that the final steps of the process do not affect the dimensions of your parts. In my opinion there are 2 ways to do this:

a. Preferred. As you sample the lot for SPC purposes, set aside the parts you have sampled from that lot. Now you have a group of parts with a known mean and standard deviation.

i. Perform the final steps of the process using only these previously measured parts. CAUTION: Make sure the same measurement system and same operators that performed the measurements on the before treatment parts perform the measurements of the after treatment parts to minimize the effects of gage R&R for your measurement system.

ii. Run an F test on your results to prove there is no statistical difference between the variation of the parts prior to and after treatment.

iii. Run a 2 sample T test on your results to prove there is no statistical difference between the means of the parts prior to and after treatment. DO NOT use a paired T test because you will have no assurance that you are testing the same part for before and after treatment.

If your F test and T test show there is no difference in the parts prior to and after treatment, you can report the Ppk for each lot as the Ppk for the process rather than the Ppk of the final acceptance inspection.

b. Workable but less preferred. Run the same process as above but rather than using the parts that were inspected before and after the treatment, use the entire lot before and after. This would work but would not give me as much confidence in the results because you will be using different parts for the comparison, therefore, in my opinion the comparison is less valid. I know the parts came from the same source (population), but using the same measured parts on both sides would make me more comfortable.

2. A second way to solve this is to wait until you have produced 4 lots and run the parts from similar lots through the final treatment process. But remember, with your sigma, the means are going to have to be nearly exactly the same or your Ppk answers will still be a problem.

3. A final suggestion would be to run the comparison of the samples parts prior to and after treatment 5 to 10 times to establish a solid foundation that the final steps do not affect the part dimensions. Then eliminate the final inspection.

I hope this helps.

Jim Shelor
PMP, SSBB

Thank you all for the help and advice,:thanx:

Jim's analysis of the current situation fits the data I have.

I had previously "proved it" to myself that the final operations were not significantly effecting capability and had done the analysis more or less in terms of Jim's B option.

I will now go away and do it in accordance with option A for the improved statistical base.

As I said I am fairly comfortable with the parts and process but I have a SQE who does not understand metal processing and why you can't always set up identically to the last batch.

I'm not convinced he understands statistics and his only aim is to get a final capability level his companies book says he can pass at a minimum cost to them.

maybe, maybe not. The OP is indicating that he is making all conforming parts (The customer agrees that the parts are "good" and "functional")

The capability of the entire process cannot be assessed (note assessed does not equal the mathematical calculation of Ppk...)

If the OP would post his raw data we could assess this and provide more accurate answers...
Maybe the sample size is not large enough to catch out of spec product.

Maybe the sample size is not large enough to catch out of spec product.

For the OP: I wish you good luck in trying to get out of what looks like an arbitrary Ppk target...however I do want to make a technical point in case you or others get stuck in the wringer on this one. The batch operation that 'combines' smaller lots together, resulting in the 'combined' sampling is NOT the issue. (regardless of whether or not the batch process changes the characteristic.) Even if you had no final batch process, you would still have to combine the lots to calculate the capability of the PROCESS. The process is all of the lots not just a single lot. Even when using the smallest sample size possible - 30 representative samples - the samples must be randomly drawn from the process output, not randomly drawn from one lot. And you cannot calculate the Ppk of each individual lot and simply average each Ppk. That is not correct, especially in the presence of significant lot to lot variation.

For Statistical Steven (and the OP): the case that the OP is discussing is fairly common in many industries: the largest component of variation in the process is not piece to piece it's lot to lot (or set-up to set-up)
I have an example attached that is from my own experience.

For the OP: I wish you good luck in trying to get out of what looks like an arbitrary Ppk target...however I do want to make a technical point in case you or others get stuck in the wringer on this one. The batch operation that 'combines' smaller lots together, resulting in the 'combined' sampling is NOT the issue. (regardless of whether or not the batch process changes the characteristic.) Even if you had no final batch process, you would still have to combine the lots to calculate the capability of the PROCESS. The process is all of the lots not just a single lot. Even when using the smallest sample size possible - 30 representative samples - the samples must be randomly drawn from the process output, not randomly drawn from one lot. And you cannot calculate the Ppk of each individual lot and simply average each Ppk. That is not correct, especially in the presence of significant lot to lot variation.

For Statistical Steven (and the OP): the case that the OP is discussing is fairly common in many industries: the largest component of variation in the process is not piece to piece it's lot to lot (or set-up to set-up)
I have an example attached that is from my own experience.
Bev,

I know that speaking strictly statistically, the point you are making fits the rules. But I believe there are times when we have to look past the pure statistics to the realities of the process.

This process is producing parts that meet the customer's specifications and has a defect rate of < 0.002 dpm (less than 2 defects per billion).

I don't think I am willing to go to my boss and tell him/her that we need to spend money improving a process with < 0.002 dpm error rate, regardless of what the statistics say.

We know what is causing the mean shift in the lots, we have evaluated the shift and accepted it. Just because we mix the parts together and now the result looks bad does not mean the process is broken. It means we have a special cause variation, the consumable tool, we understand the effect of it, and we know we are actually very consistently making good parts for the customer.

Jim Shelor
PMP, SSBB

Jim - I get your point and I think that's what I said in the attachment

Jim - I get your point and I think that's what I said in the attachment
Bev,

Yes that is exactly what you said in your attachment.

I skimmed your attachment prior to replying, which is always a bad thing.

I obviously need to be more careful about my research prior to posting a reply.

Have a great day,

Jim Shelor
PMP, SSBB

That is a challenging situation. The basic capability indices weren't designed for bimodal (or trimodal...) distributions like this. The calculations assume the process is stable, but here you have a very clear "special cause" of variation - the different sub-batches that are combined before the final process.

You probably have the simplest solution, and one that lives up to the spirit of the capability calculations. If you report the two separate capability results, you are giving an accurate representation of the ability of the parts to meet the customer requirements.

This is a case where you hope for a customer who is either very ignorant or very knowledgeable about capability. If they are very ignorant, they won't recognize that there might be a problem with the way you are doing things. If they are very knowledgeable, they will be able to understand the
nuances of what you are doing. It is the person with a moderate understanding that you would have to look out for.

Now this assumes that the final steps really DON'T have an affect on these dimensions. I expect you have adequately confirmed that.

A few other random brainstorms.
is there any simple way to mark the parts so that you know which sub-batch they came from?
if you produce many sub-batches and have them sitting around waiting for the final processing, you could sort these by size. Then when you go to the final processing, always try to match sub-batches with similar sizes. The Ppk still won't be as good as the individual sub-batches, but it will be much closer.
if the different populations really are that different, after the final processing, draw a slightly larger sample than you need and measure the parts. From the results, you can sort these parts back into the correct original sub-batches. Then find Ppk for each sub-batch after this final processing.Ultimately, I think the best option is just reporting the technically incorrect value (the average of the different sub-batches). This is a better indicator of the effective capability of the product than the value calculated after mixing the various sub-batches.


This kind of goes along with the quote I am using at the moment. If your customer hasn't grasped the obvious about the meaning and intent of capability, then these sorts of subtleties won't help them (or hurt them, for that matter).


Tim F

Tim is correct, AIAG PPAP 4th Edition section 2.2.11.5 Processes with One-Sided Specifications or Non-Normal Distributions states that "The above mentioned acceptance criteria (2.2.11.3) assume normality and a two-sided specification (target in the center). When this is not true, using this analysis may result in unreliable information.
My favorite quote, by the way.

Anyway, first I would specify that changes in lots are special causes. They do not affect every part in the same manner. So, treating each lot individually, showing that is generates capable data, should be adequate. Tim's suggestion to follow specific parts through the subsequent operations and review their capability afterwards is an excellent idea. I have done that with plating and heat treat operations, because they likely will affect the final distribution. Reporting the final distribution of that lot is the most useful.

If the customer insists on a capability of all incoming product to fall within 1.33 or so, then I hope that was clearly stated in their purchase order. With that, you have a total variance issue that will compress the allowed variability of each of the processes to generate capability of much higher than 1.33 for each. And, as has been noted, that is very expensive. Probably unduly expensive for the end use. The cost of "tick in the box" thinking. Not very effective.