Selecting factors for screening DoE design

Jafri

Involved In Discussions
We did screening run to find significant factors in a newly developed process. The factors were selected using a fishbone analysis. But the results indicate that the tram might have missed to include all factors for screening run.
I am wondering if there is a better way to select factors to vary in screening design, than a guess-work technique like fishbone analysis?

optomist1

A Sea of Statistics
Trusted Information Resource
Can you share your data? What exactly was performed re: screening...Fishbone??

Jafri

Involved In Discussions
I won't be able to share the fishbone. But basically this fishbone analysis was looking at all possible causes ("factors") that might influence the response variable. And then a couple of people decided that 7 out of those 20+ factors should be used in DoE, based on their experience.
But this all a totally new process. So there is a chance the team might have missed to include some potentially significant factor in DoE. So I was thinking if there is any other way to come up with factors to include in a DoE.

One book on DoE suggested, from a Shainin inspired professional, suggested Multi-Vari analysis. But I don't understand how it can help. Because Multi Vari would require a constant and stable process. Yet here, we want to develop the process itself.

Bev D

Heretical Statistician
Staff member
Super Moderator
I was going to describe how we use multi-vari to do exactly what you want. First let me say that multi-vari doesn’t require a constant and stable process - at all.

Here’s what we do: we create an inputoutput matrix that describes the known input factors for each step and the known output factors. These factors are specific and measurable. (The fishbone has an unnatural sequencing of factors that don’t match the actual physaicla sequence of factors that must come together to create an output.). We consider all factors: materials, process parameters and conditions/protocols.

THEN, we perform a multi-var on the OUTPUT Characteristic(s) of interest. At this point we call it a “components of variation” study. We make 3 of everything. 3 lots from 3 lots of materials with 3 operators. We look at within piece (if applicable), piece to piece, lot to lot, operator to operator, time to time etc. teh largest component of variation contains the critical factors (under specified operating conditions) and this gives us our first level screen to narrow down an unmanageable numbe rof factors to a manageable level. Often not requiring more than a full factorial experiment....

Jafri

Involved In Discussions
That's great. Honestly, I don't get it at this time.
But will think it through. And may have to ask a few questions for clarification.

Many thanks.

John Predmore

Quite Involved in Discussions
I am not a big fan of brainstorming. Brainstorming with a fishbone diagram spends much time fleshing out ever-tinier twigs of the tree using a comprehensive but somewhat arbitrary framework to organize the universe (Man-Machine-Method-Material-etc.). Brainstorming with a fishbone diagram is a divergent thought process, which gives no guidance into which branch might ever lead to a productive conclusion.

The advantage of Multivari is a small analytical study guides the investigator with real world data to “prune” the tree of all possibilities, and focus the team on families of variation which matter most. Once you understand behavior of the factor(s) you are looking for, you are at a better starting point to identify for testing factors, conditions and interactions which might be at work.

I found several articles online which mention or show examples of a Multivari chart. I suggest you start with the description of Multivari in MULTIVARIATE TOOLS .

Jafri

Involved In Discussions
We consider all factors: materials, process parameters and conditions/protocols.

THEN, we perform a multi-var on the OUTPUT Characteristic(s) of interest. At this point we call it a “components of variation” study. We make 3 of everything. 3 lots from 3 lots of materials with 3 operators. We look at within piece (if applicable), piece to piece, lot to lot, operator to operator, time to time etc. teh largest component of variation contains the critical factors (under specified operating conditions) and this gives us our first level screen to narrow down an unmanageable numbe rof factors to a manageable level. Often not requiring more than a full factorial experiment....
So we are selecting two levels of each known factor, and taking 3 repeats for each combination of factors.
It looks like Multi-Vari is a full-factorial DoE design. But that will be more expensive to run than a fractional factorial screening design.

If it really is full-factorial DoE then the problem remains. We are to select KNOWN factors, but we don't know if there are any UNKNOWN factors which may be significant.
What will the Multi-Vari chart show if we failed to include some unknown factors?

John Predmore

Quite Involved in Discussions
It looks like Multi-Vari is a full-factorial DoE design
There are similarities between the Multivari graphs and graphical Analysis of Means used in DOE. But the important contrast here is Multivari characterizes the behavior of the output factor, not the input factors, per se.

The article I referenced above talks about positional, cyclical and temporal characteristics of a variable. Bev used the term components of variation. I prefer the term Multivari families of variation. These different terminologies refer to the same idea. We gain insights about causal factors by studying the “behavior” of the response variable output.

The second figure in the article shows how thickness of an injected molded part might be measured at four equally-spaced points on one edge, numbered 1,2,3,4 (an edge which assembles to a mating part, let’s imagine). Note the spec limit is 0.100 to 0.110”.

The third figure shows thickness values organized by the 23 sample numbers. But the values are in the range of 0.020 to 0.060”, so these are not thickness measurements from the picture above; maybe these values are differences from nominal – I don’t know. I see there is a fifth point on every vertical line segment, most likely a median point or an average. (I did not make the chart; I did not see the data values.) From this third chart, I recognize some variation between samples (the average thickness of #14 is higher than all the others) and within samples (as depicted by uneven spacing of points on the line segments). In this presentation of the data, the variation looks random. But randomness is more a statement of our ignorance rather than any usable information about the nature of the problem. The data are not organized in a way that provides clues about the sources of variation.

The fourth figure shows thin vertical rectangles with corners numbered 1,2,3,4. [I observe there are 16 line segments in this chart compared to 23 in the previous chart, so I conclude these are not the same data. I did not make the chart. The explanation would make more sense if the same data where presented in the Multivari format, but my eye tells me that is not the case. I hope you can see past what I consider a flaw in the writing of the article. ]

The fourth figure does not number the corners on all the other vertical rectangles, but the method of Multivari employs the same organizing scheme across all the vertical rectangles, which in this example are four parts molded at four different hours of the same day. Presumably, everything else is basically the same across the four hours of the same day - the same mold, the same process settings, the same batch of plastic. Because of the way the data are organized in this plot, Multivari shows every part is thicker in the center than on outside edges. All measurements are within USL and LSL, but there is a strong non-random pattern, and the magnitude of this pattern – center thicker than the edges - is greater than any other Multivari family apparent in this chart. If I was trying to solve an assembly problem, this discovery could be a valuable clue. It does not identify the cause, it does not tell me what to fix, but it gives me valuable insight where to look for more answers. (The next illustration in the article gives a different scenario, and the Multivari in that scenario gives a completely different clue, which would lead me to different hypotheses to test.)

In this presentation of the data, the biggest source of variation is in the family of within-piece. Other families of variation depicted here are piece-to-piece, depicted by four vertical lines at each hour, and hour-to-hour, depicted by four sectors. If this injection molded part came from a multi-cavity tool, then cavity-to-cavity variation would be another common family of variation to include in the study. I see two line segments noticeably lower than the rest. It might be worthwhile to look for clues as to what those two pieces have in common. If you choose your Multivari families carefully, the cascading families encompass the entire practical universe of spatial, cyclical and temporal factors. That is your assurance that no unknown factor was overlooked. If an unknown factor fits into one family and its impact is dominant, its impact should be manifest in a Multivari analysis.

This information about which family of variation is dominant is valuable when planning your screening DOE. If you knew the factor with the biggest impact can be found within the within-piece family, it isn’t necessary to include in your DOE multiple cavities, machines, operators or days. This is the reason why Bev and I suggested Multivari is an alternative to brainstorming a fishbone diagram. If you use historical data, the "experiment" is practically free.
JP

Last edited:

Bev D

Heretical Statistician
Staff member
Super Moderator
what John said.
Plus Jafri, I think it would be helpful for you to study what a multi-vari is. It is not a DoE. You do not select factors or "set levels". The set-up is different for existing processes and products than it is for a new process or product. With a new process or product we must make material by specifically acquiring and using 3 different lots of raw materials and/or components to build 3 lots of product. the 3 lots should be built over 3 different time periods using different operators. Don't get all concerned yet about conditions and settings and parameters that you can set in manufacturing, just vary them a bit around what you think is a good target. in fact using 3 different operators will probably get you all the variation you need. collect the data on 3 sequential pieces per time period. Each time period should be what makes sense for the process. a typical tie mperid structure is start and finish and every 2 hours in between the start and finish OR whenever a change to process occurs. these changes could be inherent to the process such as refilling something or adjusting a tool or process setting. It could also be 'unplanned' such as after a machine stoppage/jam. THEN plot the data in time series, looking at within piece(if it applies. 3-5 locations within the piece or what ever makes sense), piece to piece (usually 3 sequential pieces per time period for all 3 lots.

The component (or family as John prefers) of variation that contains the largest change, contains the factors that contribute the most variation. All factors effect only 1 component/family of variation. you will then have to unconfound that family with further experimentation but this typically happens with any other screening experiment. The beauty here is that there is no brainstorming or guessing. you are dealing with all factors, known and unknown.

also remember that any screening design including fractional factorials will detect no significant factors if the levels are set too tightly....if we over constrict the variation of inputs we have every expectation that the output will show little variation.

Informational How To Avoid Compliance & Timeline Risks When Selecting A Medical Device Supplier Medical Device and FDA Regulations and Standards News 0
ISO 13485 for software company - Selecting suppliers ISO 13485:2016 - Medical Device Quality Management Systems 3
Selecting potential internal auditors Internal Auditing 3
Selecting System Precision Criteria - System Suitability Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
Advice on selecting an ISO 9001:2015 Lead Auditor Certification Course ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 22
B Selecting Parts for an MSA (Measurement System Analysis) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 7
S Selecting materials for implants to comply with ISO 10993 biocompatibility Other Medical Device Related Standards 4
B Process Control - Selecting Parts - NDC & Study Variation Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
B Selecting components to meet UL 60601-1 IEC 60601 - Medical Electrical Equipment Safety Standards Series 7
B Risk Management while selecting the Supplier Document Control Systems, Procedures, Forms and Templates 3
Selecting which MDD 93/42/EEC Annex to follow - PACS software system CE Marking (Conformité Européene) / CB Scheme 2
5 keys to selecting the right CM (Contract Manufacturer) ISO 13485:2016 - Medical Device Quality Management Systems 1
M Pre-Selecting Materials for ISO 10993-1 Biocompatibility Compliance Other ISO and International Standards and European Regulations 5
P Selecting a Registrar and completing our ISO 9001 Certification ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 15
Selecting Wires for Measuring P.D. of Thread Gages General Measurement Device and Calibration Topics 9
M Suggestions for Selecting RF Wireless Components Other US Medical Device Regulations 3
S Selecting weights for daily Calibration Check - Using just one weight? General Measurement Device and Calibration Topics 10
Selecting a Registration Agent in China China Medical Device Regulations 17
Selecting Consultants for getting drug approval in USFDA Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 1
G Gage R&R Procedures: Selecting parts (sample) to measure Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
W Selecting Non-Conformities to address Nonconformance and Corrective Action 16
Selecting the Right Problem (book chapter 1) The Reading Room 8
G Gage R&R: Selecting Number of Trials and Appraisers and Parts Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
Selecting a Sample and Determining the Reference Value for a Bias Study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
Selecting a predicate device for FDA 510(k) approval Other US Medical Device Regulations 14
W Question about sample size code letter and selecting sampling plan Inspection, Prints (Drawings), Testing, Sampling and Related Topics 7
D Selecting a Tolerance when multiple specifications exist - Different Beers Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
Optimally Selecting 30 Pieces of Product out of 34 - How do I? Statistical Analysis Tools, Techniques and SPC 9
J Selecting the best Gage - Buy according to production criteria IATF 16949 - Automotive Quality Systems Standard 0
A Is there any kind of Ranking available for selecting the Registrars? Registrars and Notified Bodies 31
L I need a reference for selecting a notified body. Can anyone make a recommendation? ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 21
L Selecting ISO 9001:2000 Internal Auditors Internal Auditing 12
Q Criteria for Selecting a Consultant or Consulting Firm Consultants and Consulting 2
B Selecting Six Sigma Black Belts Six Sigma 3
S QOS for Tooling and Equipment - Having difficulty selecting a measurable QS-9000 - American Automotive Manufacturers Standard 2
What to do with correction factors overlooked by an accredited standard (6.4.11)? ISO 17025 related Discussions 5
Human Factors / Usability validation in the time of COVID Human Factors and Ergonomics in Engineering 9
Kaizen Events - Factors Affecting Failure Lean in Manufacturing and Service Industries 13
Is Human Factors testing mandatory for a 510(k) submission? Human Factors and Ergonomics in Engineering 16
Determination of TENSION SAFETY FACTORS - Table 21 IEC 60601-1 Other Medical Device Regulations World-Wide 5
DoE (Design of Experiments) - Multiple responses with different factors Using Minitab Software 2
Usability/human factors engineering requirement (standard) for IVD medical device Other Medical Device Related Standards 2
Interesting Discussion Human Factors as Root Cause AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 13
Global medical device human factors/usability requirement IEC 62366 - Medical Device Usability Engineering 3
Medical Device News FDA News - 14-09-18 - Benefit-Risk Factors to Consider for Substantial Equivalence Other US Medical Device Regulations 0
Minitab 15 - Factorial Design - 3 factors: 4x3x2 - How to? Using Minitab Software 4
Interesting Discussion Addressing Human Factors in Corrective Action AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 23
I have 3 different factors - DOE help Statistical Analysis Tools, Techniques and SPC 3
How do I create a 3^k factorial design with factors be treated as continuous where I Using Minitab Software 0
Screening DOE with 7 Input factors and 4 responses - Significant factors Using Minitab Software 1