For screening a design, can we live with the low/poor predicted R-Sq.?

v9991

Trusted Information Resource
#1
during analysis of the results, Design Expert provides three Regression coefficient for a model,
R-Squared
Adj R-Squared
Pred R-Squared

A. the query is, can we ignore/live-with poor or low Pred R-Sq.
1) during screening designs(Res-II or Placket Burman Design),
2) there is an analysis of DOE through option for historical data ,

in both cases, we are essentially looking for not a model, but JUST co-efficient and their magnitude (apart from the main effects)

B. for exploratory work, can we do work with p-value on marginally higher side (approx 0.06-0.1 vs 0.05 )
 
Elsmar Forum Sponsor

Miner

Forum Moderator
Staff member
Admin
#2
A screening design is just that, a means of filtering a few significant factors from the many non-significant factors. It is not used for building a mathematical model. At such low resolution, it would miss all interactions anyway.

Since it is used for filtering only, I often recommend using an alpha of 0.1. The assumption is that once you have narrowed the field to the few factors of interest, you will follow up with a series of modeling or optimization experiments at a lower alpha that would confirm or dispel these borderline factors.
 
Last edited:

v9991

Trusted Information Resource
#3
A screening design is just that, a means of filtering a few significant factors from the many non-significant factors. It is not used for building a mathematical model. At such low resolution, it would miss all interactions anyway.

Since it is used for filtering only, I often recommend using an alpha of 0.1. The assumption is that once you have narrowed the filed to the few factors of interest, you will follow up with a series of modeling or optimization experiments at a lower alpha that would confirm or dispel these borderline factors.
yes, that is the intent,
in the resulting model, we are trying to draw some inference from the co-efficients of the factors.
+ can you also pl comment on the predicted R-Sq.!
 

Miner

Forum Moderator
Staff member
Admin
#4
I would not use the results of a screening design for anything more than determining the significance of a factor and directionality (i.e., +/-). To interpret coefficients, you must have a mathematical model. To have a mathematical model, you must include any significant interactions (which you cannot test with a low resolution screening design). Omitting these interactions will change the coefficients of the factors.

BTW. Please be precise on your meaning behind coefficients. The most common meaning is Bx where Y = B0 + B1X1 + B2X2... Bx is a coefficient. R^2 is also less commonly called the coefficient of determination. I have been responding based on the assumption that you meant Bx when you asked about coefficients, but am no longer certain.

Regarding R^2:
  • R^2 explains the percentage of variation explained by the mathematical model, but has one serious drawback. If you continue to add terms to the model, even if they are non-significant, R^2 will continue to increase.
  • R^2 adjusted compensates for the flaws of R^2 by penalizing you for every term in the model. If the term adds value to the model, R^2 adjusted will increase, but if the term adds no value, R^2 adjusted will decrease. The ideal model will have similar R^2 and R^2 adjusted values.
  • R^2 predicted goes even further by sequentially removing one observation from the data set, calculating the regression coefficients and predicting the missing datum. If you have over fit the model, R^2 predicted will drop. If you try to fit a linear model to a nonlinear data set, or are missing interaction, R^2 predicted will also drop.
 
Thread starter Similar threads Forum Replies Date
J Selecting factors for screening DoE design Statistical Analysis Tools, Techniques and SPC 8
W Definitive screening design in Minitab 17 Manufacturing and Related Processes 1
F Resolution III Factorial Design for a Screening Experiment Six Sigma 2
T Screening DOE (Design of Experiments) with qualitative response Quality Tools, Improvement and Analysis 23
V Screening DOE with 7 Input factors and 4 responses - Significant factors Using Minitab Software 1
S Rapid screening methods for antibiotic residues in Chicken Meat Food Safety - ISO 22000, HACCP (21 CFR 120) 1
M ITAR Requirements and Visitor Screening AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 15
P Screening DOE: A coming up headache story Quality Tools, Improvement and Analysis 4
S Screening and Screening Rational Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
C Environmental Stress Screening (ESS) - Difference between MIL-HDBK-344 & 2164A Reliability Analysis - Predictions, Testing and Standards 4
I Screening: Blackett Burman versus Fraction Factorial Quality Tools, Improvement and Analysis 2
Stijloor U.S. Intensifies Screening for Travelers From 14 Nations Travel - Hotels, Motels, Planes and Trains 1
E Environmental Screening of PCBAs in High Volume Electronics Contract Manufacturing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
Sean Kelley Safety audit violations - bloodborne pathogens, LOTO & drug screening Occupational Health & Safety Management Standards 12
A 510(k) Screening Checklist that the RA has filled out - Is that sheet for FDA use? ISO 13485:2016 - Medical Device Quality Management Systems 2
G Health Insurance - Annual health screening Career and Occupation Discussions 10
R DFA & DFM - Examples for Design for assembly and design for manufacturability Lean in Manufacturing and Service Industries 2
D Using Laboratory Notebooks in R&D and Design and Development ISO 13485:2016 - Medical Device Quality Management Systems 3
D ISO 13485 - 7.3.6 Design and development verification - Do most folks create a separate SOP? ISO 13485:2016 - Medical Device Quality Management Systems 4
K Joint approval between OEM and Manufacturer on Design Documents ISO 13485:2016 - Medical Device Quality Management Systems 4
M API 4F/7K/8C Design Package Validation Oil and Gas Industry Standards and Regulations 1
A Design History File - Not ready to share the design drawings or Bill of Material US Food and Drug Administration (FDA) 2
W Need for current design or process control FMEA and Control Plans 2
A What is the difference between Design Process, Process Design and Design Control? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 2
D Test summary report example for design validation wanted - ISO 13485 ISO 13485:2016 - Medical Device Quality Management Systems 1
B Why the Greek god Hephaestus should have done a design FMEA (DFMEA) on his giant robot APQP and PPAP 1
S Documenting Design Verification Test Results (ISO 9001) Design and Development of Products and Processes 1
DuncanGibbons Understanding the applicability of Design of Experiments to the IQ OQ PQ qualification approach Reliability Analysis - Predictions, Testing and Standards 0
S Requirement to Conduct New Shelf-life Testing? (re-do testing for design change) EU Medical Device Regulations 3
A Sample Agreement available for Outsourcing Medical Device Design activity? ISO 13485:2016 - Medical Device Quality Management Systems 1
DuncanGibbons How is the arrangement between Design and Production organisation envisaged? EASA and JAA Aviation Standards and Requirements 4
L Design & Development of a SERVICE Service Industry Specific Topics 13
C Documentation for items used for Design Verification 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
P Design verification driven by new equipment. How is this different than process validation? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
A AS9102B - 3.6 Design Characteristics and form 3 AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 3
P Design FMEA - Detection Rating criteria ISO 14971 - Medical Device Risk Management 3
U Medical Device Design finalization testing ISO 13485:2016 - Medical Device Quality Management Systems 2
S MDR Delay - MDD design Change? (before new MDR DOA) EU Medical Device Regulations 8
J Iterative design and production for custom made products ISO 13485:2016 - Medical Device Quality Management Systems 3
T Design Input detail & specificity ISO 13485:2016 - Medical Device Quality Management Systems 4
J Design file for pre-existing products - Inputs and Outputs ISO 13485:2016 - Medical Device Quality Management Systems 5
D Design Transfer Template capturing Customer Specific Requirements Other Medical Device Related Standards 3
T Design Control Procedures later in the Development Process ISO 13485:2016 - Medical Device Quality Management Systems 6
M Looking for a Presentation on Design for Excellence (DfX) Manufacturing and Related Processes 2
K Old medical devices -> 7.3.7. Design and development validation ISO 13485:2016 - Medical Device Quality Management Systems 1
R Design and Manufacture Guidelines for Surface Mount Technology Misc. Quality Assurance and Business Systems Related Topics 9
optomist1 Design Exclusion, but now we might have an outsourced Product Design ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
Q Relabeler for patent expired product - design control responsibilities? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 2
B Supplier of design and manufacture process ISO 13485:2016 - Medical Device Quality Management Systems 10
I Does anybody use Detection in medical device Design FMEA? ISO 14971 - Medical Device Risk Management 18

Similar threads

Top Bottom