So probability and statistics is a complex science. Casual understanding is not understanding it's acquaintance.
to paraphrase the tagline that one of our cove members uses: "words have different meanings"
"Experiment" is one of those words - I prefer study design but that doesn't necessarily help here. Study designs or experiments have different purposes:
The enumerative study seeks to estimate parameters of a population in order to determine what to do with it. Inspection census and surveys are all examples of enumerative studies. of course we learn something from these studies, but we only learn about the specific population under study. we cannot make predictions from these studies.
The Analytic class of studies is about understanding the underlying system to some useful level in order to predict future behavior. Analytic studies fall in to 3 broad purposes (maybe more?)
- Exploratory: typically looking fro rough patterns and extent. it describes what's happening in high level terms.
- Diagnostic: fine detail of the causal mechanism and/or what will improve it.
- Confirmatory: verification and validation that we got it right.
Although there is - hopefully - no new 'learning' from a confirmatory study, there is still immense value in demonstrating that we got it right. Many Customers and our own internal QMS require this demonstration. in automotive the demonstration takes the form of capability studies. in medical devices they take the form or IQ, OQ and PQ validations. The FDA, FAA and USDA all require some level of confirmatory study to acquire licenses to sell product. and the study design that demonstrates the validity of our 'claim' is also critical. I've known many people who biased their study to show a result they wanted to 'prove'. This is scientific malpractice at best and illegal at worst.