Questions tagged [experiment-design]

The study of how to structure an information-gathering exercise where variation is present.

The design of experiments (DOE) is the study of how to structure an information-gathering exercise where the relationship between variables is not deterministic (i.e., at least some random variation is present). In statistics, controlled experiments are often implied, meaning that study units are randomly assigned to conditions in which the level of a variable[s] has been independently manipulated (minimally, and prototypically, to create a treatment condition and a control condition).

In the design of experiments, the experimenter is often interested in the effect of some process or intervention (the "treatment") on a response variable for some objects (the "experimental units"). True experiments are typically thought necessary for inferring causality, although there is a field of study within statistics that concerns itself with how causality may be nonetheless inferred from observational data.

1721 questions
78
votes
12 answers

Famous statistical wins and horror stories for teaching purposes

I am designing a one year program in data analysis with a local community college. The program aims to prepare students to handle basic tasks in data analysis, visualization and summarization, advanced Excel skills and R programming. I would like…
56
votes
16 answers

Recommended books on experiment design?

What are the panel's recommendations for books on design of experiments? Ideally, books should be still in print or available electronically, although that may not always be feasible. If you feel moved to add a few words on what's so good about the…
walkytalky
  • 1,857
  • 2
  • 22
  • 24
51
votes
8 answers

Statistical tests when sample size is 1

I'm a high school math teacher who is a bit stumped. A Biology student came to me with his experiment wanting to know what kind of statistical analysis he can do with his data (yes, he should have decided that BEFORE the experiment, but I wasn't…
51
votes
11 answers

Famous easy to understand examples of a confounding variable invalidating a study

Are there any well-known statistical studies that were originally published and thought to be valid, but later had to be thrown out due to a confounding variable that wasn't taken into account? I'm looking for something easy to understand that…
39
votes
4 answers

ANOVA on binomial data

I am analyzing an experimental data set. The data consists of a paired vector of treatment type and a binomial outcome: Treatment Outcome A 1 B 0 C 0 D 1 A 0 ... In the outcome column, 1…
33
votes
4 answers

Independent variable = Random variable?

I'm slightly confused if an independent variable (also called predictor or feature) in a statistical model, for example the $X$ in linear regression $Y=\beta_0+\beta_1 X$, is a random variable ?
l7ll7
  • 1,075
  • 2
  • 9
  • 15
28
votes
4 answers

Pitfalls in experimental design: Avoiding dead experiments

I've come across this quote numerous times: To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of. -- Ronald Fisher (1938) To…
naught101
  • 4,973
  • 1
  • 51
  • 85
28
votes
3 answers

What if your random sample is clearly not representative?

What if you take a random sample and you can see it is clearly not representative, as in a recent question. For example, what if the population distribution is supposed to be symmetric around 0 and the sample you draw randomly has unbalanced…
Joel W.
  • 3,096
  • 3
  • 31
  • 45
26
votes
4 answers

What's wrong with (some) pseudo-randomization

I came across a study in which patients, who were all over 50 years of age, were pseudo-randomized by birth year. If the birth year were an even number, usual care, if an odd number, intervention. It's easier to implement, it's harder to subvert…
Jeremy Miles
  • 13,917
  • 6
  • 30
  • 64
26
votes
10 answers

Do you have recommendations for books to self-teach Applied Statistics at the graduate level?

I took several statistics courses in college but I found that my education was very theory driven. I was wondering if any of you had a text in Applied Statistics (at the graduate level) that you recommend or have had good experience with.
23
votes
2 answers

How do you "control" for a factor/variable?

To my understanding, "Control" can have two meanings in statistics. Control group: In an experiment, no treatment is given to the member of the control group. Ex: Placebo vs Drug: You give drugs to one group and not to the other (control), which…
23
votes
5 answers

What is a block in experimental design?

I have two questions about the notion of block in experimental design : (1) What is the difference between a block and a factor ? (2) I tried to read some books but something is not clear: it seems that the authors always assume that there is no…
Stéphane Laurent
  • 17,425
  • 5
  • 59
  • 101
21
votes
3 answers

Should feature selection be performed only on training data (or all data)?

Should be feature selection performed only on training data (or all data)? I went through some discussions and papers such as Guyon (2003) and Singhi and Liu (2006), but still not sure about right answer. My experiment setup is as follows: Dataset:…
17
votes
3 answers

What is the difference between a "statistical experiment" and a "statistical model"?

I am following A.W. van der Vaart, asymptotic statistics (1998). He talks of statistical experiments, claiming that they are different from a statistical model, but he defines neither. My question: What is a statistical experiment, a statistical…
16
votes
3 answers

Is there a general method for simulating data from a formula or analysis available?

De novo simulation of data from an experimental design data frame. With a focus on R (though other language solution would be great). In designing an experiment or a survey, simulating data and conducting an analysis on this simulated data can…
Etienne Low-Décarie
  • 1,563
  • 3
  • 16
  • 27
1
2 3
99 100