1

The following paragraph occurs in Essentials of Statistics for the Behavioral Sciences(10th edition):

If you actually wanted to construct the distribution of sample means, you would first select a random sample of a specific size (n) from a population, calculate the sample mean, and place the sample mean in a frequency distribution. Then you select another random sample with the same number of scores. Again, you calculate the sample mean and add it to your distribution.

Question: Is the random selection/sampling process (highlighted in bold) performed with replacement or without replacement?

Mainul Islam
  • 127
  • 5

2 Answers2

2

You do the sampling with replacement. The way statisticians write the sample is along the lines of $X_1,\dots,X_n\overset{iid}{\sim} F_X(x)$, meaning that every observation $X_i$ is drawn from the population $F_X(x)$ with the same probability of being a particular value as any other observation has of being that value ($iid$ means independently and identically distributed). If you put the restriction the the sampling is done without replacement, then you do not have the same probability of being the first value drawn when you get to the second draw.

Think about drawing cards from a deck. At first, there is a one in four chance of drawing a club. If the first card is a club, and I keep it out of the deck for the second draw, then your chance of being a club on the second draw is $12/51$, not $1/4$. If you put the club back in and shuffle the deck, then the chance of being a club on the second draw is again $1/4$.

Dave
  • 28,473
  • 4
  • 52
  • 104
-1

For computing the distribution of the sample mean the method described above can be used with any sampling design, that is, it is the same for the case without replacement and for the case with replacement. Indeed, considering some statistic depending on the sample $T(S)$ its sampling distribution is:

$$P(T(S)=k)=\sum_{S \ s.t. \ T(S)=k} P(S), \text{ for any } k \text{ in the image of }T.$$

In the book, $T$ is the sample mean, that is, $T(S)=\dfrac{\sum_{i\in S} X_i}{n}.$ Note that the last expression is valid for both with replacement and without replacement schemes. Indeed, for every sampling design.

However, IN SAMPLING THEORY (which is our case), if the author does not specify otherwise, the default sampling design usually is without replacement because is more accurate (it has less mean squared error (MSE) ) than the one with replacement (IN PROBABILITY AND INFERENCE THE DEFAULT SAMPLING DESIGN IS WITH REPLACEMENT).

Note that here we are computing the sampling distribution of some estimator $T(S)$ among the samples $S$ and the random element in this problem is not the value of the variable $X$ on each element of the population $X_i$ but which sample is drawn. So we deal with the probabilities $P(S)$ for each sample. In sampling theory most of times the values $X_i$ are considered deterministic and the actual random variable is the variable $e_i:$

$$e_i= \left\{ \begin{array}{lcc} 1 & if & i \in S \\ \\ 0 & if & i \notin S \end{array} \right.$$

Do not confuse sampling theory with classical probability or statistical inference in which you normally have random variables $X_1,\ldots, X_n $ which are chosen indepently (and therefore with replacement).

  • -1 It is with replacement. – Dave Apr 12 '21 at 11:07
  • Sorry, but I think you are wrong. The concept of random sample usually means different things in Inference and sampling theory. In classic statistic and inference, it means with replacement for the reason you comment above. However, in sampling theory the plot changes and usually the term random sample means without replacement. Note one important thing, in classic statistic your random variables are the observation $X_i$ but when you draw a sample the random component is not $X_i$ but which sample is selected. For this reason you are computing the distribution among the samples! – PG Segador Apr 12 '21 at 11:26
  • "Remember that random sampling requires sampling with replacement." This is what the book says on the next page on the left margin as I discovered recently. – Mainul Islam Apr 15 '21 at 03:25