Creating samples from a well-specified population (human: all adults; registered voters; individuals with diabetes; students of a university; establishment: all firms; firms with employment of 200 or more in New York City; resource: all land of a country or a state/province) using a probabilistic method, with the purpose of inference to that specific population
Sampling is used to collect data when observing whole population is not practical or not feasible (e.g., too expensive, conceptually impossible, etc.). To draw valid statistical inferences about sampled data, the mechanism by which the samples are drawn must be specified, and must involve randomization (selecting units using random numbers or random events). Randomization is necessary to be able to make probabilistic statements: one can talk about the mean or a tail probability of the sampling distribution of a statistic by virtue of looking at the histogram of this statistic as obtained by (hypothetically, or by actual exhaustive search) taking all possible samples from populaton and computing the statistic of interest based on every possible sample.
The simplest sampling method is simple random sampling (SRS): for a population of $N$ units, the SRS of size $n$ is a sampling design that assigns to each sample of size $n$ the same probability of selection $1/C_N^n$. This simplest method allows for inference that is nearly equivalent to the textbook "i.i.d." assumption. E.g., the minimum variance unbiased estimate of the population mean is the sample mean $\bar x$, and its variance is $s^2(1-n/N)/n$ where $s^2 = \sum (x_i - \bar x)^2/(n-1)$, and the factor $1-n/N$ is the finite population correction. However, if any other selection method was used to obtain the sample, the analysis methods must be modified to account for the features of this selection method. For instance, a naive understanding of sampling may entail thinking that if every unit in the population has the same probability of selection $n/N$, then the "i.i.d." analysis methods are applicable. This is not so; for a systematic sampling design (all units are arranged in the list, a starting point $k$ is chosen randomly as a number between 1 and $[N/n]$, and the units $k, k+[N/n], k+2[N/n], ...$ are taken into the sample), the sampling variance cannot even be estimated!
In samples of human and natural resource populations, the most typical twists on sampling selection methods include (a combination of):
- Stratification: selecting units independently within well-defined groups (e.g., regions or states in geographic samples; industry and size of an enterprize in establishment surveys; type of land use in natural resource surveys; etc.). Typically, although not necessarily, stratification leads to reduction of sampling variance.
- Multistage selection: selecting units within a specific hierarchy (schools within districts, then students within schools in education surveys; counties within states, then city blocks within counties, then households within city blocks in geographic samples; etc.). Multistage samples are also known as cluster samples (clusters of units rather than individual units are sampled at the early stages of selection). Clustering typically increases sampling variances.
- Unequal probability of selection, usually associated either with a need to obtain a sufficient number of observations for certain groups of populations, or with a need to balance costs of the survey. Unequal probabilities of selection must be accounted for by specifying (and using in analysis) sampling weights. Unweighted estimates will typically be biased, and hence of no real interest.
In some disciplines, the term "sample" is intended to mean "an observation", a single record containing data on one particular unit of analysis. More often, the term "sample" is used to denote a collection of units for which observations were made, measurements were taken, responses were obtained, etc. Furthermore, some disciplines use the term "sampling" rather loosely to indicate the process of collection data on arbitratrily taken units from the population. However, scientifically rigorous inferences can only be obtained from the samples that are random, i.e., a randomization mechanism is built into the data collection process.
To find out more, visit Wikipedia page, take a look at What Is a Survey? booklet of the American Statistical Association, or read introductory textbooks such as Lohr (2009), Kish (1995) or Cochran (1977). A complete and thorough discussion of how survey statistics should be analyzed in R is given in Lumley (2010).
Potentially related tags: survey, sample-size, response-rate, stratification, svy
Other uses of the concept of sampling: see sampling, survey-weights
