Questions tagged [non-response]

Nonresponse (spelled as a single word) is the term used in survey literature to describe missing data problems; encompasses the issues of unit nonresponse, item nonresponse, and nonresponse biases

Nonresponse (spelled as a single word) is the term used in survey literature to describe missing data problems. Three issues are associated with the generic term of nonresponse:

  1. Unit nonresponse is a situation when a sampled unit does not provide any survey data on any of the survey variables at all. (This looks like a blank line in the rectangular data set.)
  2. Item nonresponse is a situation when a sampled unit provides responses on some, but not all of the survey variables. (This looks like Swiss cheese holes in the rectangular data set.)
  3. Nonresponse bias arises when the standard analyses leads to the estimates that are biased for their (finite) population targets. In terms of the standard Rubin's missing data classification, this indicates that the data are not missing at random (NMAR), although the analogies are not exact as the NMAR concept is formulated for the data likelihood which is difficult to define in informative ways for sampling designs.

The issue of nonresponse and the biases that it may cause has been the central concern for survey statisticians in the past 20 or so years. Groves (2006) is the most cited paper in Public Opinion Quarterly, one of about five journals on survey methodology. Well-designed and well-executed surveys maintain relatively little bias with response rates as low as 10% (Pew 2012).

Unit nonresponse is usually corrected by survey weights. Valliant, Dever and Kreuter (2013) provide some guidance as well as an R package to implement some of the corrections.

Item nonresponse can usually be corrected, at least partially, by imputation of the missing data.

A measure of the relative importance of nonresponse is response rate, which is the ratio of the number of completed interviews over the number of eligible reporting units in the sample. The definition is fairly technical, and involves the multiple categories of the final survey dispositions. The main reference is Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys by the American Association for Public Opinion Research (http://www.aapor.org/Publications-Media/AAPOR-Journals/Standard-Definitions.aspx). The relevant edition, as of mid 2016, is the 9th edition.

39 questions
8
votes
1 answer

Variables for post-stratification weights?

What justifies the usage of a variable for post-stratification? I am working with a constituent survey of a non-profit's constituent with 2500 responses out of a much larger sample and even larger population. I have many variables about the target…
Andrew
  • 656
  • 5
  • 11
7
votes
1 answer

Can one compute confidence intervals for a census with high nonresponse rates?

If a questionnaire was administered to an entire population (group of interest) and the response rate was 68% can the questionnaire results be generalized to the population (100%, including the 32% missing)? Can confidence intervals be used for the…
nadia
  • 89
  • 5
7
votes
2 answers

recommendations to analyze a survey of the entire sample frame with a 20% response rate

we surveyed all 10,000 professionals in a particular industry. The industry is highly-regulated, so we have contact information for everyone in our population of interest. We attempted to contact 100% of the population. We now have a data set…
Anthony Damico
  • 272
  • 2
  • 17
6
votes
2 answers

For "Was this page helpful" data, should I take response rate into account?

To help me focus on which pages on a site to improve, I'm looking at user feedback to the question "Was this page helpful?" (Answers are "Yes" or "No".) The response rate (responses divided by unique pageviews) varies across different pages. And in…
Joe Pairman
  • 205
  • 1
  • 4
5
votes
1 answer

Is overfitting an issue if all I care about is training error

I am working on a project where we perform non-response adjustment by weighting survey respondents by their probability of response. In order to do this, we need to estimate each respondents probability of response using a model (typically a…
5
votes
0 answers

Predicting response propensity in a rolling data collection

I'm working with a survey that uses a rolling data collection format (i.e., there are multiple waves of sampling and initial contacts). I'm trying to develop a model to predict how likely a sample member is to respond to the survey within 7 weeks…
5
votes
4 answers

How to avoid and handle survey non-response?

One objective of a survey can be to understand the proportion male vs. female users. To the extent a specific gender correlates to particular use cases of a technology, product design, product/feature prioritization and marketing can be differently…
user2661
  • 151
  • 1
  • 3
5
votes
6 answers

Inference to the population when the survey response rate is only 30%

I have conducted a survey in which the questionnaires were sent out to 450 individuals, but only 30% of them answered the questionnaires. Is it still valid to interpret the usual inference analysis (i.e., the inference analysis developed under the…
PaulS
  • 425
  • 4
  • 15
4
votes
1 answer

Calibration of weights for market research survey

I have a stratified random sample based on sampling frame formed from our CRM systems data. Now when I look responses in different strata they seem to differ. Some stratas have much higher response rate. My survey variables are mostly…
Analyst
  • 2,527
  • 10
  • 11
4
votes
1 answer

Should item nonresponses be removed before calculating Welch's t-test?

I performed a survey in which one of the questions used a grid to let people express preferences for different features of something, using a Likert-like scale for each feature. I coded the values as 1-5. The people surveyed fall into two disjoint…
mhucka
  • 141
  • 3
4
votes
1 answer

Calculate confidence levels of a stratified sample with missing units

I'm conducting a stratified survey among health institutions; 700 units were allocated into one of the 30 strata designed to reflect the proportionality of the population. However, it's been difficult to obtain even 450 questionnaires, even after…
3
votes
1 answer

How to calculate confidence interval when only a part of the samples are valid?

I will simplify our problem in this way. Say there are 100,000 cases in total to examine. Due to the time limitation, we randomly selected 2,000 of them. Then we found 1,000 of them are invalid, so we have only 1,000 valid cases left. Finally we…
3
votes
0 answers

Adjusting for non-response bias in a survey

Say I have a target population of 10,000 for a survey, and only 2,000 respond, because they likely feel strongly about the survey topic (either happy or angry). I have clear, abundant non-response bias. Say one of the questions is to rate something…
3
votes
1 answer

How does non-response bias affect the Confidence Interval

I asked earlier what is the idea behind putting up Table 3. I have a follow up question on how non-response bias identified can justify the different sizes of Confidence Interval(CI) in Figure 3. My reasoning is that assuming p-value are small for…
3
votes
2 answers

References on handling nonresponse (participant dropout) bias in experimental setting

Imagine I do a randomized experiment at the beginning of the school year. Incoming freshmen (a) participate in a diversity class or (b) do not. At the end of the year, I send them emails asking to fill out one 4-point Likert question on how they…
Mark White
  • 8,712
  • 4
  • 23
  • 61
1
2 3