2

Given

  • Likert scale survey responses: "Very Satisfied", "Very Dissatisfied", "Dissatisfied", "Neutral", "Neutral", "Very Satisfied", "Satisfied", "Very Satisfied", "Very Satisfied", "Satisfied", "Very Satisfied"
  • Map between Likert scale and numeric values: "Very Dissatisfied": 1, "Dissatisfied": 2, "Neutral": 3, "Satisfied": 4, "Very Satisfied": 5
  • Manager's claim: The typical sentiment of customers at her store is more than neutral.

Questions (with attempts)

  1. Translate the raw data into pseudo numbers.
    • x = [5, 1, 2, 3, 3, 5, 4, 5, 5, 4, 5]
  2. Specify clearly the "parameter" in the nonparametric setting that best captures what must be measured to assess the manager's claim. Denote that quantity $\theta$.
    • Since mean is greatly affected by outliers, we should use median: $\theta = \text{median}(x)$.
  3. Write down the null and alternative hypotheses of the hypothesis testing task translates the manager's claim.
    • ${\tt H_0}$: $\theta \leq 3$ vs. ${\tt H_A}$: $\theta > 3$.

This is where I start to be unsure of my answers.

  1. Write down the formula of the appropriate test statistic $B$ for the sign test to be used to assess the manager's claim [Hint: Use the exclusion approach on ties, so that used sample size is reduced]. $$ B = \sum_{i=1}^{n} \psi_i \text{ where } \psi_i = \begin{cases} \begin{matrix} 1 & \text{if } x_i > 3 \\ 0 & \text{if } x_i < 3 \end{matrix} \end{cases} $$
  2. Write down the sampling distribution of $B$.
    • $B$ is the sum of Bernoulli random variables, so it has a binomial distribution: $B \sim \text{Binom}(n, p)$.
    • Since we have two values of $x$ that equal $3$, we ignore them, giving $n=9$. Since we're dealing with a bionomial distribution, $p=0.5$.
  3. At significance level $\alpha=0.05$, write down the rejection region ${\tt RR}_{0.05}$.
    • Since $n \times p_0 = 4.5$ and $n \times (1-p_0) = 4.5$ are both less than $5$, we cannot use the approximate test statistic: $Z = \frac{B - \text{mean}(B)}{\sqrt{\text{variance}(B)}} = \frac{B - n * p_0}{\sqrt{n * p_0 (1 - n * p_0)}}$
    • Thus, our test statistic is $\text{min}(B<3, B>3) = 2$.
    • ${\tt RR}_{\alpha=0.05} = \{B\colon B_{\tt obs} \geq 2$.
  4. Compute $B_{\tt obs}$, the observed value of the test statistic for the data.
    • $B_{\tt obs} = 7$
  5. Provide your final decision on this test at significance level $\alpha=0.05$.
    • $B_{\tt obs} = 7$ falls inside of the rejection region, so we reject $H_0$ in favor of $H_A$.
  6. Compute the p-value for this test and comment on what it says.
    • p-value $= \text{Prob}(B \geq B_{obs} | H_0 \text{is true})$
    • In ${\tt R}$: p-value $= 1-\text{pbinom}(7, 9, 0.5) = 0.01953125$.
    • Since the p-value is less than $\alpha$, reject $H_0$.
  7. Provide a $95\%$ lower confidence bound for $\theta$.
    • Not sure how to do this.
  8. Use the ${\tt R}$ function ${\tt SIGN.test()}$ to perform the same test performed step by step earlier.
    • sign_test <- SIGN.test(x, md=3, alternative='greater', conf.level = 0.95)
    • pvalue <- sign_test$p.value
    • Since the pvalue $= 0.089$ is greater than $\alpha = 0.05$, we fail to reject $H_0$.

The conclusion from Question 11 disagrees with those from Question 8 and 9. Since we have the sample data, we know that the $\text{median}(x) = 4$, so we definitely should be rejecting $H_0$, but the ${\tt SIGN.test()}$ function is telling us that we can't.

I don't just want answers; I want to understand how to solve problems like this. Thank you in advance for any help you can give me!

inkalchemist1994
  • 155
  • 9
  • The calculated median of your sample is 4, but with a small sample size, you don't have good evidence that the median of the population isn't 3. If you had a similar set of data, but more observations, you might get a lower p-value. For example, if you modify the data so that, `x = c(x, x, x)`, you'll see you get the same sample median, but a lower p-value. I also recommend not using `alternative='greater'`; instead I would approach this as a two-sided test. Your `SIGN.test` looks good. But for the equivalent binomial test you can use `binom.test(7, 9, 0.5)`. – Sal Mangiafico Feb 17 '19 at 13:44
  • Why do you suggest a two-sided test? If I use `alternative='two.sided'`, my pvalue for Question 11 becomes `0.1797`; my conclusion doesn't change. Why is there disagreement from the `SIGN.test`? My intuition tells me that the `SIGN.test` should be more robust, but I have two other answers (8 and 9) that disagree with it. – inkalchemist1994 Feb 17 '19 at 14:21
  • The `SIGN.test` is correct. (You could use either a one-or two sided test). The `binom.test(7,9,0.5)` will be the same (either as a one-sided or two-sided test.) I think `1-pbinom(7,9,0.5)` is not correct. I'm not sure what's wrong with your reasoning outside of using R. – Sal Mangiafico Feb 17 '19 at 15:09
  • ***In general:*** I think the cases when we should be using a one-sided test are few. In theory you choose what statistical test before you look at the data. So if you are trying to determine if the data differ from 3, you would be interested to know either if the data were less than 3 OR greater than 3. ***In your case:*** I see now in #3, that you defined your hypotheses as one-sided tests, so it's fine to use the one-sided test. – Sal Mangiafico Feb 17 '19 at 15:17
  • `1-pbinom(7,9,0.5) = 0.01953125`, but I've seen some people do `1-pbinom(7-1,9,0.5) = 0.08984375`, which gives the same p-value as `SIGN.test(x, md=3, alternative='greater', conf.level=0.95) = 0.08984`. Also, `binom.test(7, 9, 0.5, alternative='greater')` gives the p-value `0.08984`. – inkalchemist1994 Feb 17 '19 at 16:05
  • I guess my biggest confusion is that the tests tell us we don't have enough evidence to reject $H_0$, but manual inspection of the data says we definitely should be? – inkalchemist1994 Feb 17 '19 at 16:07
  • Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/89844/discussion-between-inkalchemist1994-and-sal-mangiafico). – inkalchemist1994 Feb 17 '19 at 16:17

2 Answers2

2

Excluding ties makes it easier to get the p-value exactly by writing down "at least as extreme as observed" means. We want to compute the probability of 7 or more positive responses out of 9. This is 7, 8 or 9 positive responses. Since there are two negative responses (1 or 2) and two positive responses (3 and 4), the probability of negative response is 1/2 and the probability of a negative response is also 1/2. So the p-value is:

# Pr{9 pos, 0 neg} + Pr{8 pos, 1 neg} + Pr{7 pos, 2 neg}
choose(9,9) * (1/2)^9 + choose(9, 8) * (1/2)^9 + choose(9,7) * (1/2)^9
#> 0.08984375

This is exactly the same p-value as the one-sided binomial test when prob=0.5 because the binomial test is doing the same computation, with a caveat:

pbinom(7,9,0.5) = P{X<=7}, so 1 - binom(7,9,0.5) = P{X>7}   <-- this would exclude 7 positive responses
pbinom(6,0,0.5) = P{X<=6}, so 1 - binom(6,9,0.5) = P{X>6} = P{X>=7}
dipetkov
  • 261
  • 1
  • 3
1

This is just an extended comment, continuing the discussion with the OP.

By definition, the p-value is the probability of getting data as extreme as the observed, assuming that the null hypothesis is true. Your result is that, if you had a population with a median of 3, and you randomly sampled this population for 9 observations ignoring those equal to 3, there's a probability of 0.09 that at least 7 of those observations would be greater than 3.

It's ultimately up to you if you find that to be good evidence against the null hypothesis.

N = 10000

Greater = rep(0,N)

A = c(1,2,4,5)

for(i in 1:N){

B = sample(A, size = 9, replace = TRUE)

if(sum(B>3)>=7){Greater[i]=1}
}

sum(Greater)/N

  ### c. 0.090
Sal Mangiafico
  • 7,128
  • 2
  • 10
  • 24