3

Hi I only did some basic stats at school but now I need to rank some data for a project, and I've run into this kind of scenario

Game A: 1000 Reviews, 900 Positive and 100 Negative, so 90% positive.

Game B: 20 Reviews, 20 Positive and 0 Negative, so 100% positive.

It clearly isn't fair to say Game B is better than Game A just because it have a higher percentage of positive reviews, but I still need a way to conclude their positive percentage. How should I do so?

EDIT: In my case, it's more than just two data, it's about hundred or thousands of data, some value are just extremely large and some are small.

zenoh
  • 131
  • 3
  • 1
    For each game , let $p$ be the probability of positive review (one different $p$ for each different game). Tou could present confidence intervals for each $p$. If you have many games and want a ranking, there are statistical methods for that too, but maybe complicated with only "basic stats". Then you would need to formulate more precisely your goal. Can you post a link to your data? – kjetil b halvorsen Aug 26 '15 at 17:15
  • One approach that has been used is to calculate a "lower bound" on the positive proportion based on a [binomial proportion confidence interval](https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval). [Evan Miller](http://www.evanmiller.org/how-not-to-sort-by-average-rating.html) discussed the Wilson interval, but any of the others could be used. This sort of approach has been widely used for comparing various kinds of ratings (e.g. I gather it's used by reddit in ranking comments). ... ctd – Glen_b Aug 26 '15 at 22:54
  • ctd... This is the subject of a number of posts on our site, but you should probably start with [this one](http://stats.stackexchange.com/questions/15979/how-to-find-confidence-intervals-for-ratings) since the discussion there (at least briefly) mentions problems and alternatives. \[Edit -- It looks like reddit's use of it had - at the time [this](https://possiblywrong.wordpress.com/2011/06/05/reddits-comment-ranking-algorithm/) was written - some implementation issues.\] – Glen_b Aug 26 '15 at 23:12

2 Answers2

1

Because the standard Binomial confidence interval doesn't work when $p=0$ or $p=1$ you can try using the exact binomial confidence interval to calculate a confidence score for the 1st and second games and then compare the scores.

from scipy.stats import beta

# calculate lower bound for game A
lower_a = 1-beta.ppf(.975, 101, 900)
# calculate upper bound for game A
upper_a = 1-beta.ppf(0.025, 100, 901)

# calculate lower bound for game B
lower_b = 1-beta.ppf(.975, 1, 20)

Then you get your answer

Game A: $0.879 \leq \mu_A \leq 0.918$

Game B: $0.83 \leq \mu_B \leq 1.0$

So from your current data you can't tell if either game is better than the other.

Greg
  • 71
  • 4
0

You could use the Fisher's exact test to test the significance of your result.

For a quick check, you can use this online calculator and plug in your numbers.

Regenschein
  • 305
  • 4
  • 13
  • is the Fisher's exact test only for two data? Men and Woman, A and B etc... In my case it's about hundred or thousands of data, some are just extremely large and some are small. – zenoh Aug 26 '15 at 16:04
  • I didn't see that in the question. I suggest to update the question. Thanks. – Regenschein Aug 26 '15 at 19:10
  • There is a similiar question http://stackoverflow.com/questions/20499745/fisher-test-more-than-2-groups and here http://stats.stackexchange.com/questions/1805/fishers-exact-test-in-contingency-tables-larger-than-2x2 – Regenschein Aug 26 '15 at 19:12