Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Most Popular

more tags
1500 questions
92
votes
2 answers

Subscript notation in expectations

What is the exact meaning of the subscript notation $\mathbb{E}_X[f(X)]$ in conditional expectations in the framework of measure theory ? These subscripts do not appear in the definition of conditional expectation, but we may see for example in this…
Emile
  • 3,150
  • 2
  • 20
  • 17
92
votes
8 answers

How to compute precision/recall for multiclass-multilabel classification?

I'm wondering how to calculate precision and recall measures for multiclass multilabel classification, i.e. classification where there are more than two labels, and where each instance can have multiple labels?
Vam
  • 1,245
  • 1
  • 10
  • 9
91
votes
5 answers

Kendall Tau or Spearman's rho?

In which cases should one prefer the one over the other? I found someone who claims an advantage for Kendall, for pedagogical reasons, are there other reasons?
Tal Galili
  • 19,935
  • 32
  • 133
  • 195
91
votes
6 answers

If I have a 58% chance of winning a point, what's the chance of me winning a ping pong game to 21, win by 2?

I have a bet with a co-worker that out of 50 ping pong games (first to win 21 points, win by 2), I will win all 50. So far we've played 15 games and on average I win 58% of the points, plus I've won all the games so far. So we're wondering if I have…
richard
  • 881
  • 1
  • 7
  • 5
91
votes
7 answers

How to efficiently manage a statistical analysis project?

We often hear of project management and design patterns in computer science, but less frequently in statistical analysis. However, it seems that a decisive step toward designing an effective and durable statistical project is to keep things…
chl
  • 50,972
  • 18
  • 205
  • 364
91
votes
4 answers

When to use Fisher and Neyman-Pearson framework?

I've been reading a lot lately about the differences between Fisher's method of hypothesis testing and the Neyman-Pearson school of thought. My question is, ignoring philosophical objections for a moment; when should we use the Fisher's approach of…
Stijn
  • 1,550
  • 1
  • 12
  • 20
91
votes
7 answers

Why to optimize max log probability instead of probability

In most machine learning tasks where you can formulate some probability $p$ which should be maximised, we would actually optimize the log probability $\log p$ instead of the probability for some parameters $\theta$. E.g. in maximum likelihood…
Albert
  • 1,145
  • 1
  • 9
  • 12
91
votes
2 answers

Given the power of computers these days, is there ever a reason to do a chi-squared test rather than Fisher's exact test?

Given that software can do the Fisher's exact test calculation so easily nowadays, is there any circumstance where, theoretically or practically, the chi-squared test is actually preferable to Fisher's exact test? Advantages of the Fisher's exact…
pmgjones
  • 5,543
  • 8
  • 36
  • 36
91
votes
1 answer

How to apply Neural Network to time series forecasting?

I'm new to machine learning, and I have been trying to figure out how to apply neural network to time series forecasting. I have found resource related to my query, but I seem to still be a bit lost. I think a basic explanation without too much…
solartic
  • 1,045
  • 1
  • 8
  • 8
90
votes
9 answers

What is meant by a "random variable"?

What do they mean when they say "random variable"?
Baltimark
  • 2,028
  • 3
  • 19
  • 20
90
votes
10 answers

Is there a minimum sample size required for the t-test to be valid?

I'm currently working on a quasi-experimental research paper. I only have a sample size of 15 due to low population within the chosen area and that only 15 fit my criteria. Is 15 the minimum sample size to compute for t-test and F-test? If so, where…
Czarina Francoise
  • 901
  • 1
  • 7
  • 4
90
votes
7 answers

Euclidean distance is usually not good for sparse data (and more general case)?

I have seen somewhere that classical distances (like Euclidean distance) become weakly discriminant when we have multidimensional and sparse data. Why? Do you have an example of two sparse data vectors where the Euclidean distance does not perform…
shn
  • 2,479
  • 9
  • 31
  • 38
90
votes
5 answers

How to 'sum' a standard deviation?

I have a monthly average for a value and a standard deviation corresponding to that average. I am now computing the annual average as the sum of monthly averages, how can I represent the standard deviation for the summed average ? For example…
klonq
  • 1,167
  • 2
  • 9
  • 9
90
votes
6 answers

Feature selection for "final" model when performing cross-validation in machine learning

I am getting a bit confused about feature selection and machine learning and I was wondering if you could help me out. I have a microarray dataset that is classified into two groups and has 1000s of features. My aim is to get a small number of…
danielsbrewer
  • 2,385
  • 3
  • 20
  • 17
90
votes
15 answers

What do you call an average that does not include outliers?

What do you call an average that does not include outliers? For example if you have a set: {90,89,92,91,5} avg = 73.4 but excluding the outlier (5) we have {90,89,92,91(,5)} avg = 90.5 How do you describe this average in statistics?
Tawani
  • 1,003
  • 1
  • 7
  • 5
1 2 3
…
99 100