Most Popular
1500 questions
39
votes
3 answers
Why is RSS distributed chi square times n-p?
I would like to understand why, under the OLS model, the RSS (residual sum of squares) is distributed $$\chi^2\cdot (n-p)$$ ($p$ being the number of parameters in the model, $n$ the number of observations).
I apologize for asking such a basic…
Tal Galili
- 19,935
- 32
- 133
- 195
39
votes
7 answers
What are the most common biases humans make when collecting or interpreting data?
I am an econ/stat major. I am aware that economists have tried to modify their assumptions about human behavior and rationality by identifying situations in which people don't behave rationally. For example, suppose I offer you a 100% chance of a…
Stan Shunpike
- 3,623
- 2
- 27
- 36
39
votes
3 answers
What does entropy tell us?
I am reading about entropy and am having a hard time conceptualizing what it means in the continuous case. The wiki page states the following:
The probability distribution of the events, coupled with the
information amount of every event, forms…
RustyStatistician
- 1,709
- 3
- 13
- 35
39
votes
3 answers
Relation between confidence interval and testing statistical hypothesis for t-test
It is well known that confidence intervals and testing statistical hypothesis are strongly related. My questions is focused on comparison of means for two groups based on a numerical variable. Let's assume that such hypothesis is tested using…
Lan
- 1,231
- 2
- 14
- 16
39
votes
2 answers
Theory behind partial least squares regression
Can anyone recommend a good exposition of the theory behind partial least squares regression (available online) for someone who understands SVD and PCA? I have looked at many sources online and have not found anything that had the right combination…
ClarPaul
- 1,130
- 11
- 18
39
votes
2 answers
Why is Laplace prior producing sparse solutions?
I was looking through the literature on regularization, and often see paragraphs that links L2 regulatization with Gaussian prior, and L1 with Laplace centered on zero.
I know how these priors look like, but I don't understand, how it translates to,…
Dmitry Smirnov
- 635
- 1
- 6
- 11
39
votes
6 answers
What's the difference between logistic regression and perceptron?
I'm going through Andrew Ng's lecture notes on Machine Learning.
The notes introduce us to logistic regression and then to perceptron. While describing Perceptron, the notes say that we just change the definition of the threshold function used for…
GrowinMan
- 831
- 2
- 8
- 8
39
votes
6 answers
Why does the variance of the Random walk increase?
The random walk that is defined as $Y_{t} = Y_{t-1} + e_t$, where $e_t$ is white noise. Denotes that the current position is the sum of the previous position + an unpredicted term.
You can prove that the mean function $\mu_t = 0 $, since
$$E(Y_{t})…
Isbister
- 679
- 1
- 8
- 17
39
votes
1 answer
What is the intuitive reason behind doing rotations in Factor Analysis/PCA & how to select appropriate rotation?
My Questions
What is the intuitive reason behind doing rotations of factors in factor analysis (or components in PCA)?
My understanding is, if variables are almost equally loaded in the top components (or factors) then obviously it is difficult to…
GeorgeOfTheRF
- 5,063
- 14
- 42
- 51
39
votes
3 answers
What is compound symmetry in plain english?
I recently realized that a mixed-model with only subject as a random factor and the other factors as fixed factors is equivalent to an ANOVA when setting the correlational structure of the mixed model to compound symmetry.
Therefore I would like to…
Henrik
- 13,314
- 9
- 63
- 123
39
votes
3 answers
Logistic Regression: Bernoulli vs. Binomial Response Variables
I want to perform logistic regression with the following binomial response and with $X_1$ and $X_2$ as my predictors.
I can present the same data as Bernoulli responses in the following format.
The logistic regression outputs for these 2 data sets…
A Scientist
- 553
- 1
- 5
- 7
39
votes
8 answers
Random walk on the edges of a cube
An ant is placed in a corner of a cube and cannot move. A spider starts from the opposite corner, and can move along the cube's edges in any direction $(x,y,z)$ with equal probability $1/3$. On average, how many steps will the spider need to get to…
Elizabeth Susan Joseph
- 987
- 3
- 10
- 18
39
votes
3 answers
Creating a "certainty score" from the votes in random forests?
I am looking to train a classifier that will discriminate between Type A and Type B objects with a reasonably large training set of approximately 10,000 objects, about half of which are Type A and half of which are Type B. The dataset consists of…
ialm
- 1,707
- 2
- 19
- 19
39
votes
2 answers
Why should we use t errors instead of normal errors?
In this blog post by Andrew Gelman, there is the following passage:
The Bayesian models of 50 years ago seem hopelessly simple (except, of
course, for simple problems), and I expect the Bayesian models of
today will seem hopelessly simple, 50…
Potato
- 1,025
- 1
- 11
- 12
39
votes
9 answers
What is the relationship between $Y$ and $X$ in this plot?
What is the relationship between $Y$ and $X$ in the following plot?
In my view there is negative linear relationship, But because we have a lot of outliers, the relationship is very weak. Am I right?
I want to learn how can we explain…
PSS
- 773
- 3
- 9
- 14