Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [geometry]

For on-topic questions involving geometry. For purely mathematical question about geometry it is better to ask on the math SE site

Wikipedia has articles https://en.wikipedia.org/wiki/Geometry and https://en.wikipedia.org/wiki/Computational_geometry with further references.

142 questions
83
votes
4 answers

How to visualize what canonical correlation analysis does (in comparison to what principal component analysis does)?

Canonical correlation analysis (CCA) is a technique related to principal component analysis (PCA). While it is easy to teach PCA or linear regression using a scatter plot (see a few thousand examples on google image search), I have not seen a…
figure
  • 933
  • 2
  • 7
  • 6
76
votes
1 answer

Impractical question: is it possible to find the regression line using a ruler and compass?

The ancient greeks famously sought to construct geometrical relationships using only a ruler and a compass. Given a set of points in a two dimensional plane, is it possible to find the OLS line using only such instruments? This question has…
Pablo Derbez
  • 808
  • 3
  • 9
50
votes
3 answers

Suppression effect in regression: definition and visual explanation/depiction

What is a suppressor variable in multiple regression and what might be the ways to display suppression effect visually (its mechanics or its evidence in results)? I'd like to invite everybody who has a thought, to share.
ttnphns
  • 51,648
  • 40
  • 253
  • 462
39
votes
2 answers

How does Factor Analysis explain the covariance while PCA explains the variance?

Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis": According to the highlighted part, factor analysis captures the covariance between variables in the matrix $W$. I wonder HOW? Here is…
avocado
  • 3,045
  • 5
  • 32
  • 45
30
votes
1 answer

Geometric interpretation of multiple correlation coefficient $R$ and coefficient of determination $R^2$

I am interested in the geometric meaning of the multiple correlation $R$ and coefficient of determination $R^2$ in the regression $y_i = \beta_1 + \beta_2 x_{2,i} + \dots + \beta_k x_{k,i} + \epsilon_i $, or in vector notation, $$\mathbf{y} =…
Silverfish
  • 20,678
  • 23
  • 92
  • 180
27
votes
2 answers

Geometric interpretation of penalized linear regression

I know that linear regression can be thought as "the line that is vertically closest to all the points": But there is another way to see it, by visualizing the column space, as "the projection onto the space spanned by the columns of the…
Lucas Reis
  • 1,962
  • 3
  • 16
  • 15
19
votes
2 answers

How to plot an ellipse from eigenvalues and eigenvectors in R?

Could someone come up with R code to plot an ellipse from the eigenvalues and the eigenvectors of the following matrix $$ \mathbf{A} = \left( \begin{array} {cc} 2.2 & 0.4\\ 0.4 & 2.8 \end{array} \right) $$
MYaseen208
  • 2,379
  • 7
  • 32
  • 46
19
votes
1 answer

Geometric understanding of PCA in the subject (dual) space

I am trying to get an intuitive understanding of how principal component analysis (PCA) works in subject (dual) space. Consider 2D dataset with two variables, $x_1$ and $x_2$, and $n$ data points (data matrix $\mathbf X$ is $n\times 2$ and is…
amoeba
  • 93,463
  • 28
  • 275
  • 317
18
votes
5 answers

How is the kurtosis of a distribution related to the geometry of the density function?

The kurtosis is to measure the peakedness and flatness of a distribution. The density function of the distribution, if it exists, can be viewed as a curve, and has geometric features (such as curvature, convexity, ...) related to its shape. So I…
Tim
  • 1
  • 29
  • 102
  • 189
13
votes
1 answer

Geometric interpretation of generalized linear model

For linear model $y=x\beta+e$, we can have a nice geometric interpretation of estimated model via OLS: $\hat{y}=x\hat{\beta}+\hat{e}$. $\hat{y}$ is the projection of y onto the space spanned by x and residual $\hat{e}$ is perpendicular to this space…
Vincent
  • 131
  • 3
11
votes
1 answer

Clarification in information geometry

This question is concerned with the paper Differential Geometry of Curved Exponential Families-Curvatures and Information Loss by Amari. The text goes as follows. Let $S^n=\{p_{\theta}\}$ be an $n$-dimensional manifold of probability distributions…
Ashok
  • 1,011
  • 7
  • 22
11
votes
1 answer

Geometric interpretation of the maximum likelihood estimation

I was reading the book The Identification Problem In Econometrics by Franklin M. Fisher, and was confused by the part that he demonstrates the identification by visualizing the likelihood function. The problem could be simplified as: For a…
szw
  • 111
  • 3
10
votes
2 answers

Appropriate measure to find smallest covariance matrix

In the textbook I am reading they use positive definiteness (semi-positive definiteness) to compare two covariance matrices. The idea being that if $A-B$ is pd then $B$ is smaller than $A$. But I'm struggling to get the intuition of this…
Baz
  • 1,583
  • 3
  • 13
  • 26
10
votes
4 answers

Why does Bayes' Theorem work graphically?

From a mathematical standpoint Bayes' Theorem makes perfect sense to me (i.e., deriving and proving), but what I do not know is whether or not there is a nice geometric or graphical argument that can be shown to explain Bayes' Theorem. I tried…
user25658
10
votes
2 answers

Vector calculus in statistics

I'm teaching a class on integration of functions of several variables and vector calculus this semester. The class is made up most of economics majors and engineering majors, with a smattering of math and physics folks as well. I taught this class…
Paul Siegel
  • 221
  • 1
  • 7
1
2 3
…
9 10