Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [projection]

For on-topic questions involving the mathematical concept projection, a linear transformation $P$ such that $P=P^2$. Please include also a more statistical methods tag. For purely mathematical questions about projections it is better to ask on math SE https://math.stackexchange.com/

Wikipedia has an article https://en.wikipedia.org/wiki/Projection_(linear_algebra) with further references.

80 questions
19
votes
3 answers

Utility of the Frisch-Waugh theorem

I am supposed to teach the Frish Waugh theorem in econometrics, which I have not studied. I have understood the maths behind it and I hope the idea too "the coefficient you get for a particular coefficient from a multiple linear model is equal to…
Anthony Martin
  • 1,109
  • 3
  • 11
  • 26
10
votes
3 answers

difference between linear projection and linear regression (OLS)?

In http://www.wouterdenhaan.com/numerical/slidesbayesian.pdf (approximately from page 7 to 13), ordinary least squares and linear projection are said to be different. But from my linear algebra class, I remember hearing that OLS is indeed projection…
user79337
  • 131
  • 1
  • 2
  • 5
9
votes
1 answer

Intuition using linear algebra that the rank of the projection matrix equals the rank of the design matrix

Using linear algebra to explain, can someone show the intuition? I can show that the ranks are the same by using properties of rank but can't get my head around the whole projection thing more than just thinking hazily.
python_learner
  • 579
  • 5
  • 14
8
votes
1 answer

Regularization and projection onto the $l_*$ ball

I'm trying to understand how regularization works in term of projections onto a $l_*$ ball, and Euclidean projection onto the simplex. I'm not sure I understand what we mean when we project the weight vector onto the $l_1$ or the $l_2$ balls. I can…
Bar
  • 2,492
  • 3
  • 19
  • 31
7
votes
1 answer

Do the principal components change if we apply PCA more than once (recursively) on data?

Consider a set $X=(X_1; \dots; X_n)$ of $n$ data points such that $X_i \in \mathbb{R}^d$ is a column vector. Let $Y = \text{pca_proj}(X)$ denote the projection of points in $X$ according to the PCA components i.e. $$Y_i \in \mathbb{R}^d\\ Y_i = W…
Sobi
  • 2,141
  • 11
  • 22
6
votes
3 answers

Projecting data on a sphere

I am used to working with PCA, tSNE, LLEs... They all do a great job projecting the data on a plane (or on linear subspaces of $\mathbb{R}^n$). Is there any other embedding technique that projects the data on non-linear spaces ? Like a sphere, or…
RUser4512
  • 9,226
  • 5
  • 29
  • 59
6
votes
2 answers

Relationship between Linear Projection and OLS Regression

In Wooldridge's Econometric Analysis of Cross Section and Panel Data, he defines linear projection of $y$ on $1,\mathbf{x}$, in the following way: Let's assume that $Var(\mathbf{x})$ is positive-definite, then the linear projection of $y$ on…
An old man in the sea.
  • 5,070
  • 3
  • 23
  • 57
5
votes
1 answer

Conditional expectation and variable decomposition

Suppose that $X$ and $Y$ have an uknown joint distribution $f_{XY}$. How can I formally demostrate that it always exists a unique decomposition of the form : $$ Y = E[Y|X] +\epsilon $$ without assuming any explicity form of $f_{XY}$?
omega
  • 407
  • 2
  • 9
5
votes
1 answer

How to show that demeaning the data in design matrix does not change the hat matrix

When I have a design matrix $$X = \begin{bmatrix} 1 & x_{11} & \ldots & x_{1k}\\ 1 & x_{21} & \ldots & x_{2k}\\ \vdots & \vdots & \ddots & \vdots\\ 1 & x_{n1} &\ldots & x_{nk} \end{bmatrix} $$ how can I show that subtracting the mean from…
ira
  • 399
  • 2
  • 14
5
votes
1 answer

Making two vectors uncorrelated in terms of Kendall Tau correlation

Assume that we have two normalized $n\times 1$ vectors $\bf x$ and $\bf y$. In terms of Pearson correlation, these two signals are uncorrelated if ${\bf x}^T {\bf y} = 0$. Now, assume that ${\bf x}^T {\bf y} \neq 0$. If we perform the following…
MMM
  • 758
  • 6
  • 8
4
votes
1 answer

In machine learning, especially deep learning, is it a principle to only do linear projections to a smaller dimension size?

Linear projection (or fully connected layer) is perhaps one of the most common operations in deep learning models. When doing linear projection, we can project a vector $x$ of dimension $n$ to a vector $y$ of dimension size $m$ by multiplying a…
Yu Gu
  • 153
  • 6
4
votes
1 answer

Intuition behind projection matrix

I'm new to machine learning and came across projection matrix . In a random thread it was interpreted as The matrix $X(X^\text{T} X)^{-1} X^\text{T}$ is a projection matrix, as it does precisely that: it is the linear transformation that…
offset-null1
  • 55
  • 4
4
votes
1 answer

Expected value as an orthogonal projection

I'm reading a paper in which the expected value of a random variable, $\mathbb{E}[X]$, is characterized as an orthogonal projection. This is on page 10. I've seen the geometric interpretation of conditional expectation but never $\mathbb…
statian
  • 327
  • 2
  • 8
4
votes
3 answers

Since a projection matrix is idempotent, symmetric and square, why isn't it just the identity matrix?

I was working on a question on projection matrix. Since, projection matrix is idempotent, symmetric and square matrix, it must always be equal to $I$ (Identity matrix). This can be shown by multiplying the inverse of projection matrix on both the…
Steve
  • 43
  • 1
  • 4
4
votes
0 answers

Projection of a spherical t-distribution

I'm working with an $n$-dimensional spherical t-distribution for which the density is defined as $f(x) = \frac{\Gamma(\frac{n+\nu}{2})}{(\pi \sigma^2\nu)^{\frac n2}\Gamma(\frac \nu2)} \left( 1 + \frac{\|x|\|^2}{\nu \sigma^2} \right)^{-…
Akkariz
  • 71
  • 4
1
2 3 4 5 6