Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [reduced-rank-regression]

Multivariate multiple linear regression with a constraint that the coefficient matrix should be of low rank.

Reduced rank regression (RRR) is multivariate multiple linear regression with a constraint that the coefficient matrix should be of low rank. It allows to investigate how one multivariate dataset $X$ can predict another multivariate dataset $Y$, by performing a dimensionality reduction in both $X$ and $Y$ at the same time. It is closely related to canonical correlation analysis (CCA).

17 questions
29
votes
4 answers

What is the relationship between regression and linear discriminant analysis (LDA)?

Is there a relationship between regression and linear discriminant analysis (LDA)? What are their similarities and differences? Does it make any difference if there are two classes or more than two classes?
zca0
  • 761
  • 3
  • 9
  • 14
29
votes
2 answers

What is "reduced-rank regression" all about?

I have been reading The Elements of Statistical Learning and I could not understand what Section 3.7 "Multiple outcome shrinkage and selection" is all about. It talks about RRR (reduced-rank regression), and I can only understand that the premise is…
cgo
  • 7,445
  • 10
  • 42
  • 61
20
votes
1 answer

What is the connection between partial least squares, reduced rank regression, and principal component regression?

Are reduced rank regression and principal component regression just special cases of partial least squares? This tutorial (Page 6, "Comparison of Objectives") states that when we do partial least squares without projecting X or Y (i.e., "not…
Minkov
  • 415
  • 3
  • 7
12
votes
1 answer

Friendly tutorial or introduction to reduced-rank regression

I am trying to learn Reduced-Rank Regression (RRR) from The Elements of Statistical Learning. I find the writing and them mathematics a little too prohibitive. Does any of you have a resource/text/introduction/tutorial that is friendlier as an…
cgo
  • 7,445
  • 10
  • 42
  • 61
8
votes
1 answer

Probabilistic models for partial least squares, reduced rank regression, and canonical correlation analysis?

This question results from the discussion following a previous question: What is the connection between partial least squares, reduced rank regression, and principal component regression? For principal component analysis, a commonly used…
Minkov
  • 415
  • 3
  • 7
6
votes
2 answers

How to implement reduced-rank regression in R?

How can I fit reduced-rank regression with continuous response in R? I found the package VGAM but it only fits for discrete distributions...
Daniel Falbel
  • 570
  • 1
  • 3
  • 15
5
votes
1 answer

Is there a way to specify reduced-rank regression using $\mathbf{y} = \mathbf{X}\boldsymbol\beta + \boldsymbol\epsilon$?

In grad school, I was always taught the general linear model $$\mathbf{y} = \mathbf{X}\boldsymbol\beta + \boldsymbol\epsilon\tag{1}$$ where $\mathbf{y}$ is a vector, $\mathbf{X}$ is some matrix, $\boldsymbol\beta$ is a parameter vector, and…
Clarinetist
  • 3,761
  • 3
  • 25
  • 70
5
votes
1 answer

Objective function of canonical correlation analysis (CCA)

Given two vectors of random variables $X$ and $Y$, Canonical Correlation Analysis (CCA) finds the transformation matrices $A$ and $B$ so that $\operatorname{corr}(A_{1*} X, B_{1*} Y)$ is first maximal, $\operatorname{corr}(A_{2*} X, B_{2*} Y)$ is…
statotito
  • 53
  • 6
3
votes
0 answers

Low Rank Gaussian Process vs Bayesian Linear Regression

A main benefit of Gaussian Process Regression is, that we not only get a prediction, but also a variance that we might use as indication of the prediction confidence. While bayesian linear regression also provides a variance, it is well published…
0-_-0
  • 227
  • 2
  • 5
3
votes
1 answer

Reduced rank regression with binary outcome variable

I am trying to find dietary patterns related to a disease outcome. Unfortunately, I only have the binary outcome "disease yes/no" as outcome. I tried to perform PCA on the data, but the dietary patterns are not specific enough for my outcome.…
user131483
  • 33
  • 4
3
votes
0 answers

Definition of "meta-parameter"

What is meant by the term "meta-parameter"? Can a definition, informal and/or formal, be provided? For example, in reduced-rank regression, the rank ($r$) can be referred to as a meta-parameter of the method. For instance, see these notes, which I…
Graeme Walsh
  • 3,927
  • 2
  • 24
  • 44
2
votes
1 answer

Is low rank finite-iteration manifold identification possible?

In sparse optimization, I am trying to solve the problem $$ \min_{x\in \mathbb R^{n}} \quad f(x) + \|x\|_1 $$ and at optimality, $x^*$ may be sparse. If I define the sparse manifold as $\mathcal M = \{z : z_i = 0 \text{ whenever } x_i^* = 0\}$,…
Y. S.
  • 1,237
  • 3
  • 9
  • 14
2
votes
0 answers

how to optimize reduced rank regression with constant diagnoal constraint?

I am trying to optimize a panel regression $G=\beta G+e$. $G \in R^{N\times T}$. $\beta\in R^{N\times N}$ is unknown coefficient, constrained to $diag(\beta)=0$, and reduced rank $rank(\beta)\leq r$. Formally, \begin{eqnarray} \begin{aligned} &…
kangyin ye
  • 41
  • 3
1
vote
0 answers

Noisy Observation of Matrix of Certain Rank

Consider a rank k matrix, call it M, of size nxm. All the elements are non-negative. Now do a noisy observation of it and assume independent Poissonian errors (the error on element $M_{ij}$ is Gaussian with a variance equal to the value of the…
Patrick
  • 103
  • 4
0
votes
1 answer

Mediating effects in reduced rank regression

So with reduced rank regression we identify response variables associated with our outcome of interest (Y) and model proc pls relationships between the independent variables say dietary data vs. the selected responses. If there is an association…
Patsy
  • 1
  • 2
1
2