Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [partial-least-squares]

A class of linear methods for modeling the relationship between two groups of variables, X and Y. Includes PLS regression.

Partial Least Squares (PLS) is a class of linear methods for modeling the relationship between two groups of variables (X and Y). It includes regression methods, where X are independent variables and Y are dependent, as well as modeling methods that treat X and Y symmetrically. All PLS methods revolve around maximizing covariance between linear combinations of variables in X and Y, but several different variants exist in the literature.

241 questions
39
votes
2 answers

Theory behind partial least squares regression

Can anyone recommend a good exposition of the theory behind partial least squares regression (available online) for someone who understands SVD and PCA? I have looked at many sources online and have not found anything that had the right combination…
ClarPaul
  • 1,130
  • 11
  • 18
30
votes
1 answer

PCA, LDA, CCA, and PLS

How are PCA, LDA, CCA, and PLS related? They all seem "spectral" and linear algebraic and very well understood (say 50+ years of theory built around them). They are used for very different things (PCA for dimensionality reduction, LDA for…
carlosdc
  • 3,145
  • 20
  • 25
23
votes
2 answers

The limit of "unit-variance" ridge regression estimator when $\lambda\to\infty$

Consider ridge regression with an additional constraint requiring that $\hat{\mathbf y}$ has unit sum of squares (equivalently, unit variance); if needed, one can assume that $\mathbf y$ has unit sum of squares as…
amoeba
  • 93,463
  • 28
  • 275
  • 317
20
votes
1 answer

What is the connection between partial least squares, reduced rank regression, and principal component regression?

Are reduced rank regression and principal component regression just special cases of partial least squares? This tutorial (Page 6, "Comparison of Objectives") states that when we do partial least squares without projecting X or Y (i.e., "not…
Minkov
  • 415
  • 3
  • 7
17
votes
2 answers

What is the "partial" in partial least squares methods?

In partial least squares regression (PLSR) or partial least squares structural equation modelling (PLS-SEM), what does the term "partial" refer to?
Alph
  • 515
  • 1
  • 7
  • 14
17
votes
1 answer

Regression in $p>n$ setting: how to choose regularization method (Lasso, PLS, PCR, ridge)?

I am trying see whether to go for ridge regression, LASSO, principal component regression (PCR), or Partial Least Squares (PLS) in a situation where there are large number of variables / features ($p$) and smaller number of samples ($n
Ram Sharma
  • 2,226
  • 3
  • 20
  • 24
14
votes
3 answers

Model assumptions of partial least squares (PLS) regression

I am trying to find information regarding the assumptions of PLS regression (single $y$). I am especially interested in a comparison of the assumptions of PLS with regards to those of OLS regression. I have read/skimmed through a great deal of…
Patrick
  • 1,381
  • 1
  • 15
  • 21
13
votes
1 answer

Partial least squares regression in R: why is PLS on standardized data not equivalent to maximizing correlation?

I am very new in partial least squares (PLS) and I try to understand the output of the R function plsr() in the pls package. Let us simulate data and run the PLS: library(pls) n <- 50 x1 <- rnorm(n); xx1 <- scale(x1) x2 <- rnorm(n); xx2 <-…
Stéphane Laurent
  • 17,425
  • 5
  • 59
  • 101
12
votes
1 answer

Difference between PLS regression and PLS path modeling. Criticism of PLS

This question was asked here but no one gave a good answer. So I think it's a good idea to bring it up again and also I would like to add some more comments/questions. The first question is what is the difference between "PLS path modeling" and…
Ress
  • 387
  • 2
  • 11
12
votes
1 answer

Why do all the PLS components together explain only a part of the variance of the original data?

I have a dataset consisting of 10 variables. I ran partial least squares (PLS) to predict a single response variable by these 10 variables, extracted 10 PLS components, and then computed the variance of each component. On the original data I took…
Ress
  • 387
  • 2
  • 11
12
votes
1 answer

What is the difference between "loadings" and "correlation loadings" in PCA and PLS?

One common thing to do when doing Principal Component Analysis (PCA) is to plot two loadings against each other to investigate the relationships between the variables. In the paper accompanying the PLS R package for doing Principal Component…
user1593755
  • 195
  • 1
  • 1
  • 7
10
votes
2 answers

How to compute the confidence intervals on regression coefficients in PLS?

The underlying model of PLS is that a given $n \times m$ matrix $X$ and $n$ vector $y$ are related by $$X = T P' + E,$$ $$y = T q' + f,$$ where $T$ is a latent $n \times k$ matrix, and $E, f$ are noise terms (sssuming $X, y$ are centered). PLS…
shabbychef
  • 10,388
  • 7
  • 50
  • 93
9
votes
1 answer

PLS (partial least squares) weights, loadings, and scores interpretations

In SKLearn PLSRegression, several items can be called after a model is trained: Loadings Scores Weights All the above are separated by X and Y I intuitively understand that x_scores and y_scores should have a linear relationship because that's…
Heavy Breathing
  • 431
  • 5
  • 8
8
votes
1 answer

Validation metrics (R2 and Q2) for Partial Least Squares (PLS) Regression

I'm attempting to validate my Partial Least Squares (PLS) -regression model. From documentation and other readings regarding PLS regression I've come to understand that there are generally two metrics used to evaluate the performance of the…
Tkanno
  • 183
  • 1
  • 1
  • 7
8
votes
1 answer

Probabilistic models for partial least squares, reduced rank regression, and canonical correlation analysis?

This question results from the discussion following a previous question: What is the connection between partial least squares, reduced rank regression, and principal component regression? For principal component analysis, a commonly used…
Minkov
  • 415
  • 3
  • 7
1
2 3
…
16 17