Partial least squares (PLS) regression is a statistical method that bears some relation to principal components regression; instead of finding hyperplanes of maximum variance between the response and independent variables, it finds a linear regression model by projecting the predicted variables and the observable variables to a new space. Because both the X and Y data are projected to new spaces, the PLS family of methods are known as bilinear factor models. Partial least squares discriminant analysis (PLS-DA) is a variant used when the Y is categorical.

PLS is used to find the fundamental relations between two matrices (X and Y), i.e. a latent variable approach to modeling the covariance structures in these two spaces. A PLS model will try to find the multidimensional direction in the X space that explains the maximum multidimensional variance direction in the Y space. PLS regression is particularly suited when the matrix of predictors has more variables than observations, and when there is multicollinearity among X values. By contrast, standard regression will fail in these cases (unless it is regularized).

Partial least squares was introduced by the Swedish statistician Herman O. A. Wold, who then developed it with his son, Svante Wold. An alternative term for PLS is projection to latent structures,^[1][2] but the term partial least squares is still dominant in many areas. Although the original applications were in the social sciences, PLS regression is today most widely used in chemometrics and related areas. It is also used in bioinformatics, sensometrics, neuroscience, and anthropology.

Core idea

We are given a sample of ${\displaystyle n}$ paired observations ${\displaystyle ({\vec {x}}_{i},{\vec {y}}_{i}),i\in {1,\ldots ,n}}$. In the first step ${\displaystyle j=1}$, the partial least squares regression searches for the normalized direction ${\displaystyle {\vec {p}}_{j}}$, ${\displaystyle {\vec {q}}_{j}}$ that maximizes the covariance^[3]

${\displaystyle \max _{{\vec {p}}_{j},{\vec {q}}_{j}}\operatorname {E} .}$

Note below, the algorithm is denoted in matrix notation.

Underlying model

The general underlying model of multivariate PLS with ${\displaystyle l}$ components is

${\displaystyle X=TP^{\mathrm {T} }+E}$

${\displaystyle Y=UQ^{\mathrm {T} }+F}$

where

• X is an ${\displaystyle n\times m}$ matrix of predictors
• Y is an ${\displaystyle n\times p}$ matrix of responses
• T and U are ${\displaystyle n\times \ell }$ matrices that are, respectively, projections of X (the X score, component or factor matrix) and projections of Y (the Y scores)
• P and Q are, respectively, ${\displaystyle m\times \ell }$ and ${\displaystyle p\times \ell }$ loading matrices
• and matrices E and F are the error terms, assumed to be independent and identically distributed random normal variables.

The decompositions of X and Y are made so as to maximise the covariance between T and U.

Note that this covariance is defined pair by pair: the covariance of column i of T (length n) with the column i of U (length n) is maximized. Additionally, the covariance of the column i of T with the column j of U (with ${\displaystyle i\neq j}$) is zero.

In PLSR, the loadings are thus chosen so that the scores form an orthogonal basis. This is a major difference with PCA where orthogonality is imposed onto loadings (and not the scores).

Algorithms

A number of variants of PLS exist for estimating the factor and loading matrices T, U, P and Q. Most of them construct estimates of the linear regression between X and Y as ${\displaystyle Y=X{\tilde {B}}+{\tilde {B}}_{0}}$. Some PLS algorithms are only appropriate for the case where Y is a column vector, while others deal with the general case of a matrix Y. Algorithms also differ on whether they estimate the factor matrix T as an orthogonal (that is, orthonormal) matrix or not.^{[4][5][6][7][8][9]} The final prediction will be the same for all these varieties of PLS, but the components will differ.

PLS is composed of iteratively repeating the following steps k times (for k components):

1. finding the directions of maximal covariance in input and output space
2. performing least squares regression on the input score
3. deflating the input ${\displaystyle X}$ and/or target ${\displaystyle Y}$

PLS1

PLS1 is a widely used algorithm appropriate for the vector Y case. It estimates T as an orthonormal matrix. (Caution: the t vectors in the code below may not be normalized appropriately; see talk.) In pseudocode it is expressed below (capital letters are matrices, lower case letters are vectors if they are superscripted and scalars if they are subscripted).

 1 function PLS1(X, y, ℓ)
 2     ${\displaystyle X^{(0)}\gets X}$
 3     ${\displaystyle w^{(0)}\gets X^{\mathrm {T} }y/\|X^{\mathrm {T} }y\|}$, an initial estimate of w.
 4     for ${\displaystyle k=0}$ to ${\displaystyle \ell -1}$
 5         ${\displaystyle }$Zdroj:https://en.wikipedia.org?pojem=Partial_least_squares_regression
Text je dostupný za podmienok Creative Commons Attribution/Share-Alike License 3.0 Unported; prípadne za ďalších podmienok. Podrobnejšie informácie nájdete na stránke Podmienky použitia.