Estimating Invariant Principal Components Using Diagonal Regression

Abstract

In this work we apply the method of diagonal regression to derive an alternative version of Principal Component Analysis (PCA). Diagonal regression was introduced by Ragnar Frisch (the first economics Nobel laureate) in his paper Correlation and Scatter in Statistical Variables (1928). The benefits of using diagonal regression in PCA are that it provides components that are scale-invariant (i.e. changing the units of measurement leads to an equivalent result), and which reflect both the correlation structure of the data set, and the variance structure as well. By contrast PCA based on the correlation matrix will only reflect the correlation structure of the data. The problem is formulated as a generalized eigen-analysis and is demonstrated using a numerical example which highlights some desirable properties of what we call Invariant Principal Components Analysis (IPCA).