Partial least squares fusing unsupervised learning

In this paper, partial least squares to fuse unsupervised learning, called fused clustered least squares (FCLS), is proposed. As an unsupervised method, the K-means clustering algorithm is adopted, and it clusters either the original predictors or its principal components. This unsupervised learning procedure has a function to discover unknown structures of the predictors, and this information is utilized in their further reduction. Within each cluster, the covariance of the response and the predictors is computed and successively projected onto the covariance matrix of the predictors. This is called clustered least squares. Then we fuse all clustered least squares from the various numbers of clusters. The FCLS is basically implemented by combining supervised and unsupervised statistical methods, and it overcomes the deficits that the ordinary least squares, including its popular variation of partial least squares, have in practice. Numerical studies support the theory, and its application to near infrared spectroscopy data confirms the potential advantage of FCLS in practice.