Dimensionality reduction using deep autoencoder

Abstract

All the traditional techniques originating from statistics and geometry theory for dimensionality reduction learns underlying manifold by preserving relationships in low-dimensional feature space. In order to use these linear or non-linear techniques to uncover manifold structure, one needs to know whether data lies near linear subspace or non-linear sub-manifold. Autoencoder networks are capable of finding such non-linear manifolds by learning non-linear functions of input data that can best reconstruct data at the output layer. But autoencoder still ignores to explicitly learn data relation. In recent literature, a new model was proposed by modifying the traditional learning process to consider data relation. In this thesis, first we have compared traditional autoencoder with the most widely used technique - Principle Component Analysis(PCA) in terms of classification and reconstruction results. After that, we have used the modified autoencoder model and proposed changes in the similarity function for two conventional techniques. To evaluate the performance of proposed changes we have performed extensive experiments on two handwritten digit data sets. The results show that the proposed changes achieve promising performance.