| Lee, J.A., Lendasse, A., Donckers, N., Verleysen, M. A robust nonlinear projection method 2000 :13-20 [html] |
| This paper describes a new nonlinear projection method. The aim is to design a user-friendly method, tentativ ely as easy to use as the linear PCA (Principal Component Analysis). The method is based on CCA (Curvilinear Component Analysis). This paper presen ts tw o improvements with respect to the original CCA: a better beha vior in the projection of highly nonlinear databases (like spirals) and a complete automation in the choice of the parameters value. |
| |
|
|
|
| Edelman, S., Intrator, N. Learning as extraction of low-dimensional representations 1997 [html] |
| Psychophysical findings accumulated over the past several decades indicate that perceptual tasks such as similarity judgment tend to be performed on a low-dimensional representation of the sensory data. Low dimensionality is especially important for learning, as the number of examples required for attaining a given level of performance grows exponentially with the dimensionality of the underlying representation space. In this chapter, we argue that, whereas many perceptual problems are tractable precisely because their intrinsic dimensionality is low, the raw dimensionality of the sensory data is normally high, and must be reduced by a nontrivial computational process, which, in itself, may involve learning. Following a survey of computational techniques for dimensionality reduction, we show that it is possible to learn a low-dimensional representation that captures the intrinsic low-dimensional nature of certain classes of visual objects, thereby facilitating further learning of tasks... |
| |
|
|
| Tenenbaum, J.B. Mapping a Manifold of Perceptual Observations 1998 (NIPS-10) [pdf] |
| Nonlinear dimensionality reduction is formulated here as the problem of trying to find a Euclidean feature-space embedding of a set of observations that preserves as closely as possible their intrinsic metric structure the distances between points on the observation manifold as measured along geodesic paths. Our isometric feature mapping procedure, or isomap, is able to reliably recover low-dimensional nonlinear structure in realistic perceptual data sets, such as a manifold of face images, where conventional global mapping methods find only local minima. The recovered map provides a canonical set of globally meaningful features, which allows perceptual transformations such as interpolation, extrapolation, and analogy highly nonlinear transformations in the original observation space to be computed with simple linear operations in feature space. |
| |
|
|
| Tenenbaum, J.B., de Silva, V., Langford, J.C. Nonlinear Dimensionality Reduction by Locally Linear Embedding Science 2000 (290):2323-2326 [pdf] |
| Many areas of science depend on exploratory data analysis and visualization. The need to analyze large amounts of multivariate data raises the fundamental problem of dimensionality reduction: how to discover compact representations of high-dimensional data. Here, we introduce locally linear embedding (LLE), an unsupervised learning algorithm that computes low-dimensional, neighborhood- preserving embeddings of high-dimensional inputs. Unlike clustering methods for local dimensionality reduction, LLE maps its inputs into a single global coordinate system of lower dimensionality, and its optimizations do not involve local minima. By exploiting the local symmetries of linear reconstructions, LLE is able to learn the global structure of nonlinear manifolds, such as those generated by images of faces or documents of text. |
| |
|
|
| DiCarlo, J.M., Wandell, B.A. Spectral estimation theory: beyond linear but before Bayesian Journal of the Optical Society of America A 2003 (20)7:1261-1270 [pdf] |
| Most color-acquisition devices capture spectral signals by acquiring only three samples, critically undersampling the spectral information. We analyze the problem of estimating high-dimensional spectral signals from low-dimensional device responses. We begin with the theory and geometry of linear estimation methods. These methods use linear models to characterize the likely input signals and reduce the number of estimation parameters. Next, we introduce two submanifold estimation methods. These methods are based on the observation that for many data sets the deviation between the signal and the linear estimate is systematic; the methods incorporate knowledge of these systematic deviations to improve upon linear estimation methods. We describe the geometric intuition of these methods and evaluate the submanifold method on hyperspectral image data. |
| |
|
|