| Bell, A.J., Sejnowski, T.J. An information maximisation approach to blind separation and blind deconvolution Neural Computation 1995 (7)6:1129-1159 [pdf] |
| We derive a new self-organising learning algorithm which maximises the information transferred in a network of non-linear units. The algo- rithm does not assume any knowledge of the input distributions, and is de ned here for the zero-noise limit. Under these conditions, infor- mation maximisation has extra properties not found in the linear case (Linsker 1989). The non-linearities in the transfer function are able to pick up higher-order moments of the input distributions and perform something akin to true redundancy reduction between units in the out- put representation. This enables the network to separate statistically independent components in the inputs: a higher-order generalisation of Principal Components Analysis. We apply the network to the source separation (or cocktail party) problem, successfully separating unknown mixtures of up to ten speak- ers. We also show that a variant on the network architecture is able to perform blind deconvolution (cancellation of unknown echoes and reverberation in a speech signal). Finally, we derive dependencies of information transfer on time delays. We suggest that information max- imisation provides a unifying framework for problems in `blind' signal processing. |
| |
|
|
| Ben-Shahar, O., Zucker, S.W. Geometrical Computations Explain Projection Patterns of Long Range Horizontal Connections in Visual Cortex Neural Computation 2004 (16)3:445-476, [pdf] |
| Neurons in primary visual cortex respond selectively to oriented stimuli such as edges and lines. The long-range horizontal connections between them are thought to facilitate contour integration. While many physiological and psychophysical findings suggest that collinear or association field models of good continuation dictate particular projection patterns of horizontal connections to guide this integration process, significant evidence of interactions inconsistent with these hypotheses is accumulating. We first show that natural random variations around the collinear and association field models cannot account for these inconsistencies, a fact that motivates the search for more principled explanations.We then develop a model of long-range projection fields that formalizes good continuation based on differential geometry. The analysis implicates curvature(s) in a fundamental way, and the resulting model explains both consistent data and apparent outliers. It quantitatively predicts the (typically ignored) spread in projection distribution, its nonmonotonic variance, and the differences found among individual neurons. Surprisingly, and for the first time, this model also indicates that texture (and shading) continuation can serve as alternative and complementary functional explanations to contour integration. Because current anatomical data support both (curve and texture) integration models equally and because both are important computationally, new testable predictions are derived to allow their differentiation and identification. |
| |
|
|
| Franz, M.O., Chahl, J.S., Krapp, H.G. Insect-inspired estimation of egomotion Neural Computation 2004 (6)11: [pdf] |
| Tangential neurons in the fly brain are sensitive to the typical optic flow patterns generated during egomotion. In this study, we examine whether a simplified linear model based on the organization principles in tangential neurons can be used to estimate egomotion from the optic flow. We present a theory for the construction of an estimator consisting of a linear combination of optic flow vectors that incorporates prior knowledge about the distance distribution of the environment and about the noise and egomotion statistics of the sensor. The estimator is tested on a gantry carrying an omnidirectional vision sensor. The experiments show that the proposed approach leads to accurate and robust estimates of rotation rates, whereas translation estimates are of reasonable quality, albeit less reliable. |
| |
|
|
| Philipona, D., O'Regan, J.K., Nadal, J.-P. Is there something out there ? Inferring space from sensorimotor dependencies Neural Computation 2003 (15)9 [pdf] |
| This letter suggests that in biological organisms, the perceived structure of reality, in particular the notions of body, environment, space, object, and attribute, could be a consequence of an effort on the part of brains to account for the dependency between their inputs and their outputs in terms of a smallnumberof parameters.To validate this idea, a procedure is demonstrated whereby the brain of a (simulated) organism with arbitrary input and output connectivity can deduce the dimensionality of the rigid group of the space underlying its input-output relationship, that is, the dimension of what the organism will call physical space. |
| |
|
|
| Brunel, N., Nadal, J.-P. Mutual information, Fisher information and population coding Neural Computation 1998 (10)7:1731-1757 [html] |
| In the context of parameter estimation and model selection, it is only quite recently that a direct linkbetween the Fisher informationandinformationtheoretic quantities has been exhibited.We give an interpretation of this link within the standard framework of information theory.We show that in the context of population coding, the mutual information between the activity of a large array of neurons and a stimulus to which the neurons are tuned is naturally related to the Fisher information. In the light of this result, we consider the optimization of the tuning curves parameters in the case of neurons responding to a stimulus represented by an angular variable. |
| |
|
|
| Amari, S.-I. Natural Gradient Learning for Over- and Under-Complete Bases in ICA Neural Computation 1999 [pdf] |
| Independent component analysis or blind source separation is a new technique of extracting independent signals from mixtures. It is applicable even when the number of independent sources is unknown and is larger or smaller than the number of observed mixture signals. This article extends the natural gradient learning algorithm to be applicable to these overcomplete and undercomplete cases. Here, the observed signals are assumed to be whitened by preprocessing, so that we use the natural Riemannian gradient in Stiefel manifolds. |
| |
|
|
| Ahissar, E. Temporal-to Rate-Code Conversion by Neuronal Phase-Locked Loops Neural Computation 1998 (10):597-650 [pdf] |
| Peripheral sensory activity follows the temporal structure of input signals. Central sensory processing uses also rate coding, and motor outputs appear to be primarily encoded by rate. I propose here a simple, efficient structure, converting temporal coding to rate coding by neuronal phase-locked loops (PLL). The simplest form of a PLL includes a phase detector (that is, a neuronal-plausible version of an ideal coincidence detector) and a controllable local oscillator that are connected in a negative feedback loop. The phase detector compares the firing times of the local oscillator and the input and provides an output whose firing rate is monotonically related to the time difference. The output rate is fed back to the local oscillator and forces it to phase-lock to the input. Every temporal interval at the input is associated with a specific pair of output rate and time difference values; the higher the output rate, the further the local oscillator is driven from its intrinsic frequency. Sequences of input intervals, which by definition encode input information, are thus represented by sequences of firing rates at the PLL's output. The most plausible implementation of PLL circuits is by thalamocortical loops in which populations of thalamic "relay" neurons function as phase detectors that compare the timings of cortical oscillators and sensory signals. The output in this case is encoded by the thalamic population rate. This article presents and analyzes the algorithmic and the implementation levels of the proposed PLL model and describes the implementation of the PLL model to the primate tactile system. |
| |
|
|
| Li, Z., Atick, J.J. Towards a theory of the striate cortex Neural Computation 1994 (6)1:127-146 [pdf] |
| We explore the hypothesis that linear cortical neurons are concerned with building a particular type of representation of the visual world --- one which not only preserves the information and the efficiency achieved by the retina, but in addition preserves spatial relationships in the input --- both in the plane of vision and in the depth dimension. Focusing on the {\it linear} cortical cells, we classify all transforms having these properties. They are given by representations of the scaling and translation group, and turn out to be labeled by rational numbers `$(p+q)/p$' ($p, q$ integers). Any given $(p,q)$ predicts a set of receptive fields which come at different spatial locations and scales (sizes) with a bandwidth of $\log_2[(p+q)/p]$ octaves, and, most interestingly, with a diversity of `$q$' cell varieties. The bandwidth affects the trade-off between preservation of planar and depth relations, and, we think, should be selected to match structures in natural scenes. For bandwidths between $1$ and $2$ octaves, which are the ones we feel provide the best matching, we find for each scale a minimum of two distinct cell types that reside next to each other and in phase quadrature, i.e., differ by $90^o$ in the phases of their receptive fields, as are found in the cortex, they resemble the ``even-symmetric'' and ``odd-symmetric'' simple cells in special cases. An interesting consequence of the representations presented here is that the pattern of activation in the cells in response to a translation or scaling of an object remains the same but merely shifts its locus from one group of cells to another. This work also provides a new understanding of color coding changes from the retina to the cortex. |
| |
|
|
| Bell, A.J., Sejnowski, T.J. An information maximisation approach to blind separation and blind deconvolution Neural Computation 1995 (7)6:1129-1159 |
| Ben-Shahar, O., Zucker, S.W. Geometrical Computations Explain Projection Patterns of Long Range Horizontal Connections in Visual Cortex Neural Computation 2004 (16)3:445-476, |
| Franz, M.O., Chahl, J.S., Krapp, H.G. Insect-inspired estimation of egomotion Neural Computation 2004 (6)11: |
| Philipona, D., O'Regan, J.K., Nadal, J.-P. Is there something out there ? Inferring space from sensorimotor dependencies Neural Computation 2003 (15)9 |
| Brunel, N., Nadal, J.-P. Mutual information, Fisher information and population coding Neural Computation 1998 (10)7:1731-1757 |
| Amari, S.-I. Natural Gradient Learning for Over- and Under-Complete Bases in ICA Neural Computation 1999 |
| Ahissar, E. Temporal-to Rate-Code Conversion by Neuronal Phase-Locked Loops Neural Computation 1998 (10):597-650 |
| Li, Z., Atick, J.J. Towards a theory of the striate cortex Neural Computation 1994 (6)1:127-146 |