| Bell, A.J., Sejnowski, T.J. An information maximisation approach to blind separation and blind deconvolution Neural Computation 1995 (7)6:1129-1159 [pdf] |
| We derive a new self-organising learning algorithm which maximises the information transferred in a network of non-linear units. The algo- rithm does not assume any knowledge of the input distributions, and is de ned here for the zero-noise limit. Under these conditions, infor- mation maximisation has extra properties not found in the linear case (Linsker 1989). The non-linearities in the transfer function are able to pick up higher-order moments of the input distributions and perform something akin to true redundancy reduction between units in the out- put representation. This enables the network to separate statistically independent components in the inputs: a higher-order generalisation of Principal Components Analysis. We apply the network to the source separation (or cocktail party) problem, successfully separating unknown mixtures of up to ten speak- ers. We also show that a variant on the network architecture is able to perform blind deconvolution (cancellation of unknown echoes and reverberation in a speech signal). Finally, we derive dependencies of information transfer on time delays. We suggest that information max- imisation provides a unifying framework for problems in `blind' signal processing. |
| |
|
|
| Bell, A.J., Sejnowski, T.J. The 'independent components' of natural scenes are edge filters Vision Research 1997 37:3327-3338 [html] |
| Field (1994) has suggested that neurons with line and edge selectivities found in primary visual cortex of cats and monkeys form a sparse, distributed representaton of natural scenes, and Barlow (1989) has reasoned that such responses should emerge from an unsupervised learning algorithm that attempts to find a factorial code of independent visual features. We show here that a new unsupervised learning algorithm that is based on information maximisation, a non-linear `infomax' network (Bell and Sejnowski, 1995) when applied to an ensemble of natural scenes, produces sets of visual filters that are localised and oriented. Some of these filters are Gabor-like and resemble those produced by the sparseness-maximisation network of Olshausen & Field (1996). In addition, the outputs of these filters are as independent as possible, since the infomax network is able to perform Independent Components Analysis (ICA). We compare the resulting ICA filters and their associated basis functions, with ... |
| |
|
|