keywords: information theory

Keywords | Authors | Categories | Journals

#-u | v-#

#-u	ICA (4)	homepage & quantum ... (1)	networks & graphs (2)
	neural networks (2)	neuroscience (1)	perception (3)
	statistics (8)

v-#	vision (3)

Bell, A.J., Sejnowski, T.J. The 'independent components' of natural scenes are edge filters Vision Research 1997 37:3327-3338 [html]

Field (1994) has suggested that neurons with line and edge selectivities found in primary visual cortex of cats and monkeys form a sparse, distributed representaton of natural scenes, and Barlow (1989) has reasoned that such responses should emerge from an unsupervised learning algorithm that attempts to find a factorial code of independent visual features. We show here that a new unsupervised learning algorithm that is based on information maximisation, a non-linear `infomax' network (Bell and Sejnowski, 1995) when applied to an ensemble of natural scenes, produces sets of visual filters that are localised and oriented. Some of these filters are Gabor-like and resemble those produced by the sparseness-maximisation network of Olshausen & Field (1996). In addition, the outputs of these filters are as independent as possible, since the infomax network is able to perform Independent Components Analysis (ICA). We compare the resulting ICA filters and their associated basis functions, with ...

cross-entries	information theory, Sejnowski, Terrence J.
web search	Google Scholar, PubMed, Google

17 elements

Bell, A.J., Sejnowski, T.J. An information maximisation approach to blind separation and blind deconvolution Neural Computation 1995 (7)6:1129-1159

Ziv, E., Middendorf, M., Wiggins, C. An information-theoretic approach to network modularity Econophysics Forum 2004

Watts, D.J., Strogatz, S.H. Collective dynamics of "small-world" networks Nature 1998 (393):440-442

Barlow, H.B. Current Problems in Animal Behaviour 1960 :331-360

Cover, T.M., Thomas, J.A. Elements of Information Theory 1991

Olshausen, B., Field, D. Emergence of simple-cell receptive field properties by learning a sparse code for natural images Nature 1996 (381):607-609

Atick, J.J. Entropy minimization: A design principle for sensory perception? International Journal of Neural Systems 1992 3:81-90

Cardoso, J.-F. Infomax and maximum likelihood for source separation IEEE Letters on Signal Processing 1997 (4)4:112-114

Crutchfield, J.P. Information and its metric Nonlinear Structures in Physical Systems - Pattern formation and Chaos and Waves 1990 :119-130

Attneave, F. Informational aspects of visual perception Psychological Review 1954 (61):183-193

Brunel, N., Nadal, J.-P. Mutual information, Fisher information and population coding Neural Computation 1998 (10)7:1731-1757

Nadal, J.-P. , Parga, N. Non linear neurons in the low noise limit: a factorial code maximizes information transfer Network 1994

Linsker, R. Self-organization in a perceptual network IEEE Computer 1988 (21):105-117

Bell, A.J., Sejnowski, T.J. The 'independent components' of natural scenes are edge filters Vision Research 1997 37:3327-3338

Csiszar, I. The Method of Types EEETIT: IEEE Transactions on Information Theory 1998 (44)6:2505-2523

Koch, C., Segev, I. The role of single neurons in information processing Nature Neuroscience 2000 (3):1171-1177

Nielsen, M.A. homepage

last computed Thu Dec 16 21:02:16 GMT+01:00 2004