The sensorimotor approach to color
The
laws of manipulating color
To more fully understand the laws
that govern how we "visually manipulate" color, consider first how
physicists measure surface color
Physicists define what they call
the reflectance function of a surface[1].
A reflectance function is a function that predicts, for any incoming light,
what the reflected light from the surface will be. A reflectance function can
be represented as a graph which indicates, for every individual wavelength of
visible light, to what extent the surface reflects that wavelength. For
example, this Figure shows an incoming light spectrum (on the left) being
transformed into a reflected spectrum on the right. The function that defines
this transformation is the graph in the middle, which indicates for each
wavelength what proportion of the incident light energy is reflected. The
example used is the reflectance function for a green-colored surface from the
so-called Munsell collection of "color chips" used in color tests.
Now, defining reflectance in terms
of each wavelength of visible light is fine for physicists who have
spectrometers that can measure light energy at every individual wavelength. But
humans don't have finely tuned spectrometers in their eyes, they only have photoreceptors, which are much more roughly tuned, being sensitive
to very wide ranges of wavelengths.
Therefore as far as human vision
is concerned we need a way of describing reflectance in terms of how it
modifies the responses of the three photoreceptor types that humans possess,
instead of at every individual wavelength of light.
In my laboratory we have done an
analysis of how this can be done[2].
It turns out that instead of using the physicistsÕ reflectance function, from
the biological viewpoint of our human photoreceptors, the chromatic
"behavior" of any surface can be accurately characterized by nine
numbers. The nine numbers define a 3x3 matrix R that allows you to calculate,
for any light source, how the surface will affect the responses of the three
human photoreceptor types. Suppose for example we use a light source whose
absorption by the three types of human photoreceptors is given by the three
numbers L, M, S. Then when the light is reflected off the surface, what will
impinge on the photoreceptors will be light whose absorption will be given by
the three numbers l,m,s. The matrix R allows us to calculate (l,m,s) from
(L,M,S) using matrix multiplication: (l, m, s)Õ = R (L, M, S)Õ
Once we have the matrix R for a
given surface, we will be able to tell how that surface will modify the eyeÕs
reaction to that surface, in all situations, for any incoming light.
How do we get the matrix R for a surface?
Calculating the nine numbers for a surface is a simple mathematical exercise if
you have the surface's reflectance function, as measured by physicists, and if
you know the human photoreceptor absorption functions. In my laboratory we have
calculated the matrices for a subset of the surfaces used in the Munsell set of
color chips, whose reflectance functions can be found in public databases. Some
examples are shown in the figure.
(NB the brackets are missing
from the middle matrix. Also, exact values to be confirmed...)
So what?
If you're not a mathematician these numbers won't mean very much to you. But if you're a mathematician, you'll notice something very interesting. Some of the matrices are what's called "singular". This means that instead of behaving normally, these matrices have a special behavior. Normally a 3 x 3 matrix R takes a vector in the three-dimensional space of L,M, S values into the three-dimensional space of l,m,s values. But if R is a singular matrix, it takes a vector in the three-dimensional space of L,M,S values into a two-dimensional or a one-dimensional subspace of the three dimensional space of l,m,s values.
Said in another way: the behavior
of some surfaces is simpler than normal.
These simpler surfaces transform incoming light in a more restricted way:
whatever the variations in the incoming light, the outgoing light is restricted
to a two-dimensional subspace (a plane) or a one-dimensional subspace (a line)
in what would usually be a three-dimensional space of possible l,m,s values.
We can define a "singularity
index" which describes the degree to which a given matrix R is singular in
this way[3].
The Figure below plots this singularity index for our subset of the Munsell
color chips.
And now we see something
surprising. There are two chips which are strongly singular, the red and the
yellow chips. There are a few more chips which are less strongly singular: the
blue, green, magenta and cyan chips.
And now for a little interlude....
NB final figure
will be more like the next figure, below, so the two can be compared.
Focal
colors
In the mid 1960's, Brent Berlin and Paul
Kay, two anthropologists at the University of California at Berkeley
sent out their graduate students on a curious quest. They scoured the San
Francisco Bay area for native speakers of exotic foreign languages, of which
they found 27. They showed speakers of each language a palette of about 300
colored surfaces, and asked them to mark the colors which were
"focal", in the sense that people considered them to be the best
examples of the most basic[4]
color terms in their language. To complement that study, Berlin and Kay
subsequently contacted missionaries and other collaborators in foreign
countries throughout the world, and established what is now called the "World
Color Survey"[5]. The survey
compiles data about color names in a total of 110 languages, ranging from
Abidjii in the ivory Coast through Ifugao in the Philippines to Zapoteco in
Mexico.
The figure below plots the result of their study. For each
color patch in the palette of colors they used, is shown the number of people
in the World Color Survey who considered that color to be "focal".
What is surprising is that the graph has essentially four
clear peaks.
Red, chip number G1 is clearly an all-out favorite, showing
that if a culture has a word for red, then the particular shade of red that
best represents the meaning of the word "red" will be G1, whatever
the culture! There is more variability for
the other colors, but it is clear that certain colors always win out as
compared to others. Focal yellow tends always to be patch C9, Focal green is
F17 and focal blue is F28 or H28.
Why exactly those patches?
An
interesting coincidence
But now let us compare the singularities in color we calculated
in the earlier sections with Berlin and Kay's data. The Figure below
superimposes the two graphs, plotting the focal colors of Berlin and Kay's
survey as the flat red, yellow, green and blue patches, and the singularity
index as the contour plot.
The agreement is quite surprising. Our calculated
singularities in surface color are within a few chips of the Berlin and Kay
data.
Perhaps you are not accustomed to looking at results from
experiments in the social sciences. But let me assure you that it is very rare
that data acquired by psychologists or anthropologists can be so accurately
accounted for. Humans are inherently variable, and experimental methods are
subject to all sorts of biases. Agreement between theoretical predictions and
results from experiments is rarely as good as what we see in this Figure.
It seems therefore that the sensorimotor approach to color
here registers surprising successes. By taking a philosophical stance about
what color really is (a law governing our interactions with colored surfaces),
we have been able to account for classical phenomena in color science that have
until now not been satisfactorily explained.
Of course more work needs to be done. Berlin & Kay's
data is considered controversial: the procedures used to question participants
may have caused biases. The experimenters were missionaries and other helpers
not trained in psychophysics. But still, the fact that from first principles
and simple mathematics we have found such a surprising coincidence surely is
worth noting.
Another point is that we would like to have a learning
mechanism that explains why singular
surface colors should be expected to acquire names, whereas other colors would
not. A start in this direction would be to suggest that the simpler behavior of
singular surfaces under changing lights makes them stand out from other colors.
But even though we don't yet have a mechanism, the coincidence between naming
and singularity, and the success of our predictions for unique hues are so
compelling that this suggests that there is something right about the
sensorimotor approach.
What
red is like
How has the sensorimotor approach gone further in explaining
the raw feel of color than approaches based on the "neural correlate"
approach?
The neural correlate approach postulates that the feel of,
say, red is generated by some neurophysiological mechanism. For example it
attempts to explain why red is different from green by saying that the red
sensation and green sensation are generated by different groups of neurons or
neural mechanisms.
Neurophysiological constraints do of course determine what
kinds of light we can and cannot see. For example the simple fact that we have
no photoreceptors sensitive to the ultraviolet part of the spectrum means that
we cannot see patterns that birds and bees see, like ultraviolet sunlight that
gets through clouds and that they use to orient themselves. Another important
constraint is imposed by the fact that our photoreceptors are very broadly
tuned. This means that a variety of lights that birds and bees distinguish will
all look identical to us[6].
Further constraints may be imposed by the wiring of the color system if it
introduces transformations in which information about the incoming light is
lost.
But as I have argued in the book, neural mechanisms will
never be able to explain why particular neural activations induce the
sensations that they do. Even if the known red/green and blue/yellow channels
turned out to coincide with data on unique hues (and they do not in fact!), one
would still have to explain why activation in the red/green channel provides a
different feel from activation in the blue/yellow channel, and why activation
of the red/green channel in the red direction provokes a different feel from
activation in the green direction.
Further questions that would be left open by a neural
correlate approach are: Why does color have a feel rather than no feel at all?
Why is the feel of a visual nature rather than being like, say, touch or
hearing or smell?[7]
If we take the neurophysiological approach, we at some point
always must provide some explanation of what it is exactly about each
particular neural firing pattern which gives rise to the particular feel that
it gives rise to. There would seem to be no way of making the link between feel
and neural activity in any principled way.
On the other hand under the sensorimotor approach, although
we cannot hope to describe every single aspect of the laws that govern our
interactions with the environment, we know that in principle every such aspect,
every peculiarity, every particular difference between different feels, will
potentially be constituted by a
corresponding aspect of our mode of interaction.
So whereas under the neurophysiological explanation we have
no hope of finding the link between feel and neural activity, under the
sensorimotor explanation, we at least in principle have a way of understanding all the aspects of a feel
even if we may not have access to them in detail.