The Represented Object of Color Experience Elizabeth Schier Department of Philosophy Adelaide University South Australia 5005 AUSTRALIA Phone: (+61 8) 8303 3816 Fax: (+61 8) 8303 5241 elizabeth.schier@adelaide.edu.au Abstract: Despite a wealth of data we still have no clear idea what color experiences represent. In fact, color experiences vary with so many factors that it has been claimed that colour experiences do not represent anything. The primary challenge for any representational account of colour experience is to accommodate the various psychophysical results which demonstrate that the colour experience depends not only on the spectral nature of the target but also on the spectral, spatial and figural nature of the surround. A number of theorists have proposed that this dependence of the colour a region appears to be on the spatial and spectral nature of the surround is an aspect of the visual system’s constancy mechanism. However this does not in and of itself tell us what, if anything, is represented in colour experience. Ultimately the answer to this question will be informed by one’s theory of representational content. I will argue that adopting a molecular scheme of representation enables the development of an account of the represented object of color experience that can do justice to the psychophysical data. Acknowledgements: I would like to thank Jon Opie and Gerard O’Brien for their insightful comments. An earlier draft of this paper was presented at the final McDonnel Project meeting where I received invaluable feedback. This is a preprint of an article whose final and definitive form has been published in Philosophical Psychology 20(1) 1-27 (copyright Taylor and Francis). Philosophical Psychology is available online at http://journalsonline.tandf.co.uk/  The Represented Object of Color Experience Color vision has been studied intensely by scientists and philosophers. Despite a wealth of data we still have no clear idea what color experiences represent. In fact, the color a region appears to be varies with so many factors that it has been claimed that color experiences do not represent anything . The primary challenge for any representational account of color experience is to accommodate the various psychophysical results which demonstrate that color experience depends not only on the spectral nature of a surface but also on the spectral spatial and figural nature of the entire visual scene. A number of theorists have proposed that this dependence of the color a region appears to be on the spatial and spectral nature of the surround is an aspect of the visual system’s constancy mechanism. However this does not in and of itself tell us what, if anything, is represented in color experience. Do color experiences represent the relations between the spectral and spatial aspects of the world or do they represent surface spectral reflectance, partly by using spectral and spatial information? Ultimately the answer to this question will be informed by one’s theory of representational content. I will argue that adopting a molecular scheme of representation provides a new way of thinking about the content of color experiences. In particular it is possible to develop a representational account of color vision that can accommodate the complexities of the psychophysical data. In the first section I will examine the psychophysical data that has created so much trouble for color representationalism. The second section will focus on why it may be the case that spectral, spatial and figural properties are represented together. In the third section I will introduce the distinction between molecular and lexical schemes of representation and show how adopting a molecular scheme of representation provides a new way of thinking about the representational object of color experiences. The aim of the current discussion is to consider what it is that we may be representing in color experience, not to pin-down the location of phenomenal colors. In order to stay neutral between phenomenal internalism and externalism I will call the facts that are known in color experience the 'color facts'. Whether these color facts are also the phenomenal color facts, is a question for another day. See Akins and Akins and Hahn for arguments to the effect that the representational and ontological questions are distinct. 1. The Psychophysical Data The common-sense account of color experience holds that color experiences are about the material properties of objects. When I say that the top of my desk looks brown, it seems that I am making a claim about the nature of the desk itself. This suggests that color experiences represent surface spectral reflectances (SSRs). The surface spectral reflectance is the percentage of incident light that a surface reflects at each wavelength. The fundamental reason why it has proven so difficult to provide an account of the facts that are represented in color experiences is that the color that a region appears to be depends on more than the nature of the surface. Color science has shown us that common-sense is wrong. Color experiences vary with the spectral, spatial and figural nature of the entire visual field. The challenge for the representationalist is to provide an account of what is being represented in color experience that is consistent with the psychophysical data. 1.1 The Spectral Nature of the Surround It has long been known that the spectral nature of the surround plays a crucial role in determining the color that a region appears to be. Perhaps the most persuasive evidence in support of this is the fact that black Although the achromatic shades are not always thought to be “colors” they are part of our color space and so a discussion of them is appropriate in any discussion of color vision. and brown are what is known as contrast colors. What this means is that black and brown can only be experienced when the surface that appears black or brown is surrounded by surface of contrasting color. A surface that appears brown when surrounded by contrasting regions will look orange or yellow when viewed in isolation. Similarly, a surface that appears black when surrounded by contrasting regions will look grey or even luminous when viewed in isolation . The fact that a surface can go from appearing luminous to appearing black or from appearing yellow to appearing brown simply by changing the surrounding context suggests that what we are representing in color experiences is something more than the light being reflected from the surface. The spectral nature of the surround plays an important role in determining the color appearance for all colors, not only black and brown. For instance in simultaneous color contrast the color a surface appears to be is influenced by the spectral nature of the surround. In particular, two surfaces with identical reflectances can appear to be different colors because they are surround by regions with different reflectances. For instance in figure one, the two central squares have been printed with the same ink. But they appear to be different shades of blue because of their different surrounds. If they are being illuminated with a constant source of light, the spectral distribution of the light reaching the retina from both of the central squares should be identical. Yet they appear to be different colors. This suggests that the color appearance depends not only on the nature of the light leaving the surface, but also on the spectral nature of the surround. Figure 1 Simultaneous Color Contrast. The two centre squares have the same reflectance but they appear to be different shades of blue because of the effect of the surround. Because simultaneous color contrast involves representing two identical surfaces as different it is tempting to write it off as an illusion, as an aberration that can be accommodated once we have the correct theory of color vision. This temptation is misleading. First of all, although the central blue regions in figure 1 are the same, their surrounds are different. So it is not the case that the stimuli themselves are identical. In fact, simultaneous color contrast is only an illusion on the assumption that color experiences represent surface spectral reflectances (SSR). If color experiences are about SSRs, then any case where surfaces with the same SSR are represented as different is an instance of misrepresentation. However, whether color experiences represent SSR or something more is precisely what is up for grabs. So to call simultaneous color contrast an illusion, without argument, is to beg the question. More importantly, the existence of the contrast colors provides a reason for thinking that contrast is central to color vision. As Hardin points out to write off simultaneous contrast as something that need not enter into one’s fundamental theory of colors is also to write off the possibility of giving a proper account of the nature of black and brown. This seems unacceptable, unless one is prepared to think of black and brown as “illusory” rather than “real” colors . The fact that the presence of a surrounding region is necessary to have an experience of black or brown suggests that the color experiences are representing more than the reflectance of an individual region. Unless every experience of black and brown is to be an illusion then color experiences must represent something more than the reflectance of the target. In defence of the claim that color experiences represent SSRs Byrne and Hilbert suggest that we need to distinguish between the conditions necessary for the perception of a property and the property that is perceived. They argue that all that can be concluded from the existence of contrast colors is that the presence of a contrasting surround is a necessary condition for the perception of such colors. But they claim this does not mean that an experience of brown actually represents something more than the reflectance of the target region. They draw an analogy with the effect that the nature of the surround can have on spatial properties. For instance consider the Müller-Lyer illusion (see figure 2). In this illusion two lines of equal length appear to have different lengths because of differences in their surround. The analogy with contrast colors is obvious. In both instances objects with identical properties are represented as distinct because of differences in their surround. Byrne and Hilbert suggest that we need to treat color and space equally: Illusions such as the Müller-Lyer illusion do not show that the length of an object is relative to its surround. Rather they show that sometimes an object’s surround can prevent us from seeing the length it actually has …. Similarly, the conditions necessary for the perception of color include facts about the surround, and also facts about mechanisms within the brain of the viewer. But this does not show that color is relative to the surround, or the perceiver, or that no external object has a color Although this distinction seems appropriate, simply drawing the distinction leaves unanswered how the visual system makes use of such information. More particularly it leaves unanswered what precisely is that is being represented in any particular experience. Changing the surround can change how a surface is experienced. This suggests that the visual system is taking into account the spectral nature of the surround when determining how to represent the surface. Similarly, the Müller-Lyer illusion suggests that the visual system takes into account the nature of the surround when representing length. Although the property the system is trying to represent in color experience may be SSR, it may be the case that in order to obtain a more accurate representation of SSR the system needs to take into account more factors than just the material properties of the individual surface. I will have a lot more to say on the mechanisms for achieving constancy below. For now I just want to point out that even if Byrne and Hilbert’s distinction is valid, simply drawing the distinction leaves unanswered how it is the case that the “conditions necessary for perception” are taken into account. Figure 2 The Müller-Lyer Illusion The two horizontal lines are the same length, although they appear to be different lengths because of the difference in positioning of the smaller lines at the end 1.2 Space, Figure and Color It is not only the spectral properties of the surround that affects the color appearance of a surface. The spatial and figural properties of the surfaces are also important. The most pure example of the importance of spatial properties in determining color appearance is the effect of relative size on brightness. In a series of experiments Li and Gilchrist manipulated the size of the target region with respect to the background and explored the effects that this had on the perceived brightness of the target. They found that as the size of the darker region increased both regions appeared lighter until the smaller region appeared luminous and the darker region appeared almost white. This suggests that color appearance depends not only on the spectral properties of the surround but also on the size of relative size of the surround – simply by increasing the size of the target relative to the surround the target can go from appearing black to a very light grey. The spatial complexity and structure of the stimuli also plays an important role. In fact, by increasing the spatial complexity of the stimuli an effect which is opposite to that of the standard color contrast effect can be found (see figure 3). Color assimilation occurs in stimuli that are more complex and iterated than the standard color contrast stimuli. What is interesting about assimilation is that the effect is opposite of that which is found in simultaneous color contrast. In simultaneous color contrast (figure 1) the blue square that is surrounded by the blue region appears less blue than an identical square that is surrounded by yellow. In color assimilation (figure 3) the blue squares that are surrounded by blue appear to be more blue than those that are surrounded by yellow. In general in color contrast the appearance of the target is shifted away from that of the surround whereas in color assimilation the appearance of the target is shifted towards that of the surround. Simply by increasing the spatial complexity of the stimuli the effect of the surround is reversed. This finding suggests that the effect of the surround is mediated by the spatial complexity of the stimulus. Figure 3: Color Assimilation The squares along the diagonal are printed with the same ink, but their appearance to be is shifted towards that of their immediate surround. So the squares on the left that are surrounded by yellow squares appear yellower whereas those on the right that are surrounded by blue squares appear bluer. The perceived three-dimensional layout of the surfaces in the visual field also affects color appearance. In both color contrast and color assimilation it is the nature of the immediate surround that affects the color the target appears to be. For instance the target squares on the right of figure 3 look more blue because of the influence of their immediate blue surround and do not appear to be greatly influenced by the more distant yellow squares. Although the nature of the immediate surround is important there is also evidence that more global comparisons, particularly those which are based on perceptual belongingness are also crucial . For instance Agostini and Galmonte have demonstrated that a region is contrasted with the regions that it is perceived as belonging with rather than its immediate surround. They constructed Necker cubes (figure 4) where the local contrast and figural contrast cues were opposing. We can see that the middle lines of each side of the square consist of dashed lines of an intermediate grey. The corners of the cube are black or white and the background is black or grey such that the background is opposite to the corners. The logic behind this experiment is that if contrast only occurs locally then the central dashed lines should be contrasted with the background. However if contrast is determined by global factors such as the object that the surface is perceived as belonging to then the central dashed lines should be contrasted with the corners of the “cube” they appear to belong to. Because the background and the corners are opposite shades, they will have opposing contrast effects on the central lines. Agostini and Galmonte found that the central lines were contrasted with the corners not the background. Recall that contrast has the effect of moving the appearance of the target away from the appearance of the surround. So the grey that is contrasted with a lighter shade will appear darker than that which is contrasted with a lighter shade. We can see that the lines in the figure on the left appear to be lighter than those on the right (even though they are actually the same shade of grey). This suggests that they are being contrasted not with the lighter background, but rather with the darker corners. In other words, the target grey lines were contrasted with the regions they appear to belong with rather than their immediate surround. For complex scenes the figural properties of the stimulus determine which regions are contrasted with each other and therefore in part determine the color the regions appears to be . Figure 4 Figural Cues and Contrast These Necker Cube Stimuli demonstrate the role figural factors in the perception of achromatic colors. The dashed grey lines are the same shade of grey; however they appear to be different shades because they are contrasted with their surrounds. Importantly, they are contrasted not with the background, which is their immediate surround, but rather with the corners of the cube, that is, with the regions they are perceived as belonging with In general it appears that the color a target appears to be depends on the spectral nature of the surround as well as the spatial and figural properties of the visual field as a whole. Identical surfaces can appear to be different colors because of spectral, spatial or figural differences in their surround. These findings create a number of problems for the claim that color experiences are representational. 1.3 The Challenge for the Representationalist The first difficulty that the psychophysical data raises is that there does not appear to be any simple mapping from a particular color experience, say a red experience, to some objective property of surfaces – “redness”. If you manipulate the conditions then just about any surface can appear red. But if just about anything can appear red then it is not clear that red experiences are actually representing anything. Of course we need to remember that misrepresentation and illusions occur. Sometimes we use a representing vehicle to stand in for something it is not actually about. So in searching for the represented object it would be a mistake to try and include everything that the experience is used to represent because some of these will be instances of misrepresentation. But it is hard to see how to separate the veridical from the non-veridical cases without already having answered the representational question. Two main suggestions for separating the veridical and illusory instances have been made. First Shepard has suggested that we can define normal conditions in terms of natural light . Unfortunately this is problematic. There is enormous variability in the wavelength of light across the day. Worse still there is also evidence that color constancy is worse in naturally occurring lighting conditions . It has also been suggested that we can determine what it is that color experiences represent on teleological grounds. In particular the suggestion is to focus on what the system has been designed to detect . A major problem with such an approach is the inference from function to content. Although the system may have the function of detecting surface spectral reflectance, it may be the case that in order to do this it actually needs to represent something more than the reflectance of the surface. In fact, in the next section I will argue for precisely this point. Note that my aim here is not to demonstrate conclusively that other attempts to solve these problems do not work. Rather I want to suggest that the proposed solutions are sufficiently unpalatable that it is worth seeing whether there is an alternative. I then hope to go on and provide just such an alternative. So it is not clear that we can find a principled way to separate the veridical from illusory color experiences. In the absence of such a separation the fact that there is no mapping from red experiences to some objective property of surfaces suggests that color experiences are not representational. Perhaps we can’t separate out the illusory from the veridical instances because they are all illusions. The other problem that faces representational accounts of color experience is the tension between introspection and science. Introspection tells us that color experiences are about individual surfaces. Psychophysics tells us color experiences depend on more than the nature of the light leaving the surface. How can color experiences appear to be about specific surfaces if they are actually about entire visual scene? This is particularly puzzling when we realise that we are trying to answer the question “what do color experiences represent?” How could we be so systematically wrong about how things appear to us in experience? Of course I do not want to deny that we can make mistakes about the nature of our experiences. But what I do want to suggest is that it would be nice if we could avoid attributing such a radical mistake to an entire form of experience. At least some color experiences seem to be about the properties of surfaces. If we take the psychophysics at face value then color experiences are about almost everything. Could we really be that wrong? I want to suggest that it is possible to develop an account of the represented object of color experience which can make sense of the psychophysical data and do justice to introspection. To appreciate this account we first need to consider why it is the case that color experiences are determinbed by the spectral, spatial and figural nature of the surround. 2. Achieving Constancy The fundamental problem of vision is that the retinal image is ambiguous – many different distal stimuli could have caused the one retinal image. For instance the information that the visual system receives concerning the reflectance of a surface is fundamentally ambiguous. The nature of the light reflected from a surface (the luminance spectrum) depends not only on the reflectance of the surface but also on the nature of the illuminating light . Two surfaces with the same reflectance can reflect light with different spectra because they are illuminated with light with different spectra (think of moving a white piece of paper from a blue to a red spotlight). Moreover, two surfaces with different reflectances can reflect light with the same spectra if they are illuminated with the appropriate lights (think of looking at a red piece of paper under a white spotlight and a white piece of paper under a red spotlight). So in order to represent surfaces as having the same reflectance despite changes in illumination the visual system needs to “undo” the effects of illumination on the luminance spectrum. But “the problem is under constrained because the eye receives information about only one proximal variable (luminance) which is determined jointly by two distal variables (reflectance and illumination)” . The one luminance spectrum could have been caused by an infinite number of combinations of illuminants and surface reflectances. In principle we cannot determine whether it is a white piece of paper under a red spotlight or a red piece of paper under a white spotlight. Yet we are able to achieve a high level of illumination-independent constancy . That is we are able to represent a surface as the same despite changes in the light reflected from it due to a change in the illumination. For instance the text on this page appears black whether it is viewed indoors or in full sunlight, even though the spectrum of the reflected light may vary dramatically in the two conditions. In fact, if viewed under a particularly dim light the white background of the page may reflect less light than the black text does when it is in direct sunlight (Palmer 1999a, p.125). This type of constancy is not perfect, as the white paper under the red spotlight example demonstrates. Nevertheless we are able to achieve a reasonable level of illumination-independent constancy. This means that the visual system must have mechanisms that enable it to (partially) overcome the inadequacy of the information it receives. By understanding the heuristics the visual system uses in dealing with the ambiguity of the retinal information we will see the reason behind the dependence of the color of the target on the spectral, spatial and figural nature of the surround. Although we still lack a complete account of the means by which the visual system is able to achieve a constant representation of the reflectance of a surface, a comparison of the light being reflected from different regions is a mechanism that is almost universally appealed to . A comparison is useful because each surface will tend to be affected by the illumination change in the same way. Although the actual amount of light reflected by a surface varies depending on the nature of the illumination, the reflectance of the surface is constant. This means that under the same illumination a darker surface will reflect less light than a lighter surface. So although the white background of this page when viewed in dim light may reflect less light than the black text when viewed in bright sunlight, the ratio between the reflectances of the page and the text will stay the same across changes in illumination . Under the same illumination the white background will always reflect more light than the black text. Similarly, a blue piece of paper will always reflect more short wavelength light than a red piece of paper. In general, when illumination changes, neighbouring regions will be affected in the same way such that comparing the reflectances of neighbouring regions can help the visual system to factor out the effects of the illuminant. This is why so many theorists have suggested that comparing the reflectances of adjacent regions is part of the visual system’s illumination-independent constancy mechanism. If illumination-independent constancy were achieved via such a mechanism then the fact that the color a surface appears to be depends on the spectral nature of the surround would be explicable. In particular it is part of the means by which the visual system is able to overcome the poverty of the information it receives and achieve a constant representation of the reflectance of a surface. However, as Gilchrist, Kossyfidis et al point out, contrast cannot be the only determinant of the appearance of a surface otherwise we would not be capable of background-independent constancy. Background-independent constancy refers to the fact that we are able to represent a surface as being the same despite changes in its background. We find the color contrast and assimilation figures so surprising because we usually do not see a surface as changing color as its background changes (you can test this by moving a colored object around so that it is being contrasted with different surfaces). Although surrounding a surface with regions with different SSR can change the appearance of the target, this is not something that occurs in such an extreme way outside of the laboratory as it does within. In fact I discovered when constructing the contrast and assimilation figures above that it is reasonably difficult to generate striking examples of these phenomena. So although contrast plays a role, it does not rule the roost: there needs to be other mechanisms which assist the visual system to achieve not only illumination-independent but also background-independent constancy. In order to achieve both background and illumination independent constancy the visual system needs to determine whether a change in the light being reflected from a surface is due to a change in illumination or in SSR. Drawing such a distinction is important because a correction for a change in illumination when the change is due to a change in surface reflectance would lead to the misperception of the reflectance of the surface. There are various heuristics that the visual system can use in order to distinguish between reflectance and illumination edges. For instance illumination edges tend to be fuzzy and somewhat graded, whereas reflectance edges tend to be sharper. A comparison of the luminance difference between two regions is also useful; illumination edges produce much greater changes in luminance than reflectance edges. For instance a typical white surface reflects 90% of incident photons and black 10%. So the luminance difference for reflectance edges is unlikely to be more than about 10:1. In contrast the luminance difference for illumination ratios can be up to 1000:1 . Spatial cues are also useful in distinguishing between reflectance and illumination edges because they can provide information about the nature of the illumination striking a surface. For instance if a surface is perpendicular to the direction of light from a particular source it will receive maximum illumination; the more a surface is slanted from a light source, the less light it will receive. Similarly, the further away from a light source a surface is the less light it will receive . Surfaces at different depths and/or orientations usually receive different amounts of illumination. So if depth information indicates that two regions are not coplanar, an edge between them tends to be perceived as an illumination edge rather than a reflectance edge. Determining the relative positions of surfaces and the illuminant can assist the visual system in detecting any differences between the illumination reaching two surfaces. Cues as to the three-dimensional lay-out of objects can also assist the visual system in determining where shadows are likely to be cast and therefore in determining where there are likely to be changes in illumination . Figural cues can also help. For instance if two surfaces are part of the same object they are likely to be similar distances from the light source and therefore are more likely receiving the same amount of light than if they belong to different objects . This suggests that the dependence of the color a region appears to be on the spatial and figural properties of the surround is part of the means by which the visual system achieves both background-independent and illumination-independent constancy. In fact we can understand the results of Agostini and Galmonte’s study by considering the role of spatial cues in distinguishing illumination from reflectance edges. Recall that they found that a surface was contrasted with the surfaces it was perceived as belonging with, not with its immediate surround. We can now appreciate the logic behind this result. Surfaces of the same object tend to be receiving similar illumination because their distance from the illuminant tends to be the same. The nature of the Necker cube illusion is such that the background is perceived to be at a different depth than the sides of the cube. As surfaces at similar depths are more likely to be receiving the same illumination it makes more sense to contrast the dashed lines with the other “sides” of the cube. Because they are more likely to be under the same illumination there is less chance of the luminance difference between them being due to both a reflectance and illumination difference. In contrast as they are perceived as being at different depths it is quite likely that the luminance difference between the dashed and the background is due to both an illumination and reflectance difference. Because of this it is likely that a comparison of the dashed lines with the background would confound the illumination and reflectance difference between them. So the dashed lines are not contrasted with the background. What we can see here is that the visual system uses figural cues in order to determine which regions are likely to be in the same plane and therefore more likely to be receiving the same illumination in order to decide which regions should be contrasted. In general spatial and figural information provides cues that can assist in determining the nature of the illumination striking a surface. This suggests that instead of being an indication that color experiences do not represent facts, the dependence of the color of a region on the spatial and figural nature of the surround is part of the visual system's means of retrieving as much information as possible from the ambiguous retinal information. The nature of the light reaching the eye is determined by (at least) the nature of the illumination, the material properties of the object and the spatial arrangement of the surfaces and illuminants. Given that the nature of the proximal stimulus is determined by the interaction between these distal properties it makes sense to model this interaction in order to more accurately represent the distal properties. This suggests that spectral, spatial and figural properties are represented together in order to increase the accuracy of the representation of the world. This claim receives further support from the fact that taking spectral properties into account helps achieve a more accurate representation of the spatial properties of the stimulus. The retinal information concerning spatial properties is also ambiguous. The retinal image is two-dimensional whereas the world is three dimensional, so there is an inevitable loss of information and ambiguity as to the distal cause of the retinal image. Importantly there is some evidence that suggests that spectral information can assist the visual system to achieve some measure of spatial constancy. Spectral cues are implicated in many models of shape and object perception and there is a growing body of evidence that object perception is more accurate for scenes in which there is chromatic variation . For instance one mechanism of shape perception is to determine shape from shading . The intensity of the light reflected from a surface will vary depending on its orientation from the source of illumination. A surface that is perpendicular from the light source will be receiving more light than one that is parallel to the illuminant. This means that by comparing the intensity of the light reflected from the surfaces it is possible to determine their relative orientations. If the position of the light source is known it is also possible to determine the absolute positioning of the surfaces. In general shading information can provide information as to the three-dimensional shape of objects. At first glance shading information does not seem to require color vision. What distinguishes color vision from achromatic vision is the ability to distinguish between light of the same intensity but with different spectral distributions. But shading information concerns differences in intensity not differences in the wavelength. However this appearance is misleading. Wavelength information is useful in distinguishing between illumination from reflectance edges and therefore in identifying shadows. For instance a change in illumination will usually only cause a change in the intensity, but not the spectral distribution of the reflected light. So by looking for a change in spectral distribution, that is, by considering wavelength information, the visual system can determine whether an edge is a reflectance or illumination edge and therefore decide whether to take the edge into account when calculating shape from shading. Importantly, there is experimental evidence that wavelength information influences the perception of shadows . So not only do spatial and figural cues assist in achieving a more accurate representation of the spectral properties of the world it also appears that spectral cues can assist in achieving a more accurate representation of the spatial and figural properties of the world. What the example of shape from shading demonstrates is the co-dependence of the information that the visual system receives about different properties in the world. The nature of the light reaching the eyes is determined by the spatial layout of objects and light sources as well as the spectral nature of objects and light sources. Given the ambiguity of the retinal information, it makes sense to model the co-dependence of these features when representing the world. By allowing the representations of things that interact in the world to mutually constrain each other the possible interpretations of the retinal image are reduced. So the dependence of the color that region appears to be on the spatial and figural properties of the region is not necessarily evidence for the fact that color experiences do not represent an external fact. Instead it appears to be part of the means by which the visual system deals with the ambiguity of the retinal information and achieves a more accurate representation of the spectral and spatial properties of the world. Further support for the claim that spectral, spatial and figural properties are modelled together can be found in the fact that color experiences capture the “statistical regularities” of visual scenes . For instance Long and Purves analysed images of natural scenes to determine the effect of the surround on a particular region. The aim was to see if context effects such as color contrast and assimilation could be explained in terms of the typical environmental cause of such stimulus configurations. They calculated the probability that a particular region would have a certain spectral configuration given the spectral characteristics and spatial complexity of the surround. They found that in natural scenes with a more simple spatial structure similar to that of the standard contrast stimuli that a surface in a green context tends to be more red and that a surface in a red context tends to be more green. In other words the spectral return of the target tended to be shifted away from that of the surround. In contrast, for natural images with a more complex spatial structure similar to that of assimilation stimuli they found that the spectral return of the target was shifted towards that of the background. So for highly iterated stimuli a surface on a green background tended to be greener whereas one on a red background tended to be redder. Notice the similarity between the statistical analysis of images and the psychophysical data considered earlier. Recall that in contrast stimuli, the appearance of the target is shifted away from the surround. Long and Purves found that in images of natural scenes with a spatial complexity similar to that of contrast stimuli, that the spectral return of the target tends to be shifted away from that of the surround. Similarly, in color assimilation, the appearance of a region is shifted towards that of the surround. In their analysis of images, Long and Purves found that in scenes that were spatially similar to assimilation stimuli that the spectral return of the target was shifted towards that of the surround. In general, the perceived colors of the targets in color contrast, constancy, and assimilation stimuli are all predicted by the way that their positions are shifted in subjective color space by the typical co-occurrence in natural scenes of the spectral returns of the targets together with the relevant spectral context and spatial configuration . In other words there is a similarity between the spectral and spatial structure of natural scenes and the relations between phenomenal color experiences such that the shifts in color space that various types of surround induce in a target resembles the spectral returns of the target and its surround in natural scenes. This finding that color experiences resemble the spectral and spatial properties of natural scenes supports the claim that the spectral and spatial properties of the world are modelled together. One apparent problem for the suggestion that spectral and spatial information is processed together is that an influential view of the visual cortex holds that there are separate visual pathways for the processing of information about different visible properties such as color, depth, form and motion . There is considerable supporting psychophysical evidence which suggests that color vision and depth and motion vision are dissociated . However more recent physiological evidence has called into question the standard claim that colors, form, motion and depth are processed separately . Similarly, numerous psychophysical studies have demonstrated that motion perception is possible at isoluminance and that wavelength information is used in depth perception . Even if it were demonstrated that there is functional segregation in the visual cortex, this finding would be consistent with the claim that various visual features are modelled together. There is no reason why there cannot be separate parts of the overall model which are specialised for modelling different things. All that is required is that there are sufficient connections between the separate areas such that the way in which spatial properties are represented influences the way spectral properties are represented. The molecular view of vision is consistent with there being numerous interconnected networks in the visual system. In line with this picture there is anatomical evidence that there are numerous bi-directional connections between the different areas of the visual cortex . The claim that spectral, spatial and figural properties are modelled together is consistent with our current empirical knowledge. But the idea that the purpose of color vision is to help us see the world more clearly is not particularly new . More importantly we have not yet determined the nature of the facts that are represented in color experiences. Do color experiences represent surface reflectance or do they represent the interaction between the spectral, spatial figural properties of the world? What we need is a representational theory that can accommodate this interdependence. 3. Modelling the Visual World We have seen that the color appearance of a surface depends not only on the reflectance of the surface but also on the spectral, spatial and figural nature of the entire visual field. The dependence also goes the other way – the way in which the spatial properties of a scene are represented depends on the spectral properties of the stimulus. We have also seen that this interdependence is a part of the visual system’s constancy mechanism. Although we can make sense of the interdependence between the representation of spatial, figural and spectral properties we have yet to determine what color experiences are about. In this section I will argue that adopting a molecular scheme of representation provides us with a new way of thinking about the content of visual experiences. In particular we will see that it is possible to make sense of the interdependence between the representation of spectral, spatial and figural information while still maintaining that color experiences represent. 3.1 Molecular and Lexical Schemes of Representation Cummins (1996) draws a distinction between molecular and lexical schemes of representation. A molecular scheme of representation is one in which the minimal semantic elements “have no meanings independent of their occurrence in a containing “complete” representation” . In contrast, in a lexical scheme the minimal semantic elements have meaning independently of their context in the larger representation. Whether the scheme is molecular or not depends in part on the theory of content determination. In particular, theories of content which ground content in the second-order resemblance relation between the vehicles and the objects are necessarily molecular . Second-order resemblance is relational resemblance. Instead of being physically identical to the object the representing vehicles are relationally similar to the object. The relations between the vehicles resemble the relations between the objects . Although it is not widely acknowledged, functional/conceptual/causal role semantics provide a second-order resemblance theory of content . The ‘picture theory’ and structural resemblance theory of representational content are also based in second-order resemblance. All of these second-order resemblance theories of representational content are necessarily molecular and therefore could be the theory of content that underlies this “molecular” view of vision. Other theories of content which are not necessarily molecular would need to in some way specify that vision is molecular if they are to be consistent with this account. Consider two different schemes for constructing the simple picture in figure 5. The elements in the boxes on either side of the picture are the basic representational building blocks out of which the picture is constructed. We can see that for the molecular scheme the basic representational building blocks have their meaning in virtue of their place in the larger representation. For instance the circle only represents a head or a wheel in virtue of its place in the larger representation. In contrast the representational building blocks in the lexical scheme on the right do have a meaning independently of their place in the larger representation – the house represents a house independently of its relations to the car and the person. In molecular schemes individual vehicles have content in virtue of their relations to other vehicles whereas in lexical schemes vehicles have content in and of themselves. It is important to note that molecular schemes are not necessarily holistic in any problematic way . ‘It is diagnostic of holistic schemes that changing the meaning of existing basic elements (or adding or removing them) changes the meaning of others’ . The problem with holism is that it is plausible that the basic elements of two people are different such that every representation they had would mean different things. This would have the effect that intentional generalisations and therefore a scientific study of the mind would be impossible . However it is not the case that adding new basic representational elements changes the meaning of all of the existing elements for molecular schemes. For instance adding a new shape to the representational palette in figure 5, say a star, doesn’t affect the fact that the circle represents a head or a wheel. In general for molecular schemes, although content is determined by the relations between the vehicles, adding or removing a basic element does not necessarily change the meaning of all of the vehicles. This means that it is possible for two representational systems to mean the same thing by a representation without having to be representationally identical. Molecular schemes of representation are not necessarily holistic, at least not in the sense that threatens the possibility of intentional explanations. Figure 5: Molecular and Lexical Schemes for constructing a simple picture. The scheme on the left is molecular whereas the one on the left is lexical The crucial aspect of molecular schemes of representation is that because a vehicle has its content in virtue of its relations to the other vehicles, it is a mistake to ask for molecular schemes what an individual vehicle represents in and of itself. For instance the circle represents nothing in and of itself. It is only when it is placed in a larger representational context that it represents a head or a wheel or some leaves. In general for molecular schemes of representation we can only talk about the content of an individual vehicle in virtue of its place in the overall system of vehicles. It is primarily the structure of vehicles that represents such that an individual vehicle only has meaning because of its place in the overall structure. This means that instead of focusing on the question “what do red experiences represent?” we need to ask “what does the structure which red experiences are a part of represent?” 3.2 The Molecular Account of Vision This change in focus from individual vehicles to the system of vehicles makes it possible to develop an account of the nature of the facts represented in visual experience that can do justice to the psychophysical data. Given that a consideration of spatial properties is useful in more accurately representing the spectral properties of the world, and vice versa, perhaps these properties are represented together in one large model. Because spectral and spatial information interact in determining the nature of the light reaching the retina, by allowing these features to interact in the visual model of the world a more veridical representation of the spectral and spatial properties of the world can be achieved. The idea is that the representational structure which color experiences are a part represents of not only the spectral properties of the world but also the spatial properties of the world. The current suggestion is similar to Akins suggestion that the primary purpose of the spectral system may not be to see objects as colored, but rather to assist us in the general task of sensorimotor guidance using visual information. Within this visual model color experiences may represent the spectral properties of objects, but they would only have this representational role derivatively. The representational relation between phenomenal redness and a particular surface reflectance would be dependent on its place in the larger representational structure. As the structure includes non-spectral features of the world, the color of a surface depends on these non-spectral features. Importantly, adopting a molecular scheme of representation provides representational way of implementing this idea. Recall that one of the major difficulties that the psychophysical data poses for a representational account of color is that there does not seem to be any simple mapping from red experiences to some objective property of surfaces – “redness” – in the world. This raises the possibility that red experiences do not represent anything at all. But if color experiences are molecular then it is a mistake to look for a mapping from an individual experience to some feature in the world. Individual color experiences would not represent in and of themselves. There is no simple mapping from the circle in figure 5 to objective properties. In one context it represents a head, in another a wheel and in another leaves. But the fact that the circle can represent many different things doesn’t mean it doesn’t represent anything at all. Rather it is simply a result of the fact that it represents in virtue of its relations to other things. Similarly red experiences can represent something even though there is no mapping from red experiences to some objective property. In one instance a particular red experience may represent a surface with one particular spectral reflectance and in another instance the same type of experience may represent a surface with an entirely different SSR. As with the circle the absence of a simple mapping from red experiences to objective properties does not mean that they do not represent anything. Rather it is a reflection of the fact that red experiences do not represent anything in and of themselves. The second difficulty posed by the psychophysical data was reconciling the data with the introspective view of color. It seems that our color experiences are about the material properties of the surface. But the psychophysical data suggests that color experiences depend on the spectral, spatial and figural properties of the entire visual scene. The puzzle is how can we be so wrong about our experiences? On the molecular view of vision the puzzle is solved because there is a sense in which an individual experience is about a particular surface while at the same time it is also true that the experience depends on how the rest of the visual scene is represented. Consider again the circles in figure 5. Although in every instance what the circle represents depends on its relation to the other shapes, it is nevertheless the case that in any individual instance of the circle represents something specific such as a head or a wheel. Similarly, within the context of a visual experience an individual color experience may represent something specific about a surface. It is just that it only represents the surface in virtue of its relations to the other experiences. The point is that one is asking the wrong question when they ask what red experiences represent. It is more accurate to ask: what do visual experiences represent? The answer is they represent at least the interaction between the spectral, spatial and figural properties of the world. So a simple visual experience may represent not only the relation between the reflectance of the region and its surround, but the spatial relations of the surfaces as well, such as the fact that the surface is part of a particular object and spatially distinct from the surround. A more realistic example would be my current experience of my red folder sitting on my light brown desk. This experience needs to be considered as a whole, as a representation of the rectangular nature of the folder, of the roughness of the old desk, of the variation in illumination across the desk and folder due to the light coming in the window and of the reflectances of the desk and folder. Within the context of this overall representation it is possible to talk about how my phenomenal red experience of the part of the folder which is in shadow represents a particular surface reflectance under a particular illuminant whereas my (slightly different) phenomenal red experience of the part of the folder which is in direct illumination represents a surface with the same reflectance under a different illuminant and my phenomenal brown experience of the desk in shadow represents a surface with a different reflectance under the same illuminant as the first surface. However these relations between color experiences and particular combinations of surface reflectances and illuminants only hold in virtue of their place in the overall set of vehicles that constitutes the experience as a whole. It is not possible to consider the representational content of my red experience in isolation from the other representational vehicles because it only has its representational content in virtue of its relations to the other vehicles. So although there is a sense in which an individual color experience is about the spectral properties of a surface. But it is primarily the structure that represents and for vision the structure also models the non-spectral properties of the world. So it is more accurate to say that color experiences are part of our visual model of the world and that this visual model represents (at least) the spectral, spatial and figural properties of the world. On the molecular view of vision the fact that the spectral, spatial and figural properties of the world are represented together becomes comprehensible. The nature of the retinal image is determined by the interaction between spatial, spectral and figural features of the external world so it makes sense for the visual system to model these features together. Instead of being a complication that needs to be explained away, the interdependence between the representation of spectral and spatial information is evidence of the sophistication of the visual system. More importantly we can provide a representational account of how our visual system is able to achieve this task. In molecular schemes of representation the fact that there is no mapping from individual representing vehicles to some objective property does not mean that the vehicles have no representational content. The fact that there is no objective property that red experiences represent in and of themselves does not mean that red experiences are not representations. Rather it is a consequence of the fact that red experiences do not represent anything in and of themselves. Adopting a molecular scheme makes possible a sophisticated color representationalism that can do justice to the psychophysical data. Notes  References http://astro1.panet.utoledo.edu/~webforce/_Activ/induction.htm