Permanent Collection

I am a neuroscientist. I spend my days studying how the brain changes in response to some sort of outside influence, or perturbation; I study perturbations that are caused by brain diseases, like epilepsy or major depressive disorder; I also study perturbations that are caused by years of training to enhance a particular skill, like competitive Scrabble play. But even simple experiences can have lasting brain effects. I had the privilege of taking a tour with Elizabeth Diggon and Parisa Radmanesh of the Esker Foundation’s 2016 exhibitions, Jack Bush: In Studio and Colleen Heslin: Needles and Pins, and this experience has changed my brain. Specifically, my brain has changed because months after seeing the exhibition, I can re-experience aspects of the afternoon. I am conscious of the fact that I saw the exhibition. My brain now allows me to re-experience hearing that Jack Bush had heart disease, and being told that the scoop-like shapes that dominate his painting, Onslaught (1969), represent heart palpitations. My brain also allows me to re-experience being surprised that such an innocuous shape formed by happy colours symbolizes disease.

How exactly did that afternoon change my brain from a neuroscientist’s perspective? The impression made by any form of visual art is fundamentally constrained by the kinds of computations performed by the visual system, and thus reflects aspects of how the visual system is wired. Visual artists employ an array of optical tricks to construct illusions of depth, colour, light and form to guide attention, evoke emotional responses, and create lasting impressions or memories. While it is clear artists have been studying visual phenomena far longer than neuroscientists, their understanding of visual processes is largely intuitive. As scientists, we use the study of art to articulate the intuitively derived understanding. Some disciplines, like psychology and philosophy, try to understand how the human brain works by asking questions from the perceptual or cognitive perspective; other disciplines, like neurobiology, tackle the problem by investigating from the other end of the tunnel, starting with the retinal signals.

Regardless of the perspective, when we view visual art, external light stimuli, or what I like to think of as the palette, are received by our canvas, the eye, processed by our interpreter, the brain, integrated with the internal state of the viewer, and ultimately constructed into a perception, decision, or action (e.g., an emotion, thought, or memory).

The Palette: Light

A given perception depends on the spectral composition of the illuminating light, the physical structure of the object, the local visual context, and the spatial resolution. Light, composed of energy packets called photons, travels in the form of electromagnetic waves—self-propagating oscillations of perpendicular electric and magnetic fields defined by certain wavelengths. Light that can be detected by the human eye has wavelengths that range from about 400 to 700 nanometers (nm). For humans, the wavelength of visible light is associated with the different colours of the rainbow: violet, indigo, blue, green, yellow, orange, and red. Depending on the molecular structure of the objects that light encounters, these waves of light are absorbed or reflected. Reflected light enters the eye. For example, when we look at Colleen Heslin’s Reasons to Travel (2015), we see mostly light blue fabric, with a little red and black at the edges. When light hits the painting, on the parts where we see blue, the wavelengths around 475 nm are reflected, and all other wavelengths are absorbed. For red, wavelengths around 650 nm are reflected, and other wavelengths are absorbed. For black, all wavelengths are absorbed, and very little light is reflected. Therefore, the light that reaches our eye is comprised of a select subset of the waveforms in the original, illuminating light.

The Canvas: The Eye

Once light enters the eye, it is focused on the retina, the eye’s “canvas.” The retina is located at the back of the eye, and is composed of an array of about 125 million light-sensitive cells called photoreceptors, which convert light energy into neural signal. There are two types of photoreceptors, rods and cones. Rod photoreceptors are very sensitive to the presence or absence of light, but are essentially colourblind. Cone photoreceptors, on the other hand, are crucial for colour vision. There are three types of cones – blue-sensitive, green-sensitive, and red-sensitive –and their combined responses determine our percept of colour. People who are born missing one or more types of cones have varying types of colourblindness. For example, people with red-green colourblindness are missing either green-sensitive cones or red-sensitive cones.

I have normal colour vision (i.e., my retina has all three types of cones). When I look at Heslin’s Not Me (2015), my eye is drawn to the right side of the painting, where the colours alternate between red and green. My friend who is red-green colourblind (i.e., his retina is missing red- or green-sensitive cones) does not share this same experience with me. For him, the red-green alternation is lost. Instead, he described the right side of the painting as having very little contrast because the colours are close in hue, saturation, and brightness. His eye is drawn to the far left side, where a lighter, unsaturated green is juxtaposed with a darker, saturated green. Because this relatively small section of the painting is so dominant, the balance is off for him. Alternately, we share a similar experience when we look at Heslin’s False Start (2015). My colourblind friend is able to differentiate between the hues in this painting very much like I do. We both love the grey colour field, with smaller elements of greenish-yellow that subtly frame the middle of the painting.

The Interpreter: The Brain

During the 19^th and 20^th centuries, several visual artists, such as Pablo Picasso and Henri Matisse, divided the visual experience of art into components such as colour and luminance in a manner surprisingly similar to the way our brain processes information (Livingstone 2002). Artists long ago recognized that colour and luminance play different roles in visual perception. Following the their intuition, visual neuroscientists have attempted to understand the neural computations that underlie the perception of colour and luminance.

Once light energy is converted to neural energy, the signals travel down the optic nerve and enter the brain to subsequent stages of the visual system. The primary visual cortex is the earliest ‘visual’ area in the brain, and is located at the back of the head in the occipital lobe. Jack Bush once told curator and critic Karen Wilkin that colour would “speak” to him, by telling him “what colour might be placed next, and so on” (Stanners 2016). Colours do speak to each other in the brain. The primary visual cortex contains colour-opponent cells that respond best when an arrangement of objects has a colour contrast such as red-green or blue-yellow. The significance of this colour-opponent neuronal organization is that we do not see, and can’t even imagine, a reddish-green or yellowish-blue. Our perception of paintings can be most intense when two opponent colours are juxtaposed. For example, Jack Bush’s Blue, Green (1964) uses blue placed adjacent to yellow to create vivid colour contrast. A stripe of yellow divides an hourglass shape with blue on top and green on the bottom. Even though the top blue portion is smaller and less weighty, my eye is drawn in that direction by the blue-yellow contrast. The shift from yellow to green (non-opponent colours) is much more subdued.

From the primary visual cortex, visual signals are further processed by two visual pathways called the dorsal “where” stream and the ventral “what” stream (Goodale and Milner 1992; Ungerleider and Haxby 1994). The dorsal processing stream (in which information travels towards the top of the brain) is responsible for determining “where” an object is in relation to the viewer (depth perception) and how objects interact in our environment (motion perception). The ventral processing stream (in which information travels towards the bottom of the brain) is responsible for determining “what” an object is. This includes processing form, shape, and colour. A recent study by a colleague of mine at the University of Calgary used functional magnetic resonance imaging (fMRI) to confirm that a brain region in the ventral processing stream, V4, is necessary for colour perception (Williams et al. 2015). fMRI is a non-invasive technique for measuring and mapping brain activity. When we engage in different tasks, our brain activity fluctuates. Tasks involving different aspects of vision – for example, colour perception versus luminance perception – produce different patterns of brain activity. Using fMRI technology, Rebecca Williams and colleagues compared brain activity while viewing colourful images after Piet Mondrian to brain activity while viewing black and white versions of those same images. They showed that V4 was active only when participants viewed the coloured versions. We could perform this same experiment using Jack Bush’s paintings rather than the Mondrian images, as colour is equally crucial for his work. For example, the original version of Rose, Red and Red (1966) would produce increased activation in V4 as compared to a monochromatic version. Interestingly, with brain damage to this region, this fresh, happy, vibrant, and elegant painting would simply become a somewhat mundane collection of grey quadrilaterals and triangles.

“What” and “where” visual processing streams usually work in balance to help us define what we see, and where it is. Because the “what” processing stream involves colour perception, it interprets colour contrast between objects in a visual scene. Because the “where” processing stream is essentially colourblind, it interprets luminance contrast between objects in a visual scene. Working together, they give us a complete experience of our visual space. I mentioned before that when I look at Colleen Heslin’s painting Not Me (2015) the alternating red-green colours dominate the painting. For my colourblind friend, the red-green juxtaposition is not eye-catching. Instead, luminance changes dominate. In this context, colourblindness alters the balance of information fed to each stream. For me, information from the “what” stream dominates, while for my friend, information from the “where” stream dominates. This creates very different experiences when we each view the painting.

…and More

Visual images can be divided into components not only based on colour and luminance, but on other features as well. For example, high spatial-frequency information associated with fine detail is perceived in the central visual filed, or fovea. Low spatial-frequency information is perceived in the peripheral visual field. This happens because the fovea contains a high density of photoreceptors, and almost every photoreceptor has its own collector cell, allowing very detailed information to be passed to the brain. Thus, the fovea is specialized for collecting information about high-spatial frequencies associated with fine details involved in local processing. The periphery of your visual field, on the other hand, is sampled by a lower concentration of photoreceptors, and collector cells pool information by combining input form several photoreceptors. As such, the peripheral visual field is specialized to detect low-spatial frequencies associated with coarse details involved in global processing.

There is a common misperception about the human brain, that the left hemisphere processes logic and math while the right hemisphere is more artistic and creative. Though this is a brain myth, there is evidence that each hemisphere processes visual scenes in different ways (Fink et al. 1997). Scientists have shown that when standing close to a painting, the right hemisphere is more active when an observer performs local processing by paying attention to the fine details. However, the left hemisphere is more active when an observer performs global processing by standing farther away from the painting, and taking in the whole view to interpret context. These two different ways of processing visual art – local vs. global – manifests in different types of perceptions. I get a very different impression when standing very near to Heslin’s Still Life (2015) as compared to when I stand farther away. When close, I pay attention to the texture of the cotton and linen fabrics and the gold, grey and black colour composition. When far, I focus on the piece as a whole, and the combination of textures and colours gives the impression of motion. The gold portions almost twirl in the grey background.

To get back to how my tour of the Jack Bush and Colleen Heslin exhibitions changed my brain, we have to talk about memory and emotion. Memory can be thought of as a group of processes that allow our experiences to shape us by changing our brain and our behaviour. Memory processes include holding onto the details of everyday life; holding information in mind for the short term while we manipulate that information, like when we use mental arithmetic to figure out how much tip to leave; and capturing the regularities in our world to allow us to adapt our behaviour accordingly. Emotion can strengthen and even alter memories.

Memory is supported by a large set of brain regions, and even more regions become involved in the presence of emotions such as happiness, fear, and anger. As an extreme example, consider a study by Daniela Palombo and colleagues at the University of Toronto (2016). They asked passengers from flight AT236 to recall their experience of their plane crash. Using fMRI, they examined what brain regions supported recollection of the emotional life threatening memory versus nonemotional memories. The study found that the emotional memory was significantly richer in detail as compared to nonemotional memories. This increased richness was supported by increased activity in the amygdala – a region of brain that is known to be important for emotional reactions, the medial temporal lobe, anterior and posterior midline structures, and visual cortex. So the plane crash changed the brains of the people involved by allowing them to re-experience the event as a memory. The emotions associated with the plane crash made their ability to re-experience the event even stronger, and altered the way in which their brains give rise to their memories.

My Esker experience obviously was much less extreme. But the paintings that I loved are the ones that left the strongest impressions. I have vivid mental representations of Jack Bush’s Rose, Red and Red (1966) and Colleen Heslin’s False Start (2015) because these paintings made me happy. I can visualize these paintings to a much greater extent than some of the other pieces, which did not strike such a strong chord. Still, I have held on to details of the afternoon. I have used and manipulated information I learned then to illustrate the brain concepts that I wrote about. I have also altered my future behaviour in that I will seek out future exhibitions from these artists because viewing their work was a very pleasant experience.

By considering more deeply our responses to our external world, including works of art, we become aware of and learn to describe and characterize these phenomena, bringing us closer to the goal of defining the neural processes that bring about the emotions, thoughts and memories we experience as humans.

References

Fink, G.R., J.C. Marshall, P.W. Halligan, C.D. Frith, R.S. Frackowiak, and R.J. Dolan. 1997. “Hemispheric specialization for global and local processing: the effect of stimulus category,” Proceedings of the Royal Society B: Biological Science 264: 487–494.

Livingstone, M. 2002. Vision and Art: the Biology of Seeing. New York: Abrams.

Goodale M. and A.D. Milner. 1992. “Separate visual pathways for perception and action.” Trends in Neuroscience 15: 20–5.

Palombo D.J., McKinnon M.C., McIntosh A.R., Anderson A.K., Todd R.M. and Levine B. 2016. “The neural correlates of memory for a life-threatening event: An fMRI study of passengers from flight AT236.” Clinical Psychological Science 4: 312-319.

Stanners S. 2016. Jack Bush: In Studio. Calgary: Esker Foundation.

Ungerleider L.G. and Haxby J.V. 1994. “‘What’ and ‘Where’ in the human brain.” Current Opinion in Neurobiology 4: 157-165.

Williams R.J., Reutens D.C. and Hocking J. 2015. “Functional localization of the human color center by decreased water displacement using diffusion-weighted fMRI.” Brain and Behavior 5: doi: 10.1002/brb3.408.