What shadows and art tell us about the brain.

A piece of art can trigger many emotions and impressions, many of them just as the artist intended. However, the same painting may also reveal, unintentionally, much about the workings of the brain: how the brain recovers the light and space and surfaces that we see. These are privileged insights not available from studying vision with natural scenes or photographs. These insights depend on undetected errors in representation that of course never arise in real scenes and photographs (Cavanagh, Nature, 2005). Painters often stray from photorealistic styles, taking liberties with the rules of physics to achieve a more effective painting. Critically, some of these transgressions of physics such as impossible shadows, shapes, or reflections go unnoticed by viewers – these undetected errors are the ones that tell us which rules of physics actually count for visual perception. As artists find the rules they can break without penalty, they act as research neuroscientists and we have only to look at their paintings to uncover and appreciate their discoveries.

In this 1467 painting by Fra Carnevale, the light within the inner courtyard has no possible source and the rising shadows on the right wall are physically impossible. These are only a few of the many deviations of light and shadows that we do not notice. The artist has taken advantage of our insensitivities to keep the subject of the painting, the baby Mary, well illuminated.

But not all inconsistencies go unnoticed. Here, the impossible lighting in the middle panel (light from the left for the head but from the right for the shirt) is not too disturbing. But when the light direction is reversed on the nose, it is definitely noticed and disturbing.

Shadows are one scene property that artists can manipulate almost at will — there are very few rules that constrain what we will see as a shadow. These rules in natural and depicted scenes are the subject of a recent book — The Visual World of Shadows — by Roberto Casati and myself. Basically shadows must be darker than their surrounds (no great surprise there) but only along their contours. Many other rules of physics concerning color, shape, and location can be broken with impunity. Shadows here act as important tools for understanding mid-level vision where inferences are made based on the initial measurements of the scene.

Here is an example of shadow capture where the owner of the shadow is taken as the closest object, despite not being the source of the shadow. Shadow ownership has its own set of rules similar to Gestalt rules of grouping.