How do we know where things are?
1. Motion-induced position shifts
In addition to knowing what things are, we need to know where they are. Here we study the effects of target motion on the coding of location using a menagerie of motion-induced position effects: flash lag, flash drag, flash grab, double-drift, and the furrow illusion. These stimuli show that an object’s visual location is constructed at a high level where, critically, an object’s motion is discounted to predict its current location. As a result, we sometimes see a target far from its actual location.
In one type of stimulus, the the illusory position shift occurs along the motion path and, as Nijhawan (1994) proposed, it may compensate for inevitable neural delays between the arrival at the retina and the arrival in awareness. This extrapolation forward brings the perception of the target into its current position so that saccades, for example, can land on them accurately. Both saccades and perception make similar extrapolations to these targets. fMRI of one of these stimuli, the flash grab (Kohler, Tse, Cavanagh, 2017), shows that responses to the illusory locations are registered in early visual cortex, probably due to downward projections of attention to the expected (perceived) location.
For example, note in the movie here that the vertical bar that flashes at the location of the border is pulled along with the subsequent motion, extrapolated by about 100 ms of travel. The bar flashed in the empty sector is not captured or grabbed by anything and has little mislocalization. As a result a perfectly good right angle appears skewed suggesting that the motion effect on position occurs before any detection of good figures like right angles. See Cavanagh and Anstis( 2013) for more details.
In a different stimulus, the double-drift (Lisi & Cavanagh, Current Biology 2015), a gabor moves in one direction whiles its internal texture moves orthogonally to the gabor’s path. The perceived path of this double-drift stimulus shifts away orthogonally from its physical path, and the gabor’s perceived location may deviate by as much as several degrees of visual angle from its true location. Remarkably, saccades to these targets are determined by their physical not their perceived location. This striking dissociation has great diagnostic power and may allow us to determine, for example, whether the coordinate system for spatial attention is saccade-based or perception-based using behavioral and EEG tests.
fMRI results from Sirui Liu's PhD research show that the double drift illusion does not emerge in the visual cortex but is seen in the frontal lobes, where visual-spatial short-term memory areas would have the temporal integration required to support the effect. The figure shows the control stimuli without any internal motion (so no illusion) tilted physically left and right to angles that match the illusory tilts. These controls were used to train a classifier to discriminate physical left tilt from right tilt. The classifier was the used on data from the double drift stimuli which both moved only up and down but because other internal motion appeared titled left or right. A searchlight analysis ran this classifier over the entire cortex and found no evidence of representing actually tilted paths (the illusion) in visual cortices. The classification was significant in medial and dorsolateral prefrontal areas, among others. These findings raise the intriguing hypothesis that perception is a function of the frontal lobes, although where or how remains to be understood.
