Cognitive Brain Research 20 (2004) 226 – 241

www.elsevier.com/locate/cogbrainres

Research report
Brain areas underlying visual mental imagery and visual perception:
an fMRI study
Giorgio Ganis a,b,c,*, William L. Thompson a, Stephen M. Kosslyn a,d
a
Department of Psychology, Harvard University, Cambridge, MA 02138, USA
b
Department of Radiology, Massachusetts General Hospital Martinos Center, Boston, MA 02114, USA
c
Harvard Medical School, Boston, MA 02115, USA
d
Department of Neurology, Massachusetts General Hospital, Boston, MA 02114, USA

Accepted 15 February 2004

Available online 30 April 2004

Abstract

We used functional magnetic resonance imaging (fMRI) to assess the maximal degree of shared neural processing in visual mental
imagery and visual perception. Participants either visualized or saw faint drawings of simple objects, and then judged specific aspects of the
drawings (which could only be evaluated properly if they used the correct stimulus). The results document that visual imagery and visual
perception draw on most of the same neural machinery. However, although the vast majority of activated voxels were activated during both
conditions, the spatial overlap was neither complete nor uniform; the overlap was much more pronounced in frontal and parietal regions than
in temporal and occipital regions. This finding may indicate that cognitive control processes function comparably in both imagery and
perception, whereas at least some sensory processes may be engaged differently by visual imagery and perception.
D 2004 Elsevier B.V. All rights reserved.

Theme: Neural Basis of Behavior

Topic: Cognition

Keywords: Visual cognition; Visual imagery; Cognitive neuroscience; fMRI; Cognitive control processes

1. Introduction predictions, which typically bore fruit. For example, visual

imagery selectively interferes with visual perception more
During visual mental imagery, perceptual information is than auditory perception (and vice versa [8,54]), more time
retrieved from long-term memory, resulting in the subjective is required to scan greater distances across visualized
impression of ‘‘seeing with the mind’s eye’’. The phenom- objects [9,34], and eye movements during imagery are
enological similarity between visual imagery and visual similar to those made during perception [42].
perception has been noted at least since the time of the If we assume that cognitive processes arise from specific
Greek philosophers. Plato, for instance, describes mental patterns of brain activation, then the hypothesis that like-
imagery by using the metaphor of a mental artist painting modality imagery and perception share many common
pictures in the soul (Philebus 39c). At least since the 1960s, representations and processes can also be tested using
after the cognitive revolution that followed the behaviorist neuroimaging. Specifically, this view leads us to predict
years, ‘analog’ theories posited that visual mental imagery substantial overlap in neural activation during visual mental
and visual perception share numerous common representa- imagery and visual perception.
tions and processes. This hypothesis led to many behavioral To date, only one study [39] has been designed specif-
ically to (1) compare directly the pattern of brain activation
during visual mental imagery and visual perception across
* Corresponding author. Department of Psychology, Harvard Univer-
sity, 33 Kirkland Street, Cambridge, MA 02138, USA. Tel.: +1-617-495-
most of the brain, and (2) quantify the degree of overlap.
3569; fax: +1-617-496-3112. This study was designed to compare stimuli and tasks that
E-mail address: ganis@wjh.harvard.edu (G. Ganis). differed on the surface but were hypothesized to share

0926-6410/$ - see front matter D 2004 Elsevier B.V. All rights reserved.
doi:10.1016/j.cogbrainres.2004.02.012
 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241 227

numerous underlying processes [35]. The perceptual task object was taller than it was wide). The visual perception
required participants to decide whether names were appro- task was identical to the visual imagery task, with the only
priate for objects presented in canonical or non-canonical difference being that a faint picture was presented on a
views, and the imagery task required participants to visual- computer monitor throughout each trial and participants
ize upper case letters in grids and decide whether an X evaluated the visible picture (Fig. 1).
would have fallen on each letter, if it were actually present. In addition, we used functional magnetic resonance
Positron emission tomography (PET) was used to monitor imaging (fMRI), which allows us to quantify the amount
regional cerebral blood flow as participants performed these of overlap by comparing the number of voxels activated in
tasks. The aim of the study was to establish a lower bound common during imagery and perception. This study allowed
on the amount of overlap between brain regions engaged us to assess precisely the pattern of similarities and differ-
during visual mental imagery and perception. The main ences in brain activation between the two activities.
finding was that approximately two thirds of the activated Although the crucial comparisons are between the two
regions were engaged by both tasks, suggesting a substantial conditions, we nevertheless needed a baseline against
degree of overlap in the underlying processing. which to assess the degree of activation. We employed a
However, this study [39] has two major limitations. The slow event-related fMRI paradigm and used a low-level
first is that the amount of overlap was calculated on the baseline estimated from the data during the interval be-
proportion of regions that were activated in common over a tween the response on each trial and the stimulus presen-
threshold, ignoring the extent of activation. This can be tation of the following trial. Because the purpose of this
misleading; for example, a region in the dorsolateral pre- study was to compare the overall brain areas involved in
frontal cortex activated in common might have been much imagery and perception, we wanted to minimize the
smaller than a region in the parietal lobe that was activated chance of subtracting out activation of interest (cf. Refs.
only during one condition. A second limitation is that the [17,57]), which may be more likely with more complex
imagery and perception tasks differed in many ways, which baselines. Note that this type of baseline is different from
is likely to lead us to underestimate the amount of overlap the ‘‘resting baseline’’ used in PET studies [24], where
that would be found in imagery and perception tasks that are participants are asked to spend several minutes simply
more similar. ‘‘resting’’. In this paradigm, the baseline is embedded in
More recent studies have used stimuli and tasks that were the task and, from the participant’s point of view, it feels
more similar to each other (i.e., perceiving vs. imaging like a natural pause prior to the next experimental trial.
faces), but these studies have focused on occipito-temporal Both visual mental imagery and visual perception are the
regions per se and have quantified the degree of overlap only product of the interplay of multiple cortical and subcortical
in these regions. Moreover, these studies also had important
limitations. For example, consider the study reported by
Ishai et al. [27]; during the perception condition, the partic-
ipants passively viewed stimuli presented at a rate of 1 every
second, whereas during the imagery condition, they were
asked to visualize stimuli from the same stimulus category at
the same rate. Because no behavioral measurement was
collected, it is difficult to know what the participants were
actually doing in the two conditions. Furthermore, because it
is unlikely that people can image stimuli at the rate of 1
every second [37], it is not clear whether the observed
differences reflect intrinsic distinctions between visual men-
tal imagery and perception or instead reflect other differ-
ences between the two conditions. Additional studies have
examined the brain areas activated during mental navigation,
but these studies did not focus on a direct comparison
between actual navigation and mental navigation [20,47].
Fig. 1. Schematic of the structure of a trial in the imagery and perception
We designed the present study to compare the upper
conditions. In the imagery condition, participants kept their eyes closed
bound of the similarity in processing between visual imagery throughout the scan. Each trial began with the name of a previously studied
and visual perception. To do so, we devised a task that could object, presented auditorily via headphones. In the imagery condition,
be used in both imagery and perception conditions so that participants had to generate a visual mental image of the object. In the
differences in the pattern of brain activation observed in perception condition, participants saw a faint picture of the named object. In
both cases, participants had to wait for the auditory probe (4.5 s later; ‘‘W’’,
these two conditions could not be attributed to task differ-
meaning ‘‘wider than tall’’ in this schematic) specifying the judgment to be
ences. The visual mental imagery task consisted of forming a performed. Upon hearing the probe, participants performed the judgment as
mental image of a previously studied line drawing and then quickly and accurately as they could, and response times (RT) and accuracy
evaluating a probed property (e.g., judging whether the were recorded.
228 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241

regions (e.g., Refs. [35,41]). Thus, we expected both con- 2.3. Procedure
ditions to engage large sets of brain areas. Moreover, based on
the study by Kosslyn et al. [39], we expected to find a The tasks were administered by a Macintosh G3 Power-
substantial amount of overlap between the conditions. Be- book computer (Apple, Cupertino, CA), using Psyscope
cause we used two versions of the same task in the perception software [45]. The stimuli were projected via a magnetically
and imagery conditions, thus equating many of the task shielded LCD video projector onto a translucent screen
requirements in these two conditions, we expected the placed behind the head of the participants. Participants
overlap to be especially high in brain regions more heavily could see the stimuli via a front-surface mirror mounted
involved in cognitive control, such as prefrontal cortex (e.g., on the head coil.
Ref. [49]) and parietal cortex (e.g., Refs. [26,27,39]). Fur- Prior to the MRI session (3– 7 days in advance), we gave
thermore, consistent with the results of previous studies, we the participants a booklet with hard copies, white fore-
also expected imagery and perception to draw on many of the ground on black background, of all the stimuli, one per
same portions of occipital and temporal cortex, including page; we asked them to study the stimuli in preparation for
striate cortex [27,28,36,39,40,50]. the MRI experiment. Prior to the MRI session, we also
administered the Vividness of Visual Imagery Questionnaire
(VVIQ, Ref. [46]) and Edinburgh Handedness Question-
2. Materials and methods naire [51], as well as a general health history questionnaire.
In this session, we did not describe the types of questions
2.1. Participants that would be asked during the MRI session, to minimize
the possibility that object characteristics might be encoded
Twenty normal, right-handed volunteers (8 males, 12 verbally during study.
females, mean age = 21 years) participated in the study. All Each MRI session consisted of six functional scans,
participants had normal or corrected-to-normal vision, were alternating imagery and perception conditions. The imag-
right-handed, and had no history of neurological disease. All ery and perception conditions were administered in sepa-
participants gave written informed consent for the study rate scans to avoid potential artifacts due to the cognitive
according to the protocols approved by Harvard University demands associated with task switching [31 – 33]. More-
and Massachusetts General Hospital Institutional Review over, we were concerned about possible carry-over effects
Boards. if the two tasks were intermixed, which would inflate the
Five of the twenty participants were not included in the amount of apparent common activation. Because the
analyses because a large portion of their data was not imagery and perception scans alternated, it is highly
usable, either because of uncorrectable motion artifacts unlikely that the results were contaminated by scanner
(two participants) or because of equipment problems (three drift or other slow artifacts. Before the scans, we acquired
participants). The demographics of these five participants conventional anatomical MRIs; while these images were
were comparable to those of the entire group. Thus, the being obtained, the participants listened to the names of all
analyses reported here were on data from the remaining 15 the objects to be presented in the experiment, which was
participants. intended to help them understand the words during the
noisy echoplanar imaging (EPI) session. Before each
2.2. Stimuli functional scan, we gave each participant detailed instruc-
tions, which included teaching them how to make the
We prepared 96 line drawings of common objects, appropriate judgments for that block and the meanings of
using a white foreground against a black background. the probe cue abbreviations. Following this, any questions
The visual contrast of the line drawings was 15% to about the procedure were answered. We then presented
reduce brain activation that would be elicited by the four practice trials.
stimuli in visual cortex during the perception condition The structure of each trial is illustrated in Fig. 1. In the
[29]. An example of a stimulus is shown in Fig. 1. imagery scans, we turned off all room lights and asked
Additional stimuli were prepared for the practice trials participants to keep their eyes closed, to eliminate any
(four for each block). We divided the objects into two residual light that might have been present in the room
sets, and presented one in imagery blocks, the other in (e.g., from equipment LEDs). Each trial began with the
perception blocks, counterbalanced across participants. name of a picture, presented auditorily, at which point
These sets, in turn, were divided into three subsets, each participants were to generate the corresponding visual
used with a pair of judgments. The possible judgments mental image. An auditory probe was presented 4.5 s later
were: ‘‘taller than wide,’’ ‘‘wider than tall,’’ ‘‘contains and participants performed the corresponding judgment on
circular parts,’’ ‘‘contains rectangular parts,’’ ‘‘more on the visualized object. Two judgments were used in each
top,’’ and ‘‘more on bottom.’’ Each judgment was asso- scan, randomly intermixed (out of six possible judgments),
ciated with an auditory probe cue (e.g., ‘‘W’’ for ‘‘wider to reduce the chance that participants performed the judg-
than tall’’). ment before the auditory probe. Participants responded by
 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241 229

pressing one of two keys with the dominant hand. We asked mation was applied to the functional time series from the
the participants not to press either key if they could not imagery and perception conditions.
understand the name at the beginning of a trial, so that these We did not assume an a-priori hemodynamic response
trials could be later identified and discarded. Participants function; instead, we used a finite impulse response (FIR)
were instructed to avoid eye movements and to be ready for model and estimated the fMRI response at each time point
the next trial while keeping their eyes closed during the independently using multiple linear regression (AFNI pro-
interval between trials. gram ‘‘3dDeconvolve’’). The multiple regression model
In the perception scans, we asked participants to perform included an offset (i.e., mean) and a linear trend coefficient
the same task, but based on a visible picture instead of a for each scan, plus one coefficient for each time step in the
mental image. The structure of each trial was the same as in window of interest (from  1.5 s before trial onset to 16.5 s
the imagery condition, with the following differences: The after trial onset). Correct, incorrect, and no-response trials
participants kept their eyes open, and a low-contrast line were modeled separately; only the data for trials where the
drawing of the named object was presented on the screen participant’s response was correct are reported here. Next,
from the onset of the auditory name until the response was for each voxel, we normalized the estimated hemodynamic
made. Immediately after each trial, the participants were to response by dividing each resulting value by the offset
fixate on a small asterisk presented on the screen and to be coefficient (averaged across the three scans for each con-
ready for the next trial. dition). Maps of percent signal change for each participant
In both conditions, participants were instructed to try were obtained by computing the ‘‘area under the curve’’
avoid ‘‘day dreaming’’ during the interval between trials and (i.e., by taking the sum of the normalized regression
to focus on being ready to process the next trial. They were coefficients between 3 s and 12 s post-onset, which
asked to make the appropriate judgment as quickly and included most of the hemodynamic response). Note that,
accurately as possible. if the signal during the trials did not differ from the signal
Each scan consisted of 16 trials, for a total of 48 trials per between trials (baseline), then the offset and the linear trend
condition. The inter-trial interval was 21 s. On average, the parameters would explain most of the variance throughout
interval between the end of one trial and the beginning of the scan and the resulting area under the curve (i.e.,
the next was about 15 s. estimated signal change) would be close to zero. Finally,
we performed one sample (imagery and perception con-
2.4. MRI parameters ditions) and paired (perception-imagery contrast) t-tests to
generate statistical parametric maps. We only retained
We used a 3-T Siemens Allegra scanner with a whole- clusters of five or more contiguous voxels that were
head coil. Blood oxygenation changes were monitored by significant at p < 0.0001, leading to an alpha of 0.05 for
using a T2*-sensitive sequence (gradient echo, TR = 1500 the entire 3D image. This minimum cluster size was
ms, TE = 30 ms, FOV = 20 cm, flip angle = 90j, 64
64 determined using the Monte-Carlo approach described by
matrix, voxel size = 3.125
3.125
6 mm). Each scan Xiong et al. [59] and implemented by programs
resulted in 234 volumes, each composed of 15, 6 mm, ‘‘3dFWHM’’ and ‘‘AlphaSim’’ (with 1000 iterations) in
oblique slices (slice gap = 1 mm). T1-weighted EPI AFNI. This method (a) estimates the smoothing present in
images acquired at the same locations as the subsequent the data based on a variant of the algorithm described by
BOLD images were acquired just before the functional Forman et al. [16], and (b) determines the number of
scans to facilitate later coregistration of the functional clusters of a given size that would be significant at a
images with the high-resolution structural images. High- particular threshold due to chance. The probability of a
resolution, full-volume structural images were collected false positive detection across the entire image is then
for all participants using SPGR imaging before and after determined by the frequency counts of cluster sizes. Finally,
the functional scans (128, 1.3-mm-thick sagittal slices, the t maps were converted to Z score maps.
TR = 6.6 ms, TE = 2.9 ms, FOV = 25.6 cm, flip angle = 8j, It is important to note that, unlike some other studies
256
256 matrix). These high-resolution images were (e.g., Ref. [28]), we kept the map thresholds identical for the
used for spatial normalization. imagery and perception conditions. Indeed, although acti-
vation was stronger during visual perception than imagery
2.5. MRI analyses in occipital cortex, which might justify using a different
threshold, that was not the case in many other brain regions
We preprocessed and analyzed the data with AFNI [7]. (e.g., frontal cortex).
The BOLD time series were motion corrected using AFNI We quantified the amount of overlap in terms of indi-
program ‘‘3dvolreg’’. Next, we resampled the data to a vidual voxels. First, we counted the number of ‘‘common’’
4
4
4-mm grid, smoothed slightly with a Gaussian filter voxels (C), that is voxels for which the contrast Perception
(full-width half-maximum = 4 mm, AFNI program vs. Imagery was not significant and that exhibited a signif-
‘‘3dmerge’’) and spatially transformed to match the icant activation change (relative to baseline) during both
MNI305 template [6]. Note that the same spatial transfor- imagery and perception. Next, we counted the number of
230 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241

‘‘different’’ voxels (D), that is voxels for which the contrast regression analysis, also aimed at assessing the potential
Perception vs. Imagery was significant and that exhibited a effect of difficulty on the pattern of results. This analysis was
significant activation (relative to baseline), in at least one designed to identify brain regions for which activation could
condition. Finally, we calculated the percentage of shared be predicted by the RTs. Note that we will not report data
voxels (S) as: from individual participants (e.g., how many participants
showed activation in a particular brain region) because of the
C limited power of such analyses in this study (i.e., with a
S ¼ 100
:
CþD maximum of 48 trials per condition, it is likely that there are
many subthreshold activations, which would be missed) and
This method of calculating C and D voxels is similar to because the group analysis tells us all we need to know about
the conjunction analysis described by [53], but differs from the reliability of the effects across participants.
methods based on counting the number of voxels activated
in common in two conditions regardless of the significance 3.2. fMRI data
of the direct contrast between them.
To characterize better the pattern of similarity between Numerous brain regions were activated in the visual
conditions, we performed the same calculations indepen- perception and visual imagery conditions (Fig. 2 and Table
dently for multiple brain regions. For example, we calcu- 1). It is clear from Fig. 2 and Table 1 that the overall pattern
lated the percentage of shared voxels within the inferior of activation in the two conditions is remarkably similar but,
frontal gyrus, the middle frontal gyrus, and so on. Because nevertheless, this similarity is not uniform across brain
precise localization of these regions was not crucial for the regions.
issue at hand, we used ROIs defined using the Talairach Table 1 shows that the overall proportion of overlap was
Daemon database [43], as implemented in AFNI, and very high, about 92%. Note that there are numerous ‘‘un-
confirmed visually that these ROIs provided a reasonable assigned’’ voxels (U), which could not be classified as C or
fit to individual brain structures. D voxels. For instance, some voxels were significantly
active in the imagery condition but not in the perception
condition and thus could not be classified as C voxels;
3. Results however, they also could not be classified as D voxels
because the contrast Imagery vs. Perception was not signif-
We first summarize the behavioral results, and then turn icant. The vast majority of the C voxels (4301 out of 4430)
to the fMRI results. increased activation during both imagery and perception,
whereas a small minority (129 out of 4430) decreased
3.1. Performance data activation in both conditions. Most of the D voxels (371
out of 392) significantly increased activation during the
On each trial, participants pressed one of two keys in perception condition (both relative to baseline and to the
response to the probe question. Participants were instructed imagery condition) but not during the imagery condition. A
not to press either key if they could not understand the name few voxels (5 out of 392) increased activation in both
at the beginning of a trial because of scanner noise. Overall, conditions (relative to baseline) but to a greater degree
the name was missed more often during the imagery than during perception than imagery. Finally, a few voxels (16
during the perception blocks (20.6% vs. 5.2%, respectively, out of 392) decreased activation during perception relative
F(1,14) = 22.32, p < 0.001). However, for the trials on which to both baseline and the imagery condition.
a response was made, accuracy was very high and did not
differ between imagery and perception (96.2% vs. 97.3% 3.2.1. Pattern of similarity across brain regions
correct, respectively, F(1,14) = 0.78, p>0.1). The response Fig. 2 and Tables 2– 4 show that the amount of common
times (RTs) for correct trials were slower in the imagery activation during imagery and perception was not uniform
condition than in the perception condition (medians: 1384 across brain regions: the similarity was greatest in the
vs. 1232 ms, respectively, F(1,14) = 6.3, p = 0.025). The frontal and parietal cortex and smallest in the occipital
same analysis was performed on a subgroup of 12 (out of cortex. Because most regions were activated bilaterally,
15) participants, excluding the 3 participants with the highest hereafter we will imply bilateral activation when discussing
Imagery/Perception RT ratios. For this subgroup, there were specific anatomical structures unless explicitly stated other-
no reliable differences between the median RTs in the two wise. Furthermore, hereafter we will use the term ‘‘activa-
conditions (1299 vs. 1208 ms, respectively, F(1,14) = 2, tion’’ to indicate positive changes, whereas we will use
p = 0.18). After the main analysis of the brain activation data ‘‘negative activation’’ to indicate negative changes in the
with all 15 participants, we will report the results from this BOLD signal (relative to the inter-trial baseline).
subgroup of 12 participants in order to eliminate any possible
effects of differences in overall difficulty between the con- 3.2.1.1. Frontal cortex. Both imagery and perception eli-
ditions. Furthermore, we will report the results of a linear cited reliable activation, and occasionally negative activation,
 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241 231

in frontal regions (Table 3 and Fig. 2A). Reliable activation gyrus; there were small clusters of negative activation cen-
was present in the inferior frontal gyrus, the middle frontal tered in the medial and superior frontal cortex and in the
gyrus, the superior frontal gyrus, the medial frontal gyrus, the anterior cingulate. There was complete overlap between
insular cortex, the precentral gyrus, and the anterior cingulate conditions in all frontal areas, as shown in Table 2.

Fig. 2. Activation maps (N = 15) displayed on a normalized brain (coronal sections); the left side of the brain is shown on the right. (A) The pattern of similarity
in frontal regions is illustrated with six sections. The exact position of each section is shown on the sagittal view (top right). The three columns show activation
maps (Z scores) for the perception condition, the imagery condition, and the contrast perception minus imagery. Note that the pattern of overlap is 100% in the
frontal lobe (see also Table 2), as indicated by the lack of any active voxels in the third column. (B) The pattern of similarity in parietal and temporal regions is
illustrated with six sections. (C) The pattern of similarity in parietal and occipital regions is illustrated with six sections. Portions of the calcarine cortex are
visible in the second row of panels from the bottom. The left side of the brain is shown on the right.
232 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241

Fig. 2 (continued ).

3.2.1.2. Parietal cortex. Both imagery and perception and there was complete overlap in the left angular gyrus, the
elicited reliable activation in parietal regions (Table 3 and supramarginal gyrus, the inferior parietal lobule, the superior
Fig. 2C). These regions included the left angular gyrus, the parietal lobule, and the postcentral gyrus (Table 2). There was
supramarginal gyrus, the inferior parietal lobule, the superior a small cluster for which there was more negative activation
parietal lobule, the precuneus, the postcentral gyrus, and the in the perception than in the imagery condition (Table 4). This
middle and posterior cingulate. Common negative activation cluster encompassed portions of the posterior cingulate and
was present in the right supramarginal gyrus. There was left rostro-medial portions of the precuneus. In addition,
almost complete overlap in the activation of the precuneus, activation was stronger in the perception than imagery
 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241 233

Fig. 2 (continued ).

condition in a small region in the right precuneus (Table 4), temporal gyrus, the middle temporal gyrus, the superior
extending into the occipital lobe. temporal gyrus, and the transverse temporal gyrus. Regions
of common negative activation included the right middle
3.2.1.3. Temporal cortex. Both imagery and perception temporal gyrus and the right superior temporal gyrus (Table
elicited reliable activation in multiple temporal regions 3). We found complete overlap in the activation during
(Table 3 and Fig. 2B,C). These regions included the imagery and perception in the transverse temporal gyrus, the
fusiform gyrus, the parahippocampal gyrus, the inferior superior temporal gyrus, and the left middle temporal gyrus
234 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241

Table 1 Table 2
Number of voxels activated in common (C) and differently (D) during the Percentage of shared voxels (S) in each anatomical region
Perception and Imagery conditions across all imaged brain regions Brain region %Overlap (S voxels)
Voxel type Contrasts Number
L IFG 100 (241)
Per Ima Per – Ima of voxels R IFG 100 (146)
C voxels Per > 0 Ima > 0 Per – Ima = 0 4301 L MFG 100 (325)
Per < 0 Ima < 0 Per – Ima = 0 129 R MFG 100 (250)
Total 4430 L SFG 100 (93)
R SFG 100 (88)
D voxels Per > 0 Ima>0 Per – Ima > 0 5
Per > 0 Ima = 0 Per – Ima > 0 371 L MeFG 100 (113)
Per < 0 Ima = 0 Per – Ima < 0 16 R MeFG 100 (90)
Total 392 L Insula 100 (99)
R Insula 100 (87)
% Overlap 91.9
U voxels Per > 0 Ima = 0 Per – Ima = 0 1936 L PreCG 100 (231)
Per < 0 Ima = 0 Per – Ima = 0 243 R PreCG 100 (80)
L Ant CingG 100 (30)
Per = 0 Ima > 0 Per – Ima = 0 638
Per = 0 Ima < 0 Per – Ima = 0 60 R Ant CingG 100 (13)
Per = 0 Ima = 0 Per – Ima > 0 1 L AG 100 (11)
Total 2878 L SMG 100 (47)
R SMG 100 (15)
Unassigned voxels (U) are those that exhibited one significant contrast, but L IPL 100 (206)
could not be assigned to the C or D categories. There are 27 theoretical R IPL 100 (91)
contrast combinations (i.e., Per, Ima, Per – Ima contrasts, each with three L SPL 100 (80)
possible outcomes: X > 0, X = 0, X < 0); the 10 combinations shown in the R SPL 100 (53)
table are the only ones that resulted in non-empty sets of voxels. L preCuneus 98.7 (159)
R preCuneus 98.3 (121)
L postCG 100 (157)
(Table 2). The smallest overlap was in the right fusiform R postCG 100 (31)
gyrus and in the right parahippocampal gyrus (Table 2). L postCingG 55 (20)
R postCingG 85.7 (14)
Brain regions activated only during visual perception
L Trans TG 100 (18)
included portions of the inferior temporal gyrus, the right R Trans TG 100 (20)
middle temporal gyrus, the parahippocampal gyrus and the L ITG 94.4 (18)
fusiform gyrus. No brain regions in temporal cortex were R ITG 77.8 (9)
activated only by visual imagery. L MTG 100 (94)
R MTG 95.2 (62)
L STG 100 (226)
3.2.1.4. Occipital cortex. Activation in the occipital cor- R STG 100 (173)
tex was generally stronger and more widespread during L paraHG 58.3 (12)
perception than during imagery (Fig. 2C). Clearly, sensory R paraHG 21.4 (14)
input drives this cortex more strongly than input from L postCG 100 (157)
R postCG 100 (31)
information stored in memory. As a result, the overlap
L FG 30.8 (13)
between conditions was relatively small throughout the R FG 20 (15)
occipital lobe, about 26% on average (Table 2). Common L MOG 24.1 (58)
activation was present in regions of the middle occipital R MOG 11.1 (90)
gyrus, the left superior occipital gyrus, the lingual gyrus, L SOG 100 (2)
L ling 40 (45)
and the cuneus (Table 3). Substantial portions of the lingual
R ling 20.9 (86)
gyrus, middle occipital gyrus, and the cuneus, were acti- L Cun 34.8 (43)
vated only during the perception condition (Table 4). R Cun 34.9 (86)
Furthermore, the inferior occipital gyrus was activated only L LentNuc 100 (102)
in the perception condition (Tables 3 and 4). No voxels R LentNuc 100 (55)
L Claustrum 100 (29)
were activated only during imagery, and no voxels were
R Claustrun 100 (20)
more activated during imagery than perception. In addition, L Thalamus 100 (114)
there was no significant negative activation in occipital R Thalamus 94.7 (95)
cortex. L Caudate 100 (50)
We did find reliable activation in portions of the R Caudate 100 (46)
L CingG 97.6 (127)
calcarine cortex during both visual perception and visual
R CingG 100 (108)
mental imagery (Fig. 2C). The overlap in the calcarine L Cereb 95.1 (41)
cortex was 31% in the right hemisphere and 21% in the R Cereb 78.3 (46)
left hemisphere. In general, however, visual perception The total number of activated voxels (C + D voxels) for each region is
activated this structure more strongly than did visual indicated in parentheses.
imagery.
 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241 235

Table 3 Table 3 (continued)

C voxels for each contrast
Contrast Brain region % (Total) x, y, z
Contrast Brain region % (Total) x, y, z
P < 0, I < 0, P – I = 0 L SFG 16.1 (93)  17, 44, 37
P > 0, I > 0, P – I = 0 L IFG 99.6 (241)  45, 20, 13 R SFG 18.2 (88) 15, 44, 38
R IFG 100 (146) 45, 16, 14 L MeFG 24.8 (113)  4, 52, 7
L MFG 96.9 (325)  37, 22, 30 R MeFG 23.3 (90) 4, 53, 16
R MFG 98.8 (250) 37, 21, 31 L Ant CingG 26.7 (30)  6, 33, 9
L SFG 83.9 (93)  20, 34, 33 R Ant CingG 7.7 (13) 18, 35, 20
R SFG 81.8 (88) 20, 38, 30 R AG 100 (3) 50,  58, 33
L MeFG 75.2 (113)  8, 10, 45 R SMG 40 (15) 45,  54, 31
R MeFG 76.7 (90) 8, 12, 45 R MTG 11.3 (62) 48,  51, 16
L Insula 100 (99)  38,  1, 11 R STG 5.8 (173) 45,  56, 26
R Insula 100 (87) 39, 4, 7 The first number indicates the percentage of voxels for which the contrasts
L PreCG 100 (231)  43,  5, 36 indicated on the left were significant. The number in parentheses indicates
R PreCG 100 (80) 42, 0, 33 the total number of voxels (C + D voxels in the region indicated on the left).
L Ant CingG 73.3 (30)  9, 28, 20 Talairach coordinates refer to the center of mass of clusters within the
R Ant CingG 92.3 (13) 8, 25, 23 indicated region. All contrasts are at p = 0.0001. ‘‘P’’ indicates ‘‘Percep-
L AG 100 (11)  33,  60, 34 tion’’; ‘‘I’’ indicates ‘‘Imagery’’.
L SMG 100 (47)  47,  44, 33
R SMG 60 (15) 40,  43, 35
L IPL 100 (206)  44,  40, 40 The differences between imagery and perception were
R IPL 100 (91) 40,  43, 43 not restricted to spatial extent; we also found that the time
L SPL 100 (80)  28,  61, 51 course of the hemodynamic response in many occipital
R SPL 100 (53) 28,  61, 50
regions (including the calcarine cortex) was different in
L preCuneus 98.7 (159)  18,  66, 43
R preCuneus 98.3 (121) 16,  65, 44 the two conditions. The hemodynamic response in the
L postCG 100 (157)  49,  23, 39 imagery condition began 2 s after the onset of the trial,
R postCG 100 (31) 47,  24, 35 peaked around 6 s, and began to decay 2 s after the end of
L postCingG 55 (20)  4,  36, 20 the trial. The hemodynamic response in the perception
R postCingG 85.7 (14) 3,  37, 20
condition showed the same initial peak, but also a later
L CingG 97.6 (127)  7, 5, 35
R CingG 100 (108) 7, 6, 35 peak about 4 s later, perhaps due to stimulus offset.
L Trans TG 100 (18)  49,  22, 12
R Trans TG 100 (20) 47,  24, 12 3.2.1.5. Subcortical nuclei. Several subcortical nuclei
L ITG 94.4 (18)  55,  57,  6 were activated during imagery and perception, including
R ITG 77.8 (9) 52,  49,  13
thalamic nuclei, the claustrum, the caudate, and the lenti-
L MTG 100 (94)  51,  52, 3
R MTG 83.9 (62) 51,  41, 0 form nucleus. In all cases, the overlap in activation during
L STG 100 (226)  54,  25, 9 the two conditions was almost complete. However, we
R STG 94.2 (173) 52,  24, 7 failed to observe any activation in the amygdala or in the
L paraHG 58.3 (12)  17,  34, 1 hippocampus proper.
R paraHG 21.4 (14) 18,  28,  4
L FG 30.8 (13)  51,  54,  14
R FG 20 (15) 46,  53,  12 3.2.1.6. Cerebellum. We also found substantial overlap in
L MOG 24.1 (58)  51,  63, 5 the cerebellum, mainly in the culmen, declive and vermis.
R MOG 11.1 (90) 49,  62, 6 The non-overlapping voxels were engaged more during
L SOG 100 (2)  32,  75, 30 visual perception than visual mental imagery.
L lingG 40 (45)  4,  86,  5
R lingG 20.9 (86) 6,  91,  7
L Cun 34.9 (43)  12,  81, 14 3.2.2. Additional analyses. As noted earlier, the same
R Cun 34.9 (86) 5,  86, 9 analysis was carried out on a subset of 12 participants for
L LentNuc 100 (102)  23,  2, 3 which there was no difference in RTs during imagery and
R LentNuc 100 (55) 22, 1, 3 perception (reported in the Behavioral data section). The
L Claustrum 100 (29)  30, 5, 3
same pattern of results was observed as in the analysis on all
R Claustrun 100 (20) 28, 16, 4
L Thalamus 100 (114)  10,  18, 8 15 participants. However, in this analysis, there were fewer
R Thalamus 94.7 (95) 8,  17, 8 active voxels overall (6286 vs. 7700) because of the
L Caudate 100 (50)  14, 0, 14 decreased statistical power, and the overall percentage of
R Caudate 100 (46) 12, 3, 12 shared voxels, S, was slightly higher (96% vs. 91.9%). The
L CingG 97.6 (127)  7, 5, 35
distribution of the amount of overlap across the brain was
R CingG 100 (108) 7, 6, 35
L Cereb 95.1 (41)  10,  49,  9 essentially the same as in the analysis of all 15 participants.
R Cereb 78.3 (46) 11,  53,  12 We also performed a linear regression analysis to assess
P < 0, I < 0, P – I = 0 L IFG 0.4 (241)  54, 23,  12 further the effect of difficulty on the pattern of brain
L MFG 3.1 (325)  26, 28, 38 activation. In this analysis, the RT difference between the
R MFG 1.2 (250) 49, 20, 40
imagery and perception conditions for each participant was
236 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241

Table 4 those engaged during visual perception. Previous studies

D voxels for each contrast
may have failed to document this phenomenon because of
Contrast Brain region % (Total) x, y, z differences between the tasks employed.
P > 0, I > 0, P – I > 0 R lingG 1.2 (86) 6,  85,  8 Although the present study was not designed to reveal
L Cun 4.7 (43)  6,  89, 10 the roles of specific brain regions in visual mental imagery
R Cun 2.3 (86) 16,  84, 20
and visual perception, we can draw on relevant findings
P > 0, I = 0, P – I > 0 R preCuneus 1.7 (121) 24,  79, 32
L ITG 5.6 (18)  46,  69, 0 from the literature to guide us in interpreting the data.
R ITG 22.2 (9) 42,  65, 0 A number of the brain regions activated in common may
R MTG 4.8 (62) 40,  70, 12 not be necessary to carry out imagery but instead may
L paraHG 41.7 (12)  27,  55,  5 support general-purpose aspects of the tasks. For instance,
R paraHG 78.6 (14) 23,  49,  3
there was a complete overlap of activation in the precentral
L FG 69.2 (13)  28,  60,  8
R FG 80 (15) 25,  62,  9 gyrus. At least portions of the precentral gyrus were
L IOG 100 (43)  42,  79,  4 involved in the generation of the motor response, rather
R IOG 100 (23) 38,  76,  4 than in visual imagery or perception per se, as suggested by
L MOG 75.9 (58)  30,  86, 9 the late phasic activation observed, especially in the left
R MOG 88.9 (90) 33,  82, 9
hemisphere (responses were made with the right hand).
L lingG 60 (45)  19,  76,  3
R lingG 77.9 (86) 16,  77, 3 Similarly, activation in the transverse temporal gyrus and
L Cun 60.5 (43)  19,  89, 20 in the superior temporal gyrus was essentially identical in
R Cun 62.8 (86) 18,  88, 21 the two conditions, which is not surprising because this
R Thalamus 5.3 (95) 17,  30, 4 activation most likely reflects processing of the auditory
L Cereb 4.9 (41)  30,  49,  12
stimuli. Indeed, the time course of activation in these
R Cereb 21.7 (46) 21,  48,  10
P < 0, I = 0, P – I < 0 L preCuneus 1.3 (159)  6,  55, 26 regions showed two sharp peaks that closely follow the
L postCingG 45 (20)  4,  53, 19 auditory input. Note, however, that this point does not
R postCingG 14.3 (14) 2,  53, 18 undermine the observation that the set of voxels activated
L CingG 2.4 (127)  3,  53, 28 in the visual mental imagery condition is a subset of that
The first number indicates the percentage of voxels for which the contrasts activated in the visual perception condition: Eliminating
indicated on the left were significant. The number in parentheses indicates some common voxels from the total count (e.g., because
the total number of voxels (C + D voxels in the region indicated on the left).
Talairach coordinates refer to the center of mass of clusters within the
they reflect general-purpose processes) would not change
indicated region. All contrasts are at p = 0.0001. ‘‘P’’ indicates ‘‘Percep- the fact that the voxels activated during visual imagery are a
tion’’; ‘‘I’’ indicates ‘‘Imagery’’. subset of those activated during visual perception.
Imagery and perception activated frontal structures in
remarkably similar ways; in all of the regions we examined,
used to predict the differential brain activation between the spatial pattern of activation was identical. Although there
these two conditions. Thus, activation in regions identified is still no agreement on the precise role of frontal regions in
in this analysis could be due to the difference in difficulty perception and cognition, the prevalent view is that they are
between the conditions. This analysis revealed no regions in involved in numerous types of cognitive control processes
which activation was predicted by the RTs ( p = 0.001, 5 [49]. Kosslyn [35] argued that frontal cortex implements an
contiguous voxels), which suggests that task difficulty (as information shunting system, which purportedly is involved
quantified by response times) was not an important factor in in multiple aspects of visual mental imagery and visual
producing the results. perception. Numerous cognitive control processes would be
drawn upon in common in the perception and imagery
versions of the task, including those involved in the retrieval
4. Discussion of episodic information associated with the auditory probe,
the maintenance of information about the judgment to be
In this study, we used fMRI to compare the neural carried out during visual inspection, performing the visual
structures engaged during visual perception with those evaluation, and the generation of the motor response;
engaged during visual mental imagery. We specifically cognitive control processes, such as those involved in the
designed our imagery and perception tasks to be as similar generation of visual images and the identification of faint
as possible, thereby allowing us to observe an upper bound visual objects, are also likely to engage common frontal
on the possible shared structures used in imagery and regions [35]. The use of such processes may account for the
perception. As anticipated, we found extensive overlap widespread activation in numerous prefrontal regions during
between the brain areas engaged by visual perception and both imagery and perception. Note that some studies have
visual mental imagery. However, the amount of overlap reported differences in prefrontal cortex between activation
varied across the brain, and was maximal in frontal and elicited during visual imagery and visual perception (e.g.,
parietal cortices. An important finding is that the regions Refs. [27,28]). It is likely that such differences were due in
engaged by visual imagery across the brain were a subset of large part to differences between the tasks used in the two
 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241 237

conditions. Indeed, in these studies visual imagery was We did not observe any activation in the hippocampus,
usually compared with passive viewing of visual stimuli. which is not unusual for visual imagery tasks. Indeed, a
Our results suggest that, when appropriately matched, visual recent meta-analysis [36] examined 59 visual imagery
perception and visual mental imagery activate the same studies, and found that only five experiments reported
subset of prefrontal regions. hippocampal activation [20,23,28,38,47]. It is not entirely
Visual mental imagery and perception elicited overlap- clear what aspects of the tasks are in common among these
ping activation in multiple parietal regions. These regions studies, but two factors might be important to elicit hippo-
included the superior parietal lobule and the precuneus, both campal activation during visual imagery: the use of tasks
of which are thought to play crucial roles in the same with a marked spatial component and/or the presentation of
cognitive control network that includes frontal cortices to-be-visualized materials immediately before the scanning
[44]. These regions have been implicated in attentional session.
processes [5] and spatial working memory [7,55]. A study The inferior occipital gyrus and the right superior occip-
by Kosslyn et al. [39] comparing imagery and perception ital gyrus were engaged only during perception. In addition,
reported common activation in these parietal regions (whose most of the voxels in the middle occipital gyrus, the left
Talaraich coordinates were very close to the coordinates of superior occipital gyrus, the lingual gyrus, and the cuneus
the corresponding regions found in the present study). were activated only during perception, but a minority was
Additional regions in parietal cortex were activated in activated also during imagery. The reason why large por-
common, including the left angular gyrus, the supramarginal tions of occipito-temporal cortex were activated during
gyrus, and the inferior parietal lobule. Many of these regions visual perception but not during visual imagery may be that
have been implicated in visuospatial processing [11,22,48], these visual cortical regions are involved in mid-level
and thus, it is not surprising that they were activated during perceptual operations aimed at facilitating object detection
our tasks, which had a spatial component (e.g., ‘‘is the and/or identification (e.g., grouping, fast object categoriza-
object taller than wide?’’). Common activation in the post- tion based on surface properties, and so on). Such functions
central gyrus (especially in the left hemisphere) was prob- are not required for visual mental images because these
ably related to somatosensory stimulation associated with images usually do not need to be detected or identified (one
the motor response, but it could also be related to spatial knows what one is visualizing when one forms the image).
aspects of the task [56]. The only difference between An interesting prediction of this hypothesis is that tasks that
imagery and perception was found in a small cluster require the identification of visual patterns in images, such
encompassing portions of the posterior cingulate and rostral as in tasks involving the combination of elementary visual
portions of the precuneus. Within this cluster, activation shapes to form new objects [13 – 15], should elicit stronger
during visual perception was reliably more negative than activation in these regions.
during visual imagery. However, because it is unclear The partial overlap of activation in occipito-temporal
whether negative activation reflects an actual reduction of cortex during visual perception and visual mental imagery
neural activity or whether it reflects other factors not directly is consistent with findings from the studies of neurological
related to neural activity [25], we will not speculate about patients. Indeed, findings from a number of studies have
the meaning of this difference. shown that visual imagery can be preserved in the presence
In addition, we also found activation in a region in the of impaired perception in patients with damage to the
parahippocampal gyrus during visual perception and found occipital and posterior temporal lobe (e.g., Refs. [1,3,18]).
activation in a subset of this region during imagery, which is Note, however, that several occipito-temporal voxels were
consistent with findings from the previous studies that have activated in common between perception and imagery,
compared activation in the ventral stream during imagery which suggests that portions of occipito-temporal cortex
and perception (e.g., Ref. [50]). Although this region has are used in both cases. Damage to such common neural
been reported to respond more strongly to scenes depicting substrates could account for neurological cases who had
places than to objects, it nevertheless also responds to visual parallel pattern of deficits in visual imagery and visual
objects [10]. This medial temporal region may play a role in perception [12].
encoding and storing memories of visual objects and events At least some of the differences we found between
[4,58]. We observed a similar pattern of activation in the imagery and perception may reflect the stimulus properties
fusiform gyrus and in the inferior temporal gyrus, where a of the visual stimuli. Although we used visual stimuli with
subset of voxels was activated only during visual perception, relatively low contrast (15%), it is known that even such
whereas another subset was activated both during visual low-contrast levels elicit between 65% and 75% of the
perception and visual mental imagery. This finding is con- maximum response in areas V1, V2, V3, V3A and V4 [29].
sistent with those from previous studies [27,28], and sug- Some of the large areas of activation in visual cortex that
gests that these regions may be involved in storing memories were present only during visual perception probably were
of visual objects that can be re-activated both by signals from elicited by the abrupt stimulus onset, which did not occur
lower-level visual cortex and by signals from higher-level during the imagery condition. Most neurons in the primate
cortices involved in cognitive control processes. visual pathway exhibit very large responses to stimulus
238 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241

transients [19]. The second peak of activation in visual least in part, due to the apparent motion of the square
cortex during perception is consistent with this conjecture; during the saccade conditions rather than to eye move-
such activation could be modeled reasonably well by ments; furthermore, activation in striate and extrastriate
assuming two events that would coincide with stimulus cortex could be, at least in part, due to the transient
onset and offset. generated by the fovea ‘‘landing’’ on the target square
We also found reliable activation of calcarine cortex during the saccade condition. Thus, if we assume that
during both imagery and perception. This finding is con- visual cortex is deactivated during saccades, as suggested
sistent with prior neuroimaging evidence [36] and with by Paus et al., then it is unlikely that eye movements per se
evidence that rTMS of medial occipital cortex impairs both were responsible for the observed activation during visual
perception and imagery [40]. Both the task used in that imagery or perception.
study and the task used in the present study required Many additional brain regions were engaged in common,
judging details in the visually imaged stimulus. In fact, a including the medial frontal cortex, the insular region,
recent meta-analysis [36] found that having to ‘‘inspect thalamic nuclei, and the cerebellum, but we will not
details with high resolution’’ was one of the factors that speculate about the specific roles these regions may play
predicted when early visual cortex was activated during in imagery and perception. However, it is worth noting that
visual imagery. Two other important factors were imaging some of these brain regions may reflect aspects of process-
of non-spatial visual attributes (spatial attributes activate ing that relate to performing the judgments, and others to the
posterior parietal cortex, but not medial occipital cortex) representation of the shapes per se. At first glance, this
and the sensitivity of the technique (3T fMRI being a might appear to be a problem: It is possible that our
technique with high sensitivity). estimates of overlap between imagery and perception are
One might ask whether and to what extent visual cortex inflated by the common judgment processes used in the two
activation during visual imagery and visual perception conditions. We have three responses to this concern: First,
could be the result of eye movements. On the one hand, the fact that imagery activates a subset of the brain activated
during the perception condition, eye movements during by perception stands, even if we removed voxels that reflect
stimulus presentation may produce additional visual cortex processing the judgments. Second, if the representations of
activation because at the end of each saccade there is a new shape were qualitatively different in imagery and percep-
visual transient. This would not be a concern during the tion, it would not make sense to apply the same processes to
visual imagery condition because there was no light stim- them. Thus, even if some of the common activation reflects
ulating the retina. Although we did not measure eye move- the task, this still attests to the similarity in the underlying
ments directly, we were able to estimate the amount of eye representations. Third, it is not entirely clear that the task
movements present in the imagery and perception condi- can be completely separated from the representation: The
tions by looking at the coefficient of variation of the MRI mere requirement to extract certain information may cause
signal (standard deviation/mean) in the eye vitreous [2]. that information to be included in the representation. For
Overall, this coefficient was larger during perception than example, when asked to visualize a cat and then determine
during imagery [0.2 vs. 0.07, F(1,14) = 7.1, p < 0.05], sug- whether it has curved front claws, most people report that
gesting that participants made some eye movements during they only add the claws when asked the question; the task
the perception condition, and more than in the imagery partially determines what is represented.
condition. One might also ask what effect task difficulty may have
On the other hand, eye movements per se may have an on the pattern of results, given that the median RTs were
effect because some studies have suggested that visual slower for visual imagery than for visual perception. How-
cortex activation is affected by eye movements [30,52]. ever, differences in task difficulty cannot explain the results
Although there is good evidence for activation in the because the percentage of shared voxels, S, actually in-
frontal eye fields and in superior parietal cortex during creased when we analyzed the data only from a subgroup of
voluntary eye movements, the evidence regarding the 12 participants for which the RTs were matched. Further-
effects of eye movements on visual cortex activation is more, a linear regression using the RTs to predict brain
mixed: the study by Paus et al. found deactivation in striate activation indicated that the pattern of response times was
cortex during voluntary saccades, whereas the study by unlikely to be responsible for the observed pattern of brain
Kimmig et al. found increased activation in striate cortex. activation.
However, the study by Kimmig et al. may have had a Finally, it is important to address potential issues in our
confound: during the saccade condition, a small square choice of the baselines. We chose a ‘‘no-task’’ baseline
would appear in the periphery and participants had to because we wanted to minimize the chance of subtracting
saccade to it every few seconds. The control condition out activation occurring during the experimental conditions.
consisted in fixating the same square in the center of the This is reasonable, given that the emphasis of this study was
screen. Kimmig et al. reported activation in V5, as well as on the comparison between the activation across the brain
in striate and extrastriate cortex during the saccade condi- during visual perception and imagery. However, one could
tion, relative to fixation. The activation in V5 could be, at argue that using such baselines is problematic because
 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241 239

participants may experience task-unrelated images and this variable clearly shows this not to be the case: striate
thoughts (TUITs) when they are not processing external cortex activation is not affected by whether participants
information [21]: some voxels may not show activation keep their eyes closed or open during the experiment.
during the visual perception or imagery conditions simply Seventh, regions that tend to be active during ‘‘rest’’ are in
because a similar activation is elicited during the baselines. the medial frontal cortex, the posterior medial cortex, and
Furthermore, this artifact may be more pronounced during posterior lateral cortices [24]. In our study, there were
the visual imagery condition because participants kept their small regions of deactivation in the medial prefrontal
eyes closed. Although it is possible that participants experi- cortex and in the posterior medial cortex, both in visual
enced some TUITs during the interval between trials, thus imagery and perception. It is possible that deactivations in
potentially reducing the apparent activation in the experi- these specific regions may have to do with TUITs occur-
mental conditions, this is unlikely to be a major confound. ring between trials.
First, it is true that participants tend to experience TUITs In conclusion, the present results further document that
during extended resting conditions carried out in a separate visual imagery and visual perception draw on most of the
sessions, such as in PET studies that have examined brain same neural machinery. However, the overlap is neither
activation at ‘‘rest’’ [24]. However, TUITs are less likely to uniform nor complete; visual imagery and visual perception
occur in paradigms where the experimental conditions and appear to engage frontal and parietal regions in more similar
the baseline are intermixed and the ‘‘no-task’’ time is ways than occipital and temporal regions. This finding may
relatively short. Second, the fact that we found robust indicate that cognitive control processes function similarly in
activation in a multitude of brain regions during both visual both imagery and perception, but—perhaps counter-intui-
imagery and perception (as reported, the overlap was over tively—at least some sensory processes may be engaged
90%) suggests that TUITs did not contaminate patterns of differently by visual imagery and perception.
activation in the imagery condition, even though it may seem
to have afforded a greater opportunity for such events than
the perception condition. Furthermore, the pattern of activa- Acknowledgements
tion was generally consistent with that found in previous
visual imagery and perception studies. Third, TUITs during We wish to thank Dr. Haline Schendan and Dr. Nouchine
the interval between trials may introduce some spurious Hadjikhani for helpful discussion, Jennifer Shephard and
activation, but it is probably much weaker and less consistent Deborah Bell for their assistance in stimulus selection, and
throughout the scan than that produced by the explicit visual Lawrence White, Jennifer Holmes, and Mary Foley for
imagery task. TUITs do not occur each time there is a pause assistance during MRI data collection. This research was
between task stimuli [21]: most likely, the precise timing and supported by grants NIH: 5 R01 MH60734, NSF: REC-
duration of occurrence vary throughout a scan and is not 0106760, and NIMA NMA201-01-C-0032. The MRI
time-locked with the experimental paradigm. The result is a infrastructure at MGH was supported in part by the National
much weaker average signal than the time-locked one Center for Research Resources (P41RR14075) and the
elicited by the imagery/perception conditions. Fourth, the Mental Illness and Neuroscience Discovery (MIND)
use of any other baseline would encounter similar problems Institute.
because there are always processes in the experimental
condition which one could argue are the same as in the
baseline. For instance, a baseline condition involving decid- References
ing whether numbers presented continuously are odd or even
may have a substantial cognitive control component that [1] P. Bartolomeo, A.C. Bachoud-Levi, B. De Gelder, G. Denes, G. Dalla
might cancel out frontal activation. Fifth, the striking pattern Barba, P. Brugieres, J.D. Degos, Multiple-domain dissociation be-
tween impaired visual perception and preserved mental imagery in
of similarity between the brain activation elicited by the two
a patient with bilateral extrastriate lesions, Neuropsychologia 36
conditions (relative to their respective baselines) suggests (1998) 239 – 249.
that the baselines were reasonably similar. Indeed, artifactual [2] M.S. Beauchamp, Detection of eye movements from fMRI data,
differences between baselines would tend to produce arti- Magn. Reson. Med. 49 (2003) 376 – 380.
factual differences in the resulting brain activation patterns, [3] M. Behrmann, M. Moscovitch, G. Winocur, Intact visual imagery and
rather than coincidental similarities. Sixth, there is no evi- impaired visual perception in a patient with visual agnosia, J. Exp.
Psychol. Hum. Percept. Perform. 20 (1994) 1068 – 1087.
dence that TUITs occur more often when people keep their [4] J.B. Brewer, Z. Zhao, J.E. Desmond, G.H. Glover, J.D. Gabrieli,
eyes closed as opposed to when they keep their eyes open Making memories: brain activity that predicts how well visual expe-
and fixate a faint dot in an otherwise completely dark room. rience will be remembered, Science 281 (1998) 1185 – 1187.
If it were true that keeping the eyes closed induces massive [5] R. Cabeza, F. Dolcos, S.E. Prince, H.J. Rice, D.H. Weissman, L.
increases in TUITs during the baseline condition, then one Nyberg, Attention-related activity during episodic memory retrieval:
a cross-function fMRI study, Neuropsychologia 41 (2003) 390 – 399.
would expect not to see striate cortex activation during visual [6] D. Collins, P. Neelin, T. Peters, A. Evans, Automatic 3D intersubject
imagery experiments in which participants keep their eyes registration of MR volumetric data in standardized Talairach space,
closed. A recent meta-analysis [36] examining the effect of J. Comput. Assist. Tomogr. 18 (1994) 192 – 205.
240 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241

[7] S.M. Courtney, L.G. Ungerleider, K. Keil, J.V. Haxby, Object and [31] S. Konishi, K. Nakajima, I. Uchida, M. Kameyama, K. Nakahara,
spatial working memory activate separate neural systems in human K. Sekihara, Y. Miyashita, Transient activation of inferior prefrontal
cortex, Cereb. Cortex, (1996) 39 – 49. cortex during cognitive set shifting, Nat. Neurosci. 1 (1998) 80 – 84.
[8] C. Craver-Lemley, A. Reeves, How visual imagery interferes with [32] S. Konishi, M. Kawazu, I. Uchida, H. Kikyo, I. Asakura, Y. Miya-
vision, Psychol. Rev. 99 (1992) 633 – 649. shita, Contribution of working memory to transient activation in hu-
[9] M. Denis, S.M. Kosslyn, Scanning visual images: a window on the man inferior prefrontal cortex during performance of the Wisconsin
mind, Curr. Psychol. Cogn. 18 (1999) 409 – 465. Card Sorting Test, Cereb. Cortex 9 (1999) 745 – 753.
[10] R. Epstein, A. Harris, D. Stanley, N. Kanwisher, The parahippocam- [33] S. Konishi, K. Nakajima, I. Uchida, H. Kikyo, M. Kameyama,
pal place area: recognition, navigation, or encoding? Neuron 23 Y. Miyashita, Common inhibitory mechanism in human inferior
(1999) 115 – 125. prefrontal cortex revealed by event-related functional MRI, Brain
[11] I. Faillenot, J. Decety, M. Jeannerod, Human brain activity related to 122 (Pt. 5) (1999) 981 – 991.
the perception of spatial features of objects, NeuroImage 10 (1999) [34] S.M. Kosslyn, Scanning visual images: some structural implications,
114 – 124. Percept. Psychophys. 14 (1973) 90 – 94.
[12] M.J. Farah, The neurological basis of mental imagery: a componential [35] S.M. Kosslyn, Image and Brain, MIT Press, Cambridge, MA, 1994.
analysis, Cognition 18 (1984) 245 – 272. [36] S.M. Kosslyn, W.L. Thompson, When is early visual cortex acti-
[13] R.A. Finke, Imagery, creativity, and emergent structure, Conscious. vated during visual mental imagery? Psychol. Bull. 129 (2003)
Cogn. 5 (1996) 381 – 393. 723 – 746.
[14] R.A. Finke, K. Slayton, Explorations of creative visual synthesis in [37] S.M. Kosslyn, B.J. Reiser, M.J. Farah, S.L. Fliegel, Generating
mental imagery, Mem. Cogn. 16 (1988) 252 – 257. visual images: units and relations, J. Exp. Psychol. Gen. 112 (1983)
[15] R.A. Finke, S. Pinker, M.J. Farah, Reinterpreting visual patterns in 278 – 303.
mental imagery, Cogn. Sci. 13 (1989) 51 – 78. [38] S.M. Kosslyn, N.M. Alpert, W.L. Thompson, V. Maljkovic, S.B.
[16] S.D. Forman, J.D. Cohen, M. Fitzgerald, W.F. Eddy, M.A. Mintun, Weise, C.F. Chabris, Visual mental imagery activates topographically
D.C. Noll, Improved assessment of significant activation in functional organized visual cortex: PET investigations, J. Cogn. Neurosci. 5
magnetic resonance imaging (fMRI): use of a cluster-size threshold, (1993) 263 – 287.
Magn. Reson. Med. 33 (1995) 636 – 647. [39] S.M. Kosslyn, W.L. Thompson, N.M. Alpert, Neural systems shared
[17] K.J. Friston, C.J. Price, P. Fletcher, C. Moore, R.S. Frackowiak, R.J. by visual imagery and visual perception: a positron emission tomog-
Dolan, The trouble with cognitive subtraction, NeuroImage 4 (1996) raphy study, NeuroImage 6 (1997) 320 – 334.
97 – 104. [40] S.M. Kosslyn, A. Pascual-Leone, O. Felician, S. Camposano, J.P.
[18] G. Ganis, W.L. Thompson, F.W. Mast, S.M. Kosslyn, Visual imagery Keenan, W.L. Thompson, G. Ganis, K.E. Sukel, N.M. Alpert, The
in cerebral visual dysfunction, Neurol. Clin. 21 (2003) 631 – 646. role of area 17 in visual imagery: convergent evidence from PET
[19] T.J. Gawne, J.M. Martin, Responses of primate visual cortical neurons and rTMS, Science 284 (1999) 167 – 170.
to stimuli presented by flash, saccade, blink, and external darkening, [41] S.M. Kosslyn, G. Ganis, W.L. Thompson, Neural foundations of
J. Neurophysiol. 88 (2002) 2178 – 2186. imagery, Nat. Rev., Neurosci. 2 (2001) 635 – 642.
[20] O. Ghaem, E. Mellet, F. Crivello, N. Tzourio, B. Mazoyer, A. [42] B. Laeng, D.-S. Teodorescu, Eye scanpaths during visual imagery
Berthoz, M. Denis, Mental navigation along memorized routes acti- reenact those of perception of the same visual scene, Cogn. Sci. 26
vates the hippocampus, precuneus, and insula, NeuroReport 8 (1997) (2002) 207 – 231.
739 – 744. [43] J.L. Lancaster, M.G. Woldorff, L.M. Parsons, M. Liotti, C.S. Freitas,
[21] L.M. Giambra, A laboratory method for investigating influences on L. Rainey, P.V. Kochunov, D. Nickerson, S.A. Mikiten, P.T. Fox,
switching attention to task-unrelated imagery and thought, Conscious. Automated Talairach atlas labels for functional brain mapping,
Cogn. 4 (1995) 1 – 21. Hum. Brain Mapp. 10 (2000) 120 – 131.
[22] S. Gobel, V. Walsh, M.F. Rushworth, The mental number line and the [44] E. Macaluso, M. Eimer, C.D. Frith, J. Driver, Preparatory states in
human angular gyrus, NeuroImage 14 (2001) 1278 – 1289. crossmodal spatial attention: spatial specificity and possible control
[23] G. Goldenberg, I. Podreka, F. Uhl, M. Steiner, K. Willmes, L. mechanisms, Exp. Brain Res. 149 (2003) 62 – 74.
Deecke, Cerebral correlates of imagining colours, faces and a map-I. [45] B. Macwhinney, J. Cohen, J. Provost, The PsyScope experiment-
SPECT of regional cerebral blood flow, Neuropsychologia 27 (1989) building system, Spat. Vis. 11 (1997) 99 – 101.
1315 – 1328. [46] D.F. Marks, Visual imagery differences in the recall of pictures, Br. J.
[24] D.A. Gusnard, M.E. Raichle, Searching for a baseline: functional Psychol. 64 (1973) 17 – 24.
imaging and the resting human brain, Nat. Rev., Neurosci. 2 (2001) [47] E. Mellet, S. Bricogne, N. Tzourio-Mazoyer, O. Ghaëm, L. Petit,
685 – 694. L. Zago, O. Etard, A. Berthoz, B. Mazoyer, M. Denis, Neural corre-
[25] N. Harel, S.P. Lee, T. Nagaoka, D.S. Kim, S.G. Kim, Origin of lates of topographic mental exploration: the impact of route versus
negative blood oxygenation level-dependent fMRI signals, J. Cereb. survey perspective learning, NeuroImage 12 (2000) 588 – 600.
Blood Flow Metab. 22 (2002) 908 – 917. [48] E. Mellet, S. Bricogne, F. Crivello, B. Mazoyer, M. Denis, N.
[26] J.B. Hopfinger, M.H. Buonocore, G.R. Mangun, The neural me- Tzourio-Mazoyer, Neural basis of mental scanning of a topograph-
chanisms of top-down attentional control, Nat. Neurosci. 3 (2000) ic representation built from a text, Cereb. Cortex 12 (2002)
284 – 291. 1322 – 1330.
[27] A. Ishai, L.G. Ungerleider, J.V. Haxby, Distributed neural systems for [49] E.K. Miller, J.D. Cohen, An integrative theory of prefrontal cortex
the generation of visual images, Neuron 28 (2000) 979 – 990. function, Annu. Rev. Neurosci. 24 (2001) 167 – 202.
[28] A. Ishai, J.V. Haxby, L.G. Ungerleider, Visual imagery of famous [50] K.M. O’Craven, N. Kanwisher, Mental imagery of faces and places
faces: effects of memory and attention revealed by FMRI, Neuro- activates corresponding stimulus-specific brain regions, J. Cogn. Neu-
Image 17 (2002) 1729 – 1741. rosci. 12 (2000) 1013 – 1023.
[29] S. Kastner, D.H. O’Connor, M.M. Fukui, H.M. Fehd, U. Herwig, [51] R.C. Oldfield, The assessment and analysis of handedness: the Edin-
M.A. Pinsk, Functional imaging of the human lateral geniculate nu- burgh inventory, Neuropsychologia 9 (1971) 97 – 113.
cleus and pulvinar, J. Neurophysiol. 91 (2004) 438 – 448. [52] T. Paus, S. Marrett, K.J. Worsley, A.C. Evans, Extraretinal modula-
[30] H. Kimmig, M.W. Greenlee, M. Gondan, M. Schira, J. Kassubek, tion of cerebral blood flow in the human visual cortex: implications
T. Mergner, Relationship between saccadic eye movements and cor- for saccadic suppression, J. Neurophysiol. 74 (1995) 2179 – 2183.
tical activity as measured by fMRI: quantitative and qualitative [53] C.J. Price, K.J. Friston, Cognitive conjunction: a new approach to
aspects, Exp. Brain Res. 141 (2001) 184 – 194. brain activation experiments, NeuroImage 5 (1997) 261 – 270.
 G. Ganis et al. / Cognitive Brain Research 20 (2004) 226–241 241

[54] S.J. Segal, V. Fusella, Influences of imagined pictures and sounds on biguous baseline conditions in fMRI, Proc. Natl. Acad. Sci. U. S. A.
detection of visual and auditory signals, J. Exp. Psychol. 83 (1970) 98 (2001) 12760 – 12766.
458 – 464. [58] A.D. Wagner, D.L. Schacter, M. Rotte, W. Koutstaal, A. Maril, A.M.
[55] K.M. Shafritz, J.C. Gore, R. Marois, The role of the parietal cortex in Dale, B.R. Rosen, R.L. Buckner, Building memories: remembering
visual feature binding, Proc. Natl. Acad. Sci. U. S. A. 99 (2002) and forgetting of verbal experiences as predicted by brain activity,
10917 – 10922. Science 281 (1998) 1188 – 1191.
[56] A.L. Shelton, J.D. Gabrieli, Neural correlates of encoding space from [59] J. Xiong, J.-H. Gao, J.L. Lancaster, P.T. Fox, Clustered analysis for
route and survey perspectives, J. Neurosci. 22 (2002) 2711 – 2717. functional MRI activation studies of the human brain, Hum. Brain
[57] C.E. Stark, L.R. Squire, When zero is not zero: the problem of am- Mapp. 3 (1995) 287 – 301.