Troche Model
Troche Model
Edited by: This study harnessed control ratings of the contribution of different types of information
Barbara Tomasino, IRCCS E.Medea, (sensation, action, emotion, thought, social interaction, morality, time, space, quantity,
Italy
and polarity) to 400 individual abstract and concrete verbal concepts. These abstract
Reviewed by:
conceptual feature (ACF) ratings were used to generate a high dimensional semantic
Anna M. Borghi, University of
Bologna and Institute of Cognitive space, from which Euclidean distance measurements between individual concepts were
Sciences and Technologies, Italy extracted as a metric of the semantic relatedness of those words. The validity of these
Stefan Heim, RWTH Aachen distances as a marker of semantic relatedness was then tested by evaluating whether
University, Germany
they could predict the comprehension performance of a patient with global aphasia on
*Correspondence:
two verbal comprehension tasks. It was hypothesized that if the high-dimensional space
Sebastian J. Crutch, Dementia
Research Centre, Box 16, National generated from ACF control ratings approximates the organization of abstract conceptual
Hospital for Neurology and space, then words separated by small distances should be more semantically related than
Neurosurgery, Queen Square, words separated by greater distances, and should therefore be more difficult to distinguish
London WC1N 3BG, UK.
e-mail: s.crutch@ucl.ac.uk
for the comprehension-impaired patient, SKO. SKO was significantly worse at identifying
targets presented within word pairs with low ACF distances. Response accuracy was not
predicted by Latent Semantic Analysis (LSA) cosines, any of the individual feature ratings,
or any of the background variables. It is argued that this novel rating procedure provides
a window on the semantic attributes of individual abstract concepts, and that multiple
cognitive systems may influence the acquisition and organization of abstract conceptual
knowledge. More broadly, it is suggested that cognitive models of abstract conceptual
knowledge must account for the representation not only of the relationships between
abstract concepts but also of the attributes which constitute those individual concepts.
Much of the debate surrounding embodied and disembodied Motivations for inclusion of emotion information include the
theories of cognition has concerned whether sensorimotor pro- fact that most emotion words refer to abstract states, and also that
cessing plays a fundamental, interactive or epiphenomenal role emotional development precedes language development (Bloom,
in conceptual knowledge (as outlined in more detail in other 1998). However, many other cognitive systems also demonstrate
papers in this Research Topic). This debate has recently been development prior to language acquisition. Although emotion
framed or re-framed as an embodiment continuum or “graded does appear to represent a core primitive that is evident prior
grounding,” highlighting the similarities and differences between to proficient language use, the same can be said for many other
so-called strong and weak forms of the embodiment hypothe- cognitive skills (e.g., novelty detection). Thus, the focus on emo-
sis (Chatterjee, 2010; Dove, 2011; Meteyard et al., 2012; see also tion as a latent factor driving abstract word representation may
Kiefer and Pulvermüller, 2012). One notable feature of some weak in fact present only a portion of the variance of the complex
embodiment theories is their emphasis upon the contribution to phenomenon.
abstract concepts of not only motor and sensory information but We have recently reported a new approach to examining
also emotion information [e.g., Andrews et al., 2009; Kousta et al., abstract conceptual attributes, in which multidimensional ratings
2009, 2011; Newcombe et al., 2012; see Pecher et al. (2011), for are used to evaluate the contribution not only of sensory, motor
a review]. Such authors acknowledge that not all abstract words and emotion information but also of a range of additional types of
are affectively loaded, but suggest that the acquisition of such information (Crutch et al., 2012). Just as motor information rep-
affectively loaded concepts provides a framework for the subse- resented by activity in the motor, premotor, and supplementary
quent acquisition of non-affective concepts based on linguistic motor areas is hypothesized to be particularly important in the
experience alone (Meteyard et al., 2012). formation and activation of certain concepts (e.g., actions, tools;
Hauk et al., 2004; Garcea and Mahon, 2012), so it is hypothe- justification for including these particular dimensions in the cur-
sized that other cognitive domains might contribute differentially rent analysis. Dimension selection was also influenced by the
to the acquisition and organization of abstract concepts. In other practicalities of selecting dimension labels which were easily com-
words, it is proposed that affect is not the only aspect of inter- prehensible and distinguishable for the lay participants providing
nal experience (other than linguistic experience) that contributes the ratings.
to the formation and organization of abstract conceptual knowl- At a more methodological level, collecting individual word
edge. The additional types of information considered include ratings appears to offer a viable technique for examining the
social interaction, morality, executive function, quantity, time, semantic attributes of abstract concepts. Certainly a number of
space, and polarity. techniques employed to study conceptual structure in the con-
The social interaction dimension was selected following previ- crete domain are more difficult to translate into the abstract
ous work on the “words as tools” (WAT) proposal that social and sphere. For example, feature listing, in which healthy individ-
linguistic information are particularly important in the acquisi- uals are requested to list physical and functional attributes of
tion of abstract terms (e.g., Borghi et al., 2011; Scorolli et al., different entities, holds both intuitive and empirical appeal; hier-
2011), and evidence suggesting the importance of introspec- archical cluster analyses of the resulting data indicate the validity
tion for the development of such concepts (e.g., Barsalou, 1999; of the approach through the emergence of item clusters which
Van Overwalle and Baetens, 2009). The morality dimension was correspond to recognizable taxonomic categories (e.g., fruit, veg-
selected to try to capture the association between certain words etables, birds, etc.; Garrard et al., 2001; Cree and McRae, 2003;
(e.g., “courage”) and the motivation to act in accordance with McRae et al., 2005; see Figure 1A). However, the feature list-
certain social or group rules, that has been hypothesized to reflect ing approach is less easily applied to the domain of abstract
cognitive-emotional association complexes represented across a words owing to the paucity of taxonomic terms, discrete prop-
prefrontal cortex-temoro-limbic network (Moll et al., 2005). The erties, and other reliable verbal markers. For example, as the
executive function dimension was selected as certain words, par- features of “cow” might include “is an animal,” “has udders”
ticularly more abstract terms with multiple meanings or senses and “makes a mooing noise,” the equivalent features of abstract
in different contexts, might be more frequently associated with terms such as “victory” or “illusion” might be much more dif-
activity in higher order cognitive systems mediating skills such as ficult to specify. Where abstract feature listing of abstract terms
planning, selection, inhibition, executive flexibility, and strategiz- has been attempted, abstract terms have been claimed to have
ing (e.g., Stuss et al., 1995). The quantity dimension was selected fewer intrinsic item properties, more properties expressing sub-
as not only is the division between numerical and non-numerical jective experience, and properties which were less specific and
semantics well-established, but also verbal terms which relate to more related to social aspects of situations [Wiemer-Hastings and
quantity (e.g., quantifiers such as “many” and “few”) have been Xu, 2005; see also Barsalou and Wiemer-Hastings (2005), for an
shown in individuals with semantic dementia to pattern more exploratory attempt to investigate the content of three abstract
with numerical than linguistic concepts (Cappelletti et al., 2006). concepts “truth,” “freedom,” and “invention”].
The time dimension was included because our subjective sense Instead of a feature generation method, the current study
of time is fundamental to our psychology and conceptions of makes use of an abstract conceptual “feature” (ACF) rating which
reality (Allman and Meck, 2012, p. 656) and the meaning of involves asking participants not to list features but rather to
many words (e.g., “past,” “present,” “future,” “brief,” “lengthy”) rate the importance of particular types of information to the
are integrally linked to either temporal perspective or perception; meaning of a given word. Comparable Likert-scale-based rating
however the relationship between such concepts and components approaches have been employed previously to explore the con-
of specific timing theories (e.g., scalar expectancy theory; Gibbon tribution of different sensory modalities to particular object cat-
et al., 1984) remains unclear. The space dimension was assessed egories (e.g., Gainotti et al., 2009; Hoffman and Lambon Ralph,
owing to previous work in aphasic stroke patients with refrac- 2012). However, to the best of our knowledge, this approach had
tory access disorders that has suggested that spatial information not been applied to abstract words prior to our recent pilot stud-
influences the organization of geographical concepts (Crutch and ies (Crutch et al., 2012; Troche et al., 2012). Despite the similarity
Warrington, 2003, 2010a); however, little is known about how of these rating methods, differences in conceptual structure and
spatial terms are mediated neurally (e.g., spatial metaphors) but availability of a taxonomic vocabulary between abstract and con-
it has been hypothesized that right posterior temporal and pari- crete concepts still make it difficult to directly equate studies of
etal cortices may be engaged in methaphoric extensions of spatial concrete and abstract features. This is illustrated by hierarchical
events (Chatterjee, 2008). Finally, the overall polarity of con- cluster analysis of a pilot dataset of 100 abstract words rated for 9
cepts (i.e., positive, neutral, negative) was also considered as a of the cognitive dimensions listed above (Crutch, unpublished).
possible marker of the reward system (e.g., Rolls, 2000) because The resulting dendrogram (see Figure 1B) reveals conceptual
appraisal of stimulus valence is central to multiple goal-directed clusters that are intuitively coherent but less easy to label than the
behaviors, and because valence may be linked to a range of stim- taxonomic clusters found in the concrete domain (see Figure 1A).
ulus attributes (e.g., spatial “up” and “down” information, as Examples of words shown to cluster tightly together based on this
demonstrated in the space-valence congruence effect; Meier and ACF rating method include “vapor” and “illusion” which, intro-
Robinson, 2004). Naturally this is not an exhaustive list of cog- spectively, share an intangible quality, but nonetheless one which
nitive dimensions which could have been assessed, and there would be difficult to label or classify in a manner comparable to
is variability in the extent of the empirical and/or theoretical many concrete entities.
FIGURE 1 | (A) Example labeling of dendrogram based on concrete item modality ratings [from Hoffman and Lambon Ralph (2012)]. (B) Labeling of
dendrogram based on pilot 100-item abstract word ratings.
Using this ACF method, we have previously shown that some pairwise similarity-ratings task (synonyms were given a much
types of information are differentially important in the represen- higher overall similarity rating than antonyms), whereas on the
tation and organization of some types of abstract words (e.g., individual word ratings, antonyms were found to be as or even
antonyms; Crutch et al., 2012). This study also demonstrated more similar than synonyms on every cognitive dimension except
important differences between pairwise ratings of word similarity polarity.
(often regarded as the gold standard for estimating semantic sim- The aim of the current study was to examine the utility
ilarity in psycholinguistic research) and calculations of similarity of semantic similarity metrics derived from ACF ratings of
based on individual word ratings. Pair-wise ratings (e.g., how sim- abstract words. More specifically, a high dimensional seman-
ilar are these two concepts) bias the rating toward a particular tic space was generated from control ratings of the contribu-
sense or meaning of the words involved, whereas individual rat- tion to individual abstract concepts of a number of different
ings elicit data from which more flexible, context-independent types of information: sensation, action, emotion, thought, social
semantic similarity metrics can be derived. For example, when interaction, morality, time, space, quantity, and polarity. The
completing ratings of antonyms (e.g., good-bad) and synonyms validity of using inter-concept Euclidean distance within this
(e.g., good-great), participants’ awareness that “opposites” should high-dimensional space as a marker of semantic dissimilarity was
be maximally different clearly influenced their judgments on the then tested by evaluating whether these distances could predict
the comprehension performance of a patient with global apha- EXPERIMENT 1—COMPARING THE POWER OF ABSTRACT
sia. We hypothesized that this patient would find it more difficult COGNITIVE FEATURE AND LATENT SEMANTIC ANALYSIS
to discriminate between words located close together within the RATINGS TO PREDICT WORD COMPREHENSION
high-dimensional space than more distantly located concepts. PERFORMANCE
The ACF Euclidean distance was also compared with Latent STIMULI
Semantic Analysis (LSA; Landauer and Dumais, 1997) cosine val- The stimuli were drawn from a corpus of 400 nouns on
ues representing word co-occurrence to determine which variable which Abstract Cognitive Feature (ACF) ratings were previously
was the better predictor of patient performance. This examina- acquired (Troche et al., unpublished). Of these 400 nouns half
tion of the semantic attributes of abstract words was motivated were classified as concrete and the other half as abstract based on
by the broader assumption that cognitive models of abstract con- imageability ratings (>500 or <450, respectively) from the MRC
ceptual knowledge must consider how both the relationships Psycholinguistic Database.
between abstract concepts and the attributes which constitute Following Crutch et al. (2012), participants were requested
those individual concepts are represented. to rate individual concepts on 12 different dimensions using
7-point Likert scales. The Likert ratings from 7 (agree) to 1
CASE REPORT (disagree) indicated participants’ level of agreement with state-
SKO is a 65-year-old male former chartered surveyor who devel- ments concerning the contribution to the concept in question
oped global aphasia which resolved to a mixed non-fluent aphasia of 9 different cognitive dimensions: sensation, action, thought,
following a large left middle cerebral artery (MCA) territory emotion, social interaction, morality, time, space, and quantity.
stroke in 1997 (see Figure 2). The stroke resulted in an exten- Three further rating scales concerning the extent to which a con-
sive left fronto-parietal infarct covering almost the entire MCA cept was positive or negative (polarity) and the ease with which
territory. Summary background neuropsychological information the concept could be modified 1 or taught were also completed.
is provided in Table 1. SKO participated previously in a study of A description of these parameters as presented to participants
antonym comprehension (Crutch et al., 2012) and was selected can be found in Appendix 1 (see also Troche et al., unpub-
for both studies on the basis of a linguistic profile that included lished). Three hundred and sixty-five participants (Mean [SD]:
deficits in verbal comprehension and impaired phonological– Age = 40.8 [12.5]; Years of education = 15.3 [2.1]; 68% female)
orthographic transcoding. SKO showed impaired performance were recruited through the online program Mechanical Turk [see
on the British Picture Vocabulary Scale test of verbal comprehen- Buhrmester et al. (2011) for data on the validity and reliability
sion, and in identifying the Crutch et al. (2007) high frequency
items drawn from five categories. Furthermore, on a simple
test of spoken non-word to written non-word matching, SKO 1 One influential approach to abstract word representation has emphasized
scored near chance when the target and foil shared no phonemes the role of context availability in facilitating lexical access for concrete words
or graphemes (e.g., “bep”-“bep” or “civ”: 7/10) and at chance (Schwanenflugel and Shoben, 1983; Schwanenflugel et al., 1992). Concrete
when there was a single shared phoneme/grapheme (e.g., “bav”- words are thought to more readily stimulate a linguistic context relative to
“bem” or “bav”: 5/10). This transcoding deficit was necessary to abstract words, especially for words appearing in isolation (Schwanenflugel
and Shoben, 1985). One variable related to context availability is the ease
enable the use of a simple spoken word to written word match- with which a word can be modified or predicated. For example, a concrete
ing paradigm, involving the discrimination of two written words word such as giant is easily modified (e.g., big, friendly, scary giant) whereas
(e.g., “faith”-“faith” or “heresy”), as a measure of verbal semantic abstract words (e.g., fate) often lack such a structure. We measured ease of
processing. modification as a way of quantifying this lexical characteristic.
FIGURE 2 | MRI of SKO acquired 9 years post-stroke, demonstrating an extensive left fronto-temporo-parietal lesion. Presented are a single sagittal
slice, with nine coronal slices from anterior to posterior through the lesion area.
Table 1 | Summary neuropsychological information on patient SKO. related on ACF but not LSA (e.g., metaphor-idiom) or vice versa
(e.g., heresy-faith)].
SKO
For the purposes of stimulus selection, both ACF and LSA
WASI matrix reasoning 18/32 (T = 49) pairwise ratings underwent a linear transformation on to a com-
mon scale between 0 and 1 bounded by the minimum and
Digit span forwards 3 digits maximum value in each matrix. The transformed LSA scale was
also negated so that for each measure, low values indicate seman-
Repetition 63/90
tic relatedness (semantically close items) and high values indicate
Reading [from Brown and Ure (1969)] 11/72 semantic un-relatedness (semantically distant items). The differ-
ence between the two matrices of transformed distances (ACF
Graded non-word reading test 0/20 minus LSA) is referred to below as the ACF-LSA discrepancy
matrix.
Spoken non-word–written non-word match These matrices were then cut-down by excluding all concrete
Level I (e.g., bep-civ) 7/10 words (defined by a concreteness rating of more than 450 on
Level II (e.g., bem-bav) 5/10 the MRC Psycholinguistic Database; Coltheart, 1981). From these
reduced matrices of abstract words, word pairs were selected
British picture vocabulary scale (short) 26/32
under five conditions:
Pyramid and palm trees test
3 pictures 45/52 1. ACF maximum relatedness (ACFmax; N = 10)—most related
3 written words 34/52 words pairs from the ACF Euclidean values (irrespective of
LSA ratings; e.g., attitude-belief).
McKenna and Warrington (1978) 2. LSA maximum relatedness (LSAmax; N = 10)—most related
Naming Animals 3/10 (30%) word pairs from the LSA ratings (irrespective of ACF ratings;
Man-made artifacts 3/10 (30%) e.g., opposition-leadership).
Colors 2/10 (20%) 3. ACF more related than LSA (ACF > LSA; N = 10)—word
Body part 3/10 (30%) pairs with highest values in the ACF-LSA discrepancy matrix
Countries 9/10 (90%) (e.g., accumulation-majority).
TOTAL 20/50 (40%)
4. LSA more related than ACF (LSA > ACF; N = 10)—word
pairs with lowest (or most negative) values in the ACF-LSA
Comprehension Animals 10/10 (100%) discrepancy matrix (e.g., ignorance-truth).
Man-made artifacts 10/10 (100%) 5. Semantically unrelated (N = 10)—least related word pairs
Colors 7/10 (70%) drawn equally from the ACF (ACFmin; N = 5) and LSA
Body part 6/10 (60%) (LSAmin; N = 5) matrices (e.g., announcement-category).
Countries 10/10 (100%)
In order to minimize overlap of individual words between condi-
TOTAL 43/50 (86%)
tions, the word pairs in each condition were selected from among
the 20 highest/lowest rated pairs fitting each of the above descrip-
tions. The mean raw, transformed and discrepancy ACF and LSA
of this approach] and rating surveys were created and com- ratings are shown in Table 2. Furthermore, MDS plots of the dis-
pleted within Survey Monkey (www.surveymonkey.com). Data tance between word pairs in each of the 5 experimental conditions
were excluded if participants took less than 10 min to com- are shown in Figure 5. As expected given the definition of the
plete the survey, used less than half of the seven point Likert ACF > LSA and LSA > ACF conditions, there was no correlation
scale, or provided a run of more than 20 identical sequential between the ACF and LSA ratings for the 50 selected word pairs
responses. (r = −0.02). Additional data on the average concreteness, image-
For the current experiment, two independent symmetric ability, age of acquisition, frequency (Baayen et al., 1993) and
matrices of pairwise semantic similarity ratings were derived for length discrepancy of words in each pair are also given in Table 2.
the 400 word set. Values in the first matrix denoted the Euclidean The concreteness and imageability of items differed between con-
distance between words in a given pair based upon ACF rat- ditions [F(4, 43) = 2.80, P = 0.04 and F(4, 43) = 2.59, P = 0.05,
ings on the 12 dimensions specified above. The second matrix respectively], but there were no overall significant differences
contained pairwise LSA (www.lsa.colorado.edu) cosines. A multi- between conditions of age of acquisition [F(4, 33) = 1.72, P >
dimensional scaling (MDS) map based on ACF ratings of the 400 0.1], frequency [F(4, 45) = 1.83, P > 0.1], familiarity [F(4, 43) =
words across all 12 dimensions is shown in Figure 3. A scatterplot 1.79, P > 0.1], or word length difference [F(4, 45) = 0.10,
showing the relationship between ACF distances and LSA cosines P > 0.9].
for all pairwise combinations of the rated abstract words (N =
208 words; 21,528 combinations) is shown in Figure 4. The two PROCEDURE
scales showed a modest correlation (r = −0.31), but a number The identities of words in each pair were examined using a spo-
of word pairs showed discrepant relatedness ratings [i.e., highly ken word to written word matching paradigm. SKO was presented
FIGURE 3 | Multidimensional scaling map based on ACF ratings of the 400 words across all 12 dimensions.
with a series of arrays comprising two written words. For each transformed LSA cosine, concreteness, frequency and word length
array the examiner spoke the name of one of these words aloud, discrepancy as regressors. In addition, total scores were generated
with the identity of the target word varying between arrays in a for each word pair (/10; N = 50) and analysed using linear regres-
fixed random order. On each occasion the patient was required to sion with the same regressors. This latter model was also re-run
point to the word they had just heard. For each word pair, there replacing the ACF distance with the mean score differences for
were 10 consecutive trials, with each word probed five times in a each of the 12 individual cognitive dimensions.
pseudorandom order (maximum 3 consecutive presentations of
the same target item). Written words were presented on opposite RESULTS
sides of the screen, with the position of words varied in a pseu- SKO’s response accuracy in each of the five conditions is shown in
dorandom order so that target responses were on each side of Figure 6. Inspection of these raw data suggest that SKO responded
the screen equally often (maximum 3 consecutive presentations less accurately in the ACFmax than LSAmax condition and
of written words in the same spatial arrangement). Items were less accurately in the ACF > LSA than LSA > ACF condition.
presented with an approximate 1 s response–stimulus interval Performance in the ACFmax and ACF > LSA conditions was
(RSI) during which a blank screen was presented. This procedure worse than in the combined control condition but performance
yielded a total of 10 responses per word pair, and thus 100 in each in the LSAmax and LSA > ACF conditions was comparable to
condition, and 500 responses in total. Word pairs from each con- the combined control condition. Dividing the combined con-
dition were presented in a pseudorandom order. The word pairs trol condition into the ACFmin and LSAmin sets, performance
were presented on a MacBook Air laptop in the Print Preview on the ACFmin stimuli was superior. Indeed performance on
mode of Microsoft Word in black 55 point Arial font on a white the ACFmin stimuli was superior to performance on all main
background. experimental conditions (ACFmax, LSAmax, ACF > LSA, and
LSA > ACF) whereas performance on the LSAmin stimuli was
ANALYSIS only superior to the ACFmax and ACF > LSA conditions.
Response accuracy was assessed using two complementary analy- The logistic regression of individual item response accuracy
ses owing to the lack of independence between responses inherent revealed a highly significant effect of semantic distance as defined
in the repetitive probing procedure. A logistic regression analy- by the ACF distance (z = 3.76, P < 0.001) but not LSA cosine
sis of binary accuracy data for each response (N = 500) clustered (z = −0.03, P > 0.9). None of the control variable regressors
by word pair was conducted with transformed ACF distance, had a significant effect upon response accuracy (concreteness:
FIGURE 4 | Scatterplot showing the relationship between ACF Euclidean distances (lower values indicate greater relatedness) and LSA cosines
(higher values indicate greater relatedness) for all pairwise combinations of the rated abstract words only (N = 208 words; 21,528 combinations).
Table 2 | Mean (and standard deviation) ratings for word pairs in each of the five conditions in Experiment 1; data are provided for ACF
Euclidian distances, LSA cosines, adapted ACF and LSA ratings (where 0 is unrelated and 1 is related), ACF-LSA discrepancy (ACF adapted
rating minus LSA adapted rating), concreteness (CNC), imageability (IMG), age of acquisition (A0A), frequency (CELEX), familiarity (FAM), and
difference in number of letters (NLET).
Condition ACF LSA ACF LSA ACF-LSA CNC IMG AOA CELEX FAM NLET
euclidean cosine adapted adapted discrepancy difference
ACFmax 1.04 (0.06) 0.28 (0.16) 0.07 (0.01) 0.57 (0.15) −0.50 (0.14) 280.2 (20.6) 326.2 (48.9) 501.4 (59.2) 41.1 (28.0) 517.6 (39.7) 2.3 (1.6)
ACF > LSA 1.72 (0.33) −0.05 (0.06) 0.14 (0.03) 0.88 (0.06) −0.74 (0.05) 314.0 (24.2) 337.7 (24.0) 512.3 (27.3) 14.1 (21.4) 473.0 (52.1) 2.3 (1.2)
LSAmax 2.88 (0.97) 0.70 (0.08) 0.26 (0.10) 0.19 (0.07) 0.07 (0.12) 324.9 (64.3) 402.6 (125.4) 457.9 (136.9) 40.4 (26.6) 512.9 (81.5) 2.0 (2.2)
LSA > ACF 5.20 (1.38) 0.45 (0.15) 0.49 (0.14) 0.42 (0.14) 0.07 (0.02) 306.7 (37.9) 351.1 (37.0) 459.6 (66.8) 28.0 (23.8) 530.6 (32.9) 2.5 (1.5)
Unrelated 5.60 (2.39) −0.04 (0.11) 0.53 (0.24) 0.87 (0.10) −0.34 (0.34) 339.7 (48.8) 405.3 (79.8) 402.9 (118.2) 31.9 (28.1) 528.3 (54.7) 2.4 (2.6)
z = 0.67, P > 0.5; frequency: z = 1.99, P > 0.05; difference in frequency: P > 0.1; difference in word length: P > 0.4). When
word length: z = −0.88, P > 0.3). this model was repeated using the mean discrepancy scores
The results of the linear regression analysis of word pair for each of the 12 individual cognitive dimensions instead of
scores revealed similar results, with a significant effect of semantic the ACF distance, none of the individual rating discrepancies
relatedness as defined by ACF distance (P = 0.005) but none were found to be a significant predictor of SKO’s response
of the other regressors (LSA: P > 0.7; concreteness: P > 0.5; accuracy.
FIGURE 5 | Multidimensional scaling maps of the position of words constituting the word pairs tested in Experiment 1, showing plots for all
conditions together and each condition separately.
FIGURE 6 | Percentage correct responses in each of the following more related than ACF (LSA > ACF), and semantically unrelated (data
conditions: ACF maximum relatedness (ACFmax), ACF more related shown for both All unrelated, and separately for ACFmin and LSA min
than LSA (ACF > LSA), LSA maximum relatedness (LSAmax), LSA items).
We also examined whether these apparent differences between of patient response accuracy. In particular, we tested whether this
comprehension accuracy for words selected on the basis of the held true even when the words being examined were drawn from
ACF and LSA ratings were evident on the first response to each different areas of the semantic space as defined by representa-
target. The number of correct responses was calculated for ACF tion at different ends (high/low) of an individual rating scale.
(summing across ACFmax and ACF > LSA), LSA (summing In this case the “quantity” rating scale was selected as this was
across LSAmax and LSA > ACF), and unrelated items (sum- the single dimension which approximated most closely to one
ming across ACFmin and LSAmin). Chi-squared tests revealed of the three factors (perceptual salience, emotion/social cogni-
performance was significantly worse in the ACF condition than tion, and magnitude) which emerged from the hierarchical cluster
the unrelated condition (χ2[1] = 4.26, P = 0.04), but neither of analysis of all 400 words in the original corpus (Troche et al.,
the remaining comparisons was significant (LSA vs. unrelated: unpublished). However, equivalent results would be predicted
χ2[1] = 1.88, P > 0.1; ACF vs. LSA: χ2[1] = 0.95, P > 0.3). had another dimension been selected as a means of defining
different regions within the semantic space.
COMMENT
The ACF distance metric, based on control ratings of the con- STIMULI
tribution of different cognitive dimensions to each concept, was All abstract words (CNC rating <450) from the Troche et al. set
the only significant predictor of SKO’s response accuracy. This were rank ordered by their ratings on a single dimension: quan-
suggests that these novel ratings captured important aspects of tity. The 20 words with the highest quantity ratings and the 20
the conceptual relationship between the two words in each pair words with the lowest quantity ratings were selected. From these,
which were not captured as strongly by the co-occurrence-based two sets of 16 words were selected, and within each set words
LSA cosine. It is of note that none of the 12 individual rating dif- were formed into word pairs. Critically the mean ACF distance
ferences were found to be a significant predictor of performance; between words in high and low quantity word pairs was matched
only distance within the high-dimensional space generated from (i.e., they were very closely matched for the ACF rating of seman-
these ratings predicted response accuracy. tic relatedness; t = 0.004, P > 0.99, 2-tailed test). High and low
quantity words were also matched for concreteness, imageability,
EXPERIMENT 2—NON-REPETITIVE PROBE age of acquisition, frequency, familiarity and number of letters,
COMPREHENSION TASK phonemes, and syllables (all P > 0.05, 2-tailed test; see Table 3).
The data reported in Experiment 1 indicate that the ACF distance
is a predictor of SKO’s ability to discriminate two words. In this PROCEDURE
Experiment, we tested the complementary null hypothesis, that The task involved spoken word to written word matching as in
words matched closely for distance would yield comparable levels Experiment 1, except that each item was only probed once per
Table 3 | Mean (and standard deviation) ratings for high and low quantity items (Experiment 2) on multidimensional ACF semantic ratings
(ACFdist), concreteness (CNC), imageability (IMG), age of acquisition (A0A), frequency (CELEX), familiarity (FAM), and number of letters
(NLET), phonemes (PHN) and syllables (NSYL).
High quantity items 1.90 (0.26) 311.2 (52.0) 359.2 (58.7) 485.9 (42.3) 32.0 (30.4) 514.6 (42.4) 8.6 (1.9) 7.9 (1.7) 3.1 (0.8)
Low quantity items 1.88 (0.44) 317.5 (54.4) 377.1 (98.7) 482.4 (106.5) 14.3 (20.4) 477.7 (76.5) 7.7 (3.0) 6.6 (2.8) 2.8 (1.1)
block. Within each block, all written word pairs were presented of word co-occurrence such as LSA that better capture linguistic
twice in a pseudorandom order, once with the spoken name of experience and contextual association.
one written word and once with the spoken name of the alternate In Experiment 1, we hypothesized that if the high-dimensional
word (N = 8 word pairs and N = 16 spoken word targets per space generated from ACF control ratings approximates the orga-
block). All low quantity items were presented in the first block, nization of abstract conceptual space, then words separated by
and all high quantity items presented in the second block. Later in small Euclidean distances should be more semantically related
the testing session, both blocks were repeated in the reverse order than words separated by greater distances, and should therefore
with a different within-block pseudorandomized trial order. This be more difficult to distinguish for our patient with a compre-
yielded a total of 32 responses in each condition. hension deficit, SKO. It should be emphasized that ACF semantic
space is based on numerical ratings for individual words not word
RESULTS AND COMMENT pairs, and therefore none of judgments gathered from controls
SKO showed identical response accuracy rates for the two con- correspond directly to the relationship between the word pairs
ditions (High quantity words = 21/32, Low quantity words = used in Experiment 1. As predicted, SKO was significantly worse
21/32). This result supports the conclusion drawn from at identifying targets presented within word pairs with low ACF
Experiment 1 that distance within the ACF high-dimensional distances. Neither LSA cosines nor any of the background vari-
space can provide reasonable metric of semantic relatedness, at ables were found to be significant predictors of response accuracy.
least in relation to the comprehension performance of patient SKO’s performance on this spoken word to written word match-
SKO. The close matching of accuracy levels across words drawn ing task is indicative of semantic processing as his phonological
from different areas within that semantic space also suggest that to orthographic transcoding route is so impaired he is forced
this metric may have utility for determining/predicting semantic to make responses on the basis of words’ semantic properties
relatedness among a diverse set of concepts. (e.g., even in the unrelated condition, SKO occasionally made
errors distinguishing items with highly distinct phonological and
DISCUSSION orthographic forms, such as “opera—responsibility”).
The aim of this study was to evaluate a novel metric for measuring In Experiment 2, we tested the complementary hypothesis
the semantic relatedness of abstract words. The study relates to the that word pairs matched closely for ACF distance would yield
current Frontiers Research Topic on sensorimotor processing and equivalent levels of response accuracy. The critical aspect of this
(abstract) conceptual knowledge because the abstract cognitive otherwise rather drab-sounding experiment was to select items
“feature” (ACF) ratings described are based on control estimates from different areas within the high-dimensional ACF space,
of the contribution of different cognitive systems to individual namely words rated at opposite ends of a particular rating dimen-
concepts. These cognitive dimensions include those central to sion, quantity. Again as predicted, SKO’s response accuracy was
strong embodiment theories of cognition (sensation and action), perfectly matched across the two conditions. This suggests that
additional domains posited by weak embodiment theorists (emo- the distance provides some measure of word relatedness across
tion), and other types of information not previously considered quite a diverse array of topics and subject areas. The failure of
in this regard (labeled: thought, social interaction, morality, time, word pairs constructed from two words both rated highly for a
space, quantity, and polarity), plus ease of modifiability and single variable to yield a higher error rate in SKO is also consis-
teaching. These ratings were designed to measure the content tent with the idea that it is the combination of different cognitive
or semantic attributes of abstract words, and thus to be loosely dimensions rather than any single dimension that contributes to
analogous to feature generation approaches to the study of the the predictive power of the ACF distance in the current mixed
structure of concrete conceptual knowledge (e.g., Garrard et al., set of abstract words. More direct support for this may be taken
2001; Cree and McRae, 2003). However, like recent attempts to from the failure of any single rating dimension to predict SKO’s
rate the contribution of different modalities to concrete concepts response accuracy in Experiment 1.
(Gainotti et al., 2009; Hoffman and Lambon Ralph, 2012), the Taken together with previous evidence of the explanatory
ACF approach avoids the constraints of linguistic labels inherent power of ACF ratings for antonym and synonym discrimina-
in feature generation. The approach also benefits from the consid- tion in aphasic patients (Crutch et al., 2012), these experiments
eration of concepts individually rather than generating (context provide preliminary support for an approach which attempts
dependent and less flexible) pairwise ratings of the specific rela- to quantify the semantic similarity of abstract words based
tionship between two words. The ACF approach described was on their constituent semantic attributes rather than their spe-
also intended to complement rather than compete with measures cific, contextually-bound relationships to other abstract words.
It should be noted that the data presented does not distin- knowledge more germane to the concrete domain (e.g., visual—
guish between embodied and disembodied theories of conceptual color, visual—parts and surface properties, visual—motion, tac-
knowledge. The broader approach for which the data argues, tile, olfactory, gustatory, auditory, functional, and encyclopedic;
namely that a number of different types of information and Cree and McRae, 2003). These dimensions could easily supercede
internal experience contribute to abstract conceptual knowledge, the broad “sensation” and “action” dimensions used in the cur-
could be incorporated within all but strong embodiment posi- rent ratings. This approach might yield a more comprehensive set
tions. However, given the published literature on the topic of of “feature” information about concrete concepts, the richness of
embodiment and our clinical experience working with seman- which would be more suitable for comparison of concepts across
tic dementia patients, our working assumption is that conceptual the entire concreteness spectrum.
knowledge does require some form of abstract representation (in Fourth, differences likely exist between the dimensions rated.
line with the disembodied and grounding by interaction posi- For example, the dimension labels used were deliberately non-
tions). The corresponding working hypothesis is that the types technical lay terms (e.g., social interaction) so the directness of
of information discussed in the current study (e.g., emotion, the mapping between the labeled dimension and the type of
social interaction, quantity, polarity) influence the acquisition information to which it was intended to refer may vary between
and organization more than the retrieval of abstract conceptual dimensions. Naturally the list of dimensions employed in the
knowledge. study was also not exhaustive with, for example, no explicit ref-
Several caveats and questions regarding the ACF methodol- erence to episodic memory. It has also been suggested recently
ogy should be raised. First, the cognitive dimensions outlined that abstract concepts may also depend in part upon brain circuits
here (e.g., quantity, polarity) are not equivalent to the sensory involved in introspection (Van Overwalle and Baetens, 2009; see
modalities referenced in strong embodiment theories (e.g., vision, Kiefer and Pulvermüller, 2012), which may relate to one or more
audition), in that they reflect secondary or higher-order asso- the rating dimensions used in the current study.
ciative processing of information acquired through the primary Fifth, the ACF rating approach was developed to examine the
sensory modalities. The activity of these brain networks may not notion that domains of cognition beyond the realm of senso-
constitute “embodiment” in the literal sense described for the rimotor and emotional processing may play an important role
sensorimotor networks, but two points are of relevance here. in the acquisition and/or organization of conceptual knowledge.
The embodiment/disembodiment debate is not binary in nature However, the current study represents only one stage in the exam-
(Meteyard et al., 2012); “weak” embodiment positions have been ination of this broad hypothesis, namely evaluating whether the
advanced which highlight the contribution of emotion/affect ratings yield a viable metric of semantic distance between abstract
(Kousta et al., 2011), another form of higher order information word concepts. The data do not, and were not intended to,
whose acquisition and/or activation in response to internal and provide any direct (neural) evidence that the pattern of com-
external stimuli is often mediated by primary sensory systems. prehension performance observed in SKO is linked causally or
In addition, non-embodiment theorists argue that much of the non-causally to the activation of these cognitive systems.
evidence cited in favor of embodied cognition in fact reflects One final point worth clarifying is that we regard the “feature”-
interactions not with primary sensory cortices but higher-order based similarity data presented in the current paper to be com-
polymodal cortices (Bedny and Caramazza, 2011). In the light of plementary to rather contradictory of previous claims about the
these two lines of (opposing) argument, the weak embodiment relatively greater importance of association than similarity for
position could potentially incorporate other types of informa- abstract words (e.g., Crutch and Warrington, 2005, 2010b; Crutch
tion outside of the primary senses (e.g., magnitude). Alternatively, and Jackson, 2011). From the outset, the theory of qualitatively
under a more disembodied framework, these additional cogni- different representational frameworks was proposed to describe
tive dimensions could be regarded as influencing the organization a relative rather than absolute distinction between the qualita-
of conceptual knowledge during acquisition and interacting with tive representational structures supporting abstract and concrete
conceptual representations when activated. words. As stated in the Introduction, to understand abstract
Second, as noted above, the ACF ratings are for individ- conceptual knowledge we need not only to investigate the rela-
ual words rather than word pairs, yielding the advantages of tionships between abstract concepts but also to explore of what
context-independence and greater flexibility. However, many those different individual concepts are composed. To that end,
words have different meanings (homonymy) and/or senses (pol- the current study builds on a small number of previous attempts
ysemy), and no precise definition was provided to control partic- to directly compare the features of abstract and concrete words
ipants. Consequently participants may have had slightly different (e.g., Wiemer-Hastings and Xu, 2005; Connell and Lynott, 2012).
meanings in mind when rating each item. As a result the posi- After all, whilst the meaning of abstract words may be shaped by
tion of each word within the high-dimensional space should be the context in which they occur, they may also be understood in
regarded as an estimate of the “true” locus of each homony- isolation or in unfamiliar or incompatible contexts.
mous/polysemous word, and the distance between pairs of words
may have greater validity for some meanings than others. ACKNOWLEDGMENTS
Third, concrete semantic space remains rather under- We thank SKO for his time, effort and patience, and are grate-
elaborated owing to (deliberate) selection of dimensions likely to ful to Dr. Jenny Crinion and Prof Cathy Price for referring
pertain to abstract concepts (see Figure 3). Previous feature gen- SKO and providing the imaging data. This work was under-
eration studies have highlighted a number of different types of taken at UCLH/UCL who received a proportion of funding from
the Department of Health’s NIHR Biomedical Research Centres Wellcome Trust 091593/Z/10/Z. This work was also supported
funding scheme. The Dementia Research Centre is an Alzheimer’s by an Alzheimer’s Research UK Senior Research Fellowship to
Research UK Co-ordinating Centre and has also received equip- Sebastian J. Crutch, a Medical Research Council Fellowship to
ment funded by Alzheimer’s Research UK. The Wellcome Trust Gerard R. Ridgway (MR/J014257/1), and a US Public Health
Centre for Neuroimaging is supported by core funding from the Service Grant to Jamie Reilly (NIH DC010197).
REFERENCES Coltheart, M. (1981). The MRC psy- Dove, G. (2011). On the need for induction, and representation
Allman, M. J., and Meck, W. H. cholinguistic database. Q. J. Exp. embodied and dis-embodied cog- of knowledge. Psychol. Rev. 104,
(2012). Pathophysiological dis- Psychol. 33A, 497–505. nition. Front. Psychol. 1:242. doi: 211–240.
tortions in time perception and Connell, L., and Lynott, D. (2012). 10.3389/fpsyg.2010.00242 McKenna, P., and Warrington, E. K.
timed performance. Brain 135, Strength of perceptual experience Gainotti, G., Ciaraffa, F., Silveri, M. (1978). Category-specific naming
656–677. predicts word processing perfor- C., and Marra, C. (2009). Mental preservation: a single case study.
Andrews, M., Vigliocco, G., and mance better than concreteness representation of normal subjects J. Neurol. Neurosurg. Psychiatry 41,
Vinson, D. (2009). Integrating or imageability. Cognition 125, about the sources of knowledge in 571–574.
experiential and distributional data 452–465. different semantic categories and McRae, K., Cree, G. S., Seidenberg,
to learn semantic representations. Cree, G. S., and McRae, K. (2003). unique entities. Neuropsychology 23, M. S., and McNorgan, C. (2005).
Psychol. Rev. 116, 463–498. Analyzing the factors underlying 803–812. Semantic feature production norms
Baayen, R. H., Piepenbrock, R., and van the structure and computation of Garcea, F. E., and Mahon, B. Z. for a large set of living and nonliv-
Rijn, H. (1993). The CELEX Lexical the meaning of chipmunk, cherry, (2012). What is in a tool concept? ing things. Behav. Res. Methods 37,
Databse (CD-ROM). Philadelphia, chisel, cheese, and cello (and many Dissociating manipulation knowl- 547–559.
PA: University of Pennsylvania. other such concrete nouns). J. Exp. edge from function knowledge. Meier, B. P., and Robinson, M. D.
Barsalou, L. W. (1999). Perceptual sym- Psychol. 132, 163–201. Mem. Cogn. 40, 1303–1313. (2004). Why the sunny side is
bol systems. Behav. Brain Sci. 22, Crutch, S. J., and Jackson, E. C. Garrard, P., Lambon Ralph, M. A., up: association between affect and
577–609. (2011). Contrasting graded effects Hodges, J. R, and Patterson, K. vertical position. Psychol. Sci. 15,
Barsalou, L. W., and Wiemer-Hastings, of semantic similarity and asso- (2001). Prototypicality, distinc- 243–247.
K. (2005). “Situating abstract con- ciation across the concreteness tiveness, and intercorrelation: Meteyard, L., Cuadrado, S. R.,
cepts,” in Grounding Cognition: The spectrum. Q. J. Exp. Psychol. 18, analyses of the semantic attributes Bahrami, B., and Vigliocco, G.
Role of Perception and Action in 1–21. of living and nonliving con- (2012). Coming of age: a review
Memory, Language, and Thought, Crutch, S. J., Randlesome, K., and cepts. Cogn. Neuropsychol. 18, of embodiment and the neuro-
eds D. Pecher and R. Zwaan (New Warrington, E. K. (2007). The 125–174. science of semantics. Cortex 48,
York, NY: Cambridge University variability of country map knowl- Gibbon, J., Church, R. M., and Meck, 788–804.
Press), 129–163. edge in normal and aphasic W. H. (1984). Scalar timing in mem- Moll, J., Zahn, R., de Oliveira-souza,
Bedny, M., and Caramazza, A. (2011). subjects: evidence from two new ory. Ann. N.Y Acad. Sci. 423, 52–77. R., and Krueger, F. (2005). Opinion:
Perception, action, and word mean- category-specific screening tests. Hauk, O., Johnsrude, I., and the neural basis of human moral
ings in the human brain: the case J. Neuropsychol. 1, 171–187 Pulvermuller, F. (2004). cognition. Nat. Rev. Neurosci. 6,
from action verbs. Ann. N.Y. Acad. Crutch, S. J., and Warrington, E. K. Somatotopic representation of 799–809.
Sci. 1224, 81–95. (2003). Spatial coding of semantic action words in human motor Newcombe, P. I., Campbell, C.,
Bloom, L. (1998). Language devel- information: knowledge of country and premotor cortex. Neuron 41, Siakaluk, P. D., and Pexman, P.
opment and emotional expression. and city names depends on their 301–307. M. (2012). Effects of Emotional
Pediatrics 102, 1272–1277. geographical proximity. Brain 126, Hoffman, P., and Lambon Ralph, and sensorimotor knowledge
Borghi, A. M., Flumini, A., Cimatti, 1821–1829. M. A. (2012). Shapes, scents in semantic processing of
F., Marocco, D., and Scorolli, C. Crutch, S. J., and Warrington, E. and sounds: quantifying the full concrete and abstract nouns.
(2011). Manipulating objects and K. (2005). Abstract and concrete multi-sensory basis of conceptual Front. Hum. Neurosci. 6:275. doi:
telling words: a study on con- concepts have structurally different knowledge. Neuropsychologia 51, 10.3389/fnhum.2012.00275
crete and abstract words acqui- representational frameworks. Brain 14–25. Pecher, D., Boot, I., and van Dantzig, S.
sition. Front. Psychol. 2:15. doi: 128, 615–627. Kiefer, M., and Pulvermüller, F. (2011). “Abstract concepts: sensory-
10.3389/fpsyg.2011.00015 Crutch, S. J., and Warrington, E. K. (2012). Conceptual representations motor grounding, metaphors,
Brown, W. P., and Ure, D. M. N. (2010a). Spatially coded seman- in mind and brain: theoretical and beyond,” in The Psychology of
(1969). Five rated characteristics of tic information about geograph- developments, current evidence Learning and Motivation, Vol. 54, ed
650 word association stimuli. Br. J. ical terms. Neuropsychologia 48, and future directions. Cortex 48, B. Ross (Burlington, VT: Academic
Psychol. 60, 223–250. 2120–2129. 805–825. Press), 217–248.
Buhrmester, M., Kwang, T., and Crutch, S. J., and Warrington, E. K. Kousta, S. T., Vigliocco, G., Vinson, D. Rolls, E. T. (2000). The orbitofrontal
Gosling, S. D. (2011). Amazon’s (2010b). The differential depen- P., Andrews, M., and Del Campo, cortex and reward. Cereb. Cortex 10,
mechanical turk. Perspect. Psychol. dence of abstract and concrete E. (2011). The representation 284–294.
Sci. 6, 3–5. words upon associative and of abstract words: why emotion Schwanenflugel, P. J., Akin, C. E., and
Cappelletti, M., Butterworth, B., and similarity-based information: matters. J. Exp. Psychol. 140, Luh, W. (1992). Context availabil-
Kopelman, M. (2006). The under- complementary semantic interfer- 14–34. ity and the recall of abstract and
standing of quantifiers in seman- ence and facilitation effects. Cogn. Kousta, S.-T., Vinson, D. P., and concrete words. Mem. Cogn. 20,
tic dementia: a single-case study. Neuropsychol. 27, 46–71. Vigliocco, G. (2009). Emotion 96–104.
Neurocase 12, 136–145. Crutch, S. J., Williams, P., Ridgway, words, regardless of polarity, Schwanenflugel, P. J., and Shoben,
Chatterjee, A. (2008). The neural orga- G. R., and Borgenicht, L. (2012). have a processing advantage over E. J. (1983). Differential context
nization of spatial thought and The role of polarity in antonym neutral words. Cognition 112, effects in the comprehension of
language. Semin. Speech Lang. 29, and synonym conceptual knowl- 473–481. abstract and concrete verbal mate-
226–238. edge: evidence from stroke apha- Landauer, T. K., and Dumais, S. rials. J. Exp. Psychol. Learn. Mem.
Chatterjee, A. (2010). Disembodying sia and multidimensional ratings T. (1997). A solution to Plato’s Cogn. 9, 82–102.
cognition. Lang. Cogn. 2, of abstract words. Neuropsychologia problem: The Latent Semantic Schwanenflugel, P. J., and Shoben, E.
79–116. 50, 2636–2644. Analysis theory of the acquisition, J. (1985). The influence of sentence
constraint on the scope of facilita- Troche, J., Paris, A., Crutch, S., and abstract and concrete concepts. feature ratings: the role of emotion, mag-
tion for upcoming words. J. Mem. Reilly, J. (2012). “Hierarchical Cogn. Sci. 29, 719–736. nitude, and other cognitive domains in
Lang. 24, 232–252. organization of abstract nouns: the organization of abstract conceptual
Scorolli, C., Jacquet, P. O., Binkofski, implications for neurolinguistic Conflict of Interest Statement: The knowledge. Front. Hum. Neurosci. 7:186.
F., Nicoletti, R., Tessari, A., and theory,” in Poster Presented at authors declare that the research doi: 10.3389/fnhum.2013.00186
Borghi, A. M. (2011). Abstract the Neurobiology of Language was conducted in the absence of any Copyright © 2013 Crutch, Troche,
and concrete phrases processing Conference (San Sebastian). commercial or financial relationships Reilly and Ridgway. This is an open-
differentially modulates cortico- Van Overwalle, F., and Baetens, K. that could be construed as a potential access article distributed under the terms
spinal excitability. Brain Res. 1488, (2009). Understanding others’ conflict of interest. of the Creative Commons Attribution
60–71. actions and goals by mirror and License, which permits use, distribution
Stuss, D. T., Shallice, T., Alexander, mentalizing systems: a meta- Received: 07 January 2013; accepted: 24 and reproduction in other forums, pro-
M. P., and Picton, T. W. (1995). A analysis. Neuroimage 48, April 2013; published online: 23 May vided the original authors and source
multidisciplinary approach to ante- 564–584. 2013. are credited and subject to any copy-
rior attentional functions. Ann. N.Y. Wiemer-Hastings, K., and Xu, X. Citation: Crutch SJ, Troche J, Reilly J and right notices concerning any third-party
Acad. Sci. 769, 191–211. (2005). Content differences for Ridgway GR (2013) Abstract conceptual graphics etc.
APPENDIX
Appendix 1. The wording and anchor points for the 7-point Likert Scales used to rate the target words on each of the 12 dimensions.
Parameter Definition