3

Evaluating Arguments

In view of the large number of shapes and sizes that arguments can take, it
should come as no surprise that evaluating them can be a challenging task.
On the other hand, if one accepts the claim that the main purpose of most
arguments is to influence beliefs and/or behavior, the practical challenge
becomes that of identifying variables that determine the effectiveness with
which arguments serve that function. This chapter and the following two
represent an attempt to do that.
Until relatively recently, studies of argumentation dealt primarily with
formal logic and deductive reasoning. Robert Trapp, Julie Yingling, and
James Wanner (1986) describe an expansion of the subject in more recent
times to make it relevant to the everyday activities of ordinary people. They
note that one of the consequences of this expansion was to complicate the
task of argument evaluation. “When formal logic was the standard against
which to judge the validity of arguments, argumentative competence was
non-problematic. A competent arguer was one who produced arguments
that conformed to formal logic” (p. 253). Trapp et al. point out that
rejecting formal logic as the sole standard for judging arguments necessi-
tates the development of a new standard for judging argumentative
competence.
There appears to be a growing awareness that there is much more
to being an effective reasoner in everyday life than being able to
distinguish between valid and invalid logical forms. Many, if not
most, of the arguments one encounters do not fit any canonical
pattern. They tend to be extended and to include deductive and
inductive (including abductive) components, to depend on unstated
assumptions, to assume – wittingly or not – a particular knowledge
base on the part of the listener or reader, to contain ambiguities and
obscurities, and to make use of various alogical stratagems, about which
more will be said later.

70

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 Evaluating Formal Arguments 71
Evaluating Formal Arguments
In logic what is needed is to mortify, not to inflame, the passion for
high generality
(Stove, 1986, p. 130).

In most of the practical reasoning situations that we encounter in our daily

lives, we are likely to be more interested in forming an opinion about the
truth or falsity of the conclusion an argument is intended to support than
about the logical validity of the argument. Before one should feel com-
pelled, on the strength of a deductive argument, to accept its conclusion as
true, one should be convinced both of the validity of the argument’s form
and of the truth of its premises. Thus, evaluating a deductive argument for
practical purposes involves more than logic; it requires also judging the
plausibility of claims.
It may also sometimes require that arguers agree on how to interpret
ambiguous terms or conflicting claims that cannot be resolved by
consideration of objective facts. The point is illustrated by an argu-
ment between Galileo and Tolomeo Nozzolini regarding whose esti-
mate of the worth of a horse, which was really worth 100 crowns, was
most in error: that of one who estimated 1,000 crowns or that of one
who estimated ten.
Sometimes a distinction is made between validity and soundness.
A deductive argument is valid if it has a correct form; it is said to be
sound if it has a correct form and its premises (and hence its conclu-
sion) are true. In everyday reasoning, soundness is generally of greater
concern than validity; what one wants to know is whether to believe an
argument’s conclusion and knowing only that the argument is valid
does not ensure that its conclusion is true. As Manfred Kienpointner
(1986) puts it, “the soundness of argumentation can’t be judged inde-
pendently of semantic and pragmatic standards underlying the argu-
mentation” (p. 279).
The top-level rules for evaluating deductive arguments are simple to
state: if the form of the argument is valid and the premises are true, the
conclusion must be true. If either the form is invalid or at least one of the
premises is false, the truth value of the conclusion is undetermined. In
evaluating such arguments, therefore one has to consider questions of both
form and content. The following arguments illustrate possible combina-
tions of valid and invalid forms with true and false premises and
conclusions.

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 72 Evaluating Arguments
• Valid argument with false major premise and false conclusion: “If tigers
eat meat, they are herbivores. Tigers eat meat. Therefore, they are
herbivores.”
• Valid argument with false premises and true conclusion: “If tigers are
herbivores, they eat meat. Tigers are herbivores. Therefore, they eat
meat.”
• Invalid argument with false minor premise and false conclusion: “If
tigers are herbivores, they eat grass. Tigers eat grass. Therefore, they are
herbivores.”
• Invalid argument with false premises and true conclusion: “If tigers are
carnivores, they eat grass. Tigers eat grass. Therefore, they are
carnivores.”
• Invalid argument with true premises and true conclusion: “If tigers are
carnivores, they eat meat. Tigers eat meat. Therefore, they are
carnivores.”

The Role of Form

The question of the logical validity of a deductive argument is a question of
form. If and only if the form is valid, the conclusion follows from the
premises. In judging a deductive argument to be valid, one is not guaran-
teeing the truth of its conclusion, one is guaranteeing only that the
conclusion is true if the premises are true – which is not to say that the
conclusion necessarily is false if one or more of the premises are false,
although it may be. Charles Hamblin (1970) contends that the claim,
which one finds in textbooks on logic, that a valid argument may be
built on false premises and may thus have a false conclusion, is
a “complete misrepresentation of the nature of argument” (p. 232). As
I understand his contention, the objection is to the idea that any argument
with false premises should be considered a good argument, even if the
argument’s conclusion follows from its premises; “whatever the textbooks
say, in practice we like our premises to be true, and we do not describe an
argument as a good one if the premises are false” (p. 232).
Hamblin (1970) reminds us that in assessing an argument, one does so
from some point of view – possibly that of the arguer, or that of the one(s)
to whom the argument is addressed, or one’s own as a spectator, or some
mixed point of view. “[A] logically-minded onlooker who judges ‘That
argument is valid’ will frequently mean ‘If I accepted those premises and

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 The Role of Form 73
did not accept that conclusion, that argument would persuade me’”
(p. 245).
We may sum up the relationship between validity and truth of formal
arguments thus: If the form is valid and the premises are true, the conclu-
sion must be true. If the form is valid and one or more of the premises are
false, the conclusion could be true or false. The only thing that is ruled out
is an argument with a valid form having true premises and a conclusion
that is false. “The simplest of all demonstrations that an argument is
invalid – and hence, perhaps, equivocal – is the demonstration that its
premises are true and its conclusion false” (Hamblin, 1970, p. 293).
A person’s ability to distinguish between valid and invalid deductive
forms can be tested by using arguments that are devoid of semantic
content. Consider the following arguments:
All bletes are crogs
All crogs are trons
Therefore, all bletes are trons.

No bletes are crogs

No crogs are trons
Therefore, no bletes are trons.
We need not know anything about bletes, crogs, or trons to recognize
that the first argument is logically valid, and therefore if its premises are
true then its conclusion must be true also, whereas the second one is
invalid, so being assured that its premises are true would not let one
conclude that its conclusion is true.
I suspect that most people will find it easier to acknowledge that the first
argument is valid than to agree that the second one is not. Perhaps if we
substitute for bletes, crogs, and trons, respectively, Scandinavians, Chinese,
and Swedes, we will find the invalidity easier to see.
No Scandinavians are Chinese
No Chinese are Swedes
Therefore, no Scandinavians are Swedes.
In neither of the invalid cases does the conclusion follow from the
premises. In evaluating the argument regarding bletes, crogs, and trons,
one can easily be misled by the fact that it has the structure of “All As are Bs
and all Bs are Cs; therefore, all As are Cs,” which is a valid argument.
However, “No As are Bs and no Bs are Cs; therefore no As are Cs” may
look valid, but it is not, which is shown by the fact that it is possible to

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 74 Evaluating Arguments
construct an argument in this form (e.g., No Scandinavians are
Chinese . . .) that has true premises but a false conclusion.
The following is an example of a formal deductive argument taken from
a logic textbook (Searles, 1948):
Nothing intelligible ever puzzles me
Logic puzzles me
Therefore, logic is unintelligible.
This is the modus tollens form of the conditional syllogism:
If A then B
Not B
Therefore not A.
Expressed in canonical modus tollens form, Searles’s example, which
came originally from Lewis Carroll, could be stated:
If it is intelligible, it does not puzzle me
It (logic) puzzles me
Therefore, it (logic) is not intelligible.
The following argument has the same form:
If he were really sympathetic to the demands of the coal miners, he
would have voted for the XYZ bill
He voted against the bill
So, he is not sympathetic to the miners’ demands.
Much of the empirical literature on argument evaluation focuses on
properties of arguments that make them more or less difficult for people to
evaluate. Difficulty is generally measured in terms of the percentage of
participants in an experiment who correctly judge an argument to be valid
or invalid, the time required to make such judgments, and difficulty ratings
on the judgments made (Rips, 1995). Models of human reasoning that
assume people reason deductively by applying rules tend to predict argu-
ment difficulty will depend on the number and complexity of the rules that
must be applied in specific instances (Braine, Reiser, & Rumain, 1984;
Rips, 1994). Lance Rips (1994) notes too that “attributing errors to subjects
can be a delicate matter that depends on how we justify the logical systems
themselves” (p. 378). A distinction may be made between the need to apply
a given rule several times and the need to apply several different rules, and
the latter is generally seen to make for greater difficulty than the former.

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 The Role of Content 75
In general, modus ponens inferences – If A then C; A; therefore C – have
been shown to be easier for people to make than modus tollens inferences –
If A then C; not C; therefore not A (Marcus & Rips, 1979; O’Brien, 1973;
Rips & Marcus, 1977; Rumain, Connell, & Braine, 1983; Wason &
Johnson-Laird, 1972). People have even greater difficulty with modus tollens
arguments when the antecedent in the conditional proposition is negative
(If not A then C; If not A then not C) (Evans 1977a; Pollard & Evans, 1980;
Roberge, 1978). Evans (1982) concludes from a review of relevant research
that people are competent to reason from antecedents to consequents but
not vice versa, and that “abstract conditional reasoning performance
indicates no more than a superficial understanding of the sentence If
P then Q, and little evidence of any depth of reasoning” (p. 231).
That people find it easier to deal with the modus ponens than with the
modus tollens form of argument is true when the conditional premise is
expressed in the form If A then C. The difference diminishes or disappears,
however, when it is expressed in the logically equivalent form A only if
C (Evans, 1977a; Roberge, 1978). With this form, modus ponens and modus
tollens arguments are about equally difficult, and both are more difficult
than modus ponens when expressed in the conventional If A then C form.
Philip Johnson-Laird and Ruth Byrne (1991) interpret this difficulty pat-
tern as supportive of the assumption that people solve deductive reasoning
problems by constructing mental models of the premises, because the
A only if C form prompts the development of two models for either
a modus ponens or modus tollens argument, whereas, the modus ponens
argument with If A then C requires only that a single model be used.
A variety of representational systems, including truth tables, Euler
diagrams, Venn diagrams, and mental-model diagrams, can often be help-
ful in evaluating the formal validity of arguments (Garnham & Oakhill,
1994; Johnson-Laird & Byrne, 1991; Nickerson, 1986b), although their
effective use is not trivially straightforward.

The Role of Content

[I]f you want the truth rather than merely something to say, you will
have a good deal less to say
(Nagel, 1986, p. 9).

The importance of content in the evaluation of deductive arguments is

sometimes overlooked. The emphasis typically is on form, which is appro-
priate but not to the exclusion of substance if the question is whether the

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 76 Evaluating Arguments
argument’s conclusion should be accepted. But results from a considerable
body of research indicate that when deductive arguments have semantic
content, people typically find it difficult to ignore that content in judging
whether they are formally valid (Nickerson, Butler, & Barch, in press;
Staudenmayer, 1975; Wason & Johnson-Laird, 1972). To be sure, one can
evaluate the logical validity of a deductive argument without reference to
its substance, but because of the ambiguous nature of language, logical
validity, by itself, is not enough to compel one to believe the conclusion to
an argument. Hamblin (1970) makes this point and punctuates it with
a quote from the third century philosopher Sextus Empiricus:
For just as we refuse our assent to the truth of the tricks performed by
jugglers and know that they are deluding us, even if we do not know how
they do it, so likewise we refuse to believe arguments which, though
seemingly plausible, are false, even when we do not know how they are
fallacious . . . if there should be an argument which leads us to a confessedly
absurd conclusion, we shall not assent to the absurdity just because of the
argument but avoid the argument just because of the absurdity (p. 96).

Consider the following deductive argument:

Nothing is better than eternal bliss.
Cold soup is better than nothing.
Therefore, cold soup is better than eternal bliss.
It clearly is valid; but equally clearly unpersuasive. What is wrong with it?
The answer is that “nothing” is ambiguous and can take on different
meanings in different contexts. In this argument, it has a meaning in the
first premise that differs from the meaning it has in the second one. The
difference is easily illustrated: I ask you, what would you like to talk about.
You say “nothing.” Are you telling me that you would like to talk about the
concept nothing? Or that you would like not to talk?
Nothing is illustrative of concepts that can have more than one meaning
depending on the context in which they occur and can wreak havoc if the
effect of context is not recognized. (See fallacy of equivocation in Chapter
8.) Anthony Flew (1968) considered the fact that such expressions exist to
have been a major discovery.
We might say that “This is past” and “This is red,” “It goes on to London”
and “It goes on to infinity,” and “Nobody came” and “Somebody came” are
pairs of grammatically similar expressions. But the members of all these pairs
of grammatically similar expressions are logically very different. That is to
say, that nothing but nonsense and paradox will result if we ask questions

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 The Role of Content 77
about one assertion which are only appropriate and significant when asked
about the other. It would be absurd, but it would also be easy, to be misled
by the grammatical similarity of “It goes on to London” to “It goes on to
infinity” into the misconception that “Infinity” like “London” refers to
a place, albeit a very queer and very inaccessible place. It is absurd, but to
some people it is also easy, to be misled by the grammatical similarity of
“Somebody came” to “Nobody came” into the misconception that
“Nobody” refers to a person as does “Somebody” (p. 7).
Although most writers emphasize the importance of form in evaluating
deductive arguments, Stove (1986) insists that the validity, or invalidity, of
arguments cannot be judged on purely formal grounds. He does not deny
that some formal arguments are valid, and others are not, but he makes
a distinction between “being valid” and “being able to be proved valid.”
Proving the validity of a form is tricky business, in his view. “Purely formal
judgments of validity, if they employ a predicate variable or a propositional
variable to pick out the class of arguments in question, are all or almost all
exposed to counter-examples, direct or indirect, or they generate para-
doxes” (p. 124). Conversely, it does not follow from the fact that a form of
argument cannot be proved to be valid that it is necessarily invalid, that it is
impossible to find valid instances of it. “There are few or no logical forms,”
Stove claims, “above a low level of generality, of which every instance is
valid: nearly every such supposed form has invalid cases or paradoxical
cases. The natural conclusion to draw is that formal logic is a myth, and
that over validity, as well as over invalidity, forms do not rule: cases do”
(p. 127).
The point that Stove is making is illustrated by the syllogistic form: A x’d
B. B is a C. Therefore, A x’d a C. This seems a perfectly valid form, but
consider the following variation on an instantiation of it offered by Willard
Quine (1960): Jack married Jill. Jill is dead. Therefore, Jack married a dead
woman. One might argue that the abstract form obviously is valid only on
certain assumptions about the nature of the entities that the symbols (A,
B and C) can represent. But was this obvious before it was made so by the
nonsensical concrete illustration? And who is to say that the presumed
validity of other (presumably valid) abstract forms does not similarly rest
on unstated assumptions the necessity of which remains to be revealed
through the discovery of concrete instantiations that make no sense?
Stove does not deny that it is possible to make true judgments of logical
validity and invalidity; he denies that such judgments can be made on the
basis of the formal properties of arguments alone. One is entitled, he
claims, to judge a particular argument to be valid (or invalid) without

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 78 Evaluating Arguments
being obliged to judge the next argument of the same form that one
encounters to be valid (or invalid) as well. This is not, in his view, cause
for concern: “to despair of the possibility of formal logical knowledge does
not at all require that we despair of the possibility of logical knowledge”
(p. 129).
In the foregoing, I have made the common distinction between validity
and truth and associated validity with logical form and truth with factual
reality. The two concepts are linked in deductive logic by the claim that if
the form of a deductive argument is valid and the premises of the argument
are true, the argument’s conclusion must be true. If either of the conditions
is not met – the form is invalid or one or more of the premises is untrue –
the truth of the conclusion is not assured. I have now made explicit an
additional requirement that must be met in order to assure that a deductive
argument in a valid form yields a conclusion that is true. That is, that the
same words appearing in the premises must have the same meaning in both
cases. My sense is that people generally assume this to be the case and, to
the extent that they think about it at all, considerate it to be sufficiently
obvious to not require comment.
Consider again the argument:
All bletes are crogs
All crogs are trons
Therefore, all bletes are trons.
Suppose I discover that there are such things as bletes, crogs, and trons.
Suppose further that I discover a blete that is not a tron, which is to say
I discover that the conclusion of the above syllogism is false, that it is not
a true description of nature. I then know that at least one of the premises of
the argument is also false. But what gives me the right to say I know this?
Note that I am claiming to know something about nature, not just about
logic. What gives me this right is the assumption – deep conviction – that
nature is logically consistent, that empirical truth harbors no logical
contradictions.
That true premises cannot yield a false conclusion in a valid deductive
argument is a thought-provoking fact, telling us, as it does, something
about the nature of truth and reality. A related thought-provoking
thought, expressed by William James (1983/1890) in the chapter on reason-
ing in Principles of Psychology, is that reasoning would probably be impos-
sible if the world did not behave lawfully. As already noted, the constraint
that the conclusion must be true if the premises are true is the only one that
holds. We cannot say, for example, that if at least one of the premises is

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 The Role of Content 79
false the conclusion is necessarily false, or that if the conclusion is true, all
the premises are necessarily true. Henry may well be a stuffed shirt, even if
one or both of the premises of the following argument is false.
All logicians are stuffed shirts.
Henry is a logician.
So, Henry is a stuffed shirt.
The question of which is the more fundamental concept – validity or
truth – is an interesting one, but one we are not likely to be able to answer
in an unequivocal way, although it seems safe to say, as already noted, that
people generally are likely to be more concerned about whether a particular
assertion is true than whether it is the conclusion of a logically valid
argument. Whether, upon discovering that the conclusion of an argument
that one believed to be valid in form and with true premises turned out to
be false, one would be more likely to question the validity of the form or
the truth status of premises is an interesting question. My guess is that one
would be more likely to do the latter, but perhaps it would depend on
specifics: the logical form involved and the nature of the premises.
To argue that validity or invalidity cannot be judged on purely formal
grounds is not to say that all logical truths are empirically based. Following
Rudolf Carnap, Stove (1986) argues that deductive logic, no less than
inductive logic, rests ultimately on intuition. Consider the assertion
“‘Abe is black and Bob is black’ entails ‘Abe is black’” or the syllogistic
claim “‘Abe is a raven and all ravens are black’ entails ‘Abe is black’.” If one
does not simply see the truth of these claims, Stove argues, there is nothing
that can be done to convince one of it; if one does not know these truths
a priori, one cannot learn them from experience.
This view is contrasted by Stove (1982) with that of Bernard Bolzano
and Carl Popper, both of whom defended the idea that deductive logic
has an empirical basis. One discovers the invalidity of a faulty deductive
argument, according to these writers, by finding a counterexample to
it – an instance of the argument with true premises and a false con-
clusion. The validity of a deductive argument is established as
a consequence of no counterexamples having been found despite con-
certed efforts to find one. Stove (1986) holds that every normal human
being has an immense amount of knowledge of what does and what
does not follow from what: “every normal human being is, in the
extent of his knowledge of such truths, a millionaire” (p. 128). As to
how much of this knowledge could have been gained from experience,
Stove argues that relatively little of it could.

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 80 Evaluating Arguments
Consider first the case of invalid arguments. We could learn from
experience that an argument was invalid if we knew its premises to be
true and we discovered its conclusion to be false, but there are countless
arguments that we know to be invalid that have both true premises and
true conclusions. Stove uses the example:
All men are mortal
Socrates is mortal
Socrates is a man.
The reader will undoubtedly agree that this is an invalid argument. But
suppose that someone insists that it is valid. How could that person be
shown empirically that it is not. One may become convinced that this
argument is invalid – say by seeing formally similar arguments with true
premises and false conclusions – but not by researching the empirical
claims of the specific argument in question. Nor can we learn from
experience the invalidity of an argument with a false premise and true
conclusion. In short, Stove argues, “[s]carcely any of the vast fund of
knowledge of invalidity which every human being possesses can be
acquired from experience” (p. 156).
With respect to our knowledge of validity, Stove makes a like claim. “No
singular judgment of validity . . . can be learnt from experience. And
a fortiori no general one can” (p. 158). He argues that the attempt to rest
our confidence in the validity of a particular schema, say the classical
Barbara form,
All A are B
All B are C
All A are C,
on observations of only truth-preserving instances of it involves one in an
infinite regress like that described by Lewis Carroll in his fable of the
tortoise and the hare. The use of an observation of an instance (I) of
a schema as a confirmation of the validity of that schema (S) assumes
a metaschema according to which I confirms S. And if all logic is to have an
empirical basis, the validity of the metaschema must itself be established
observationally, and so on ad infinitum.
Stove (1986) makes a further interesting observation about logicians’
stress on the importance of logical form. Logicians tend, he suggests, to
account for validity in terms of schemas or logical forms, whereas their own
arguments about validity typically are “constant free,” which is to say they
are arguments about what is or is not empirically true, composed of

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 The Role of Content 81
assertions whose subjects and predicates have real-world referents and not
abstractions. This being the case, he claims, their arguments cannot satisfy
their own requirements for validity.
The class of constant-free arguments may perhaps seem odd or unimpor-
tant. For this reason, I should point out that it includes most of the
arguments that philosophers and logicians employ, as distinct from those
which they have principally studied. A superficial reader might suppose that
philosophers and logicians are interested, and even interested to an inexplic-
able degree, in Socrates, or in his humanity or mortality, or in the mortality
of all men. But in fact, of course, they are not in the least interested in any of
these things. What they are interested in is, rather, individuals, properties,
existence, universality, necessity, possibility – and probability and relative-
truth-frequency! It is these things which their arguments are really about
(p. 173).
I have devoted considerable space to Stove’s treatment of formal argu-
ments as it pertains to the problem of evaluation because it represents what
strikes me as a relatively unorthodox view and makes a thought-provoking
case regarding the role of content in the evaluation of formal, or deductive,
arguments. I leave it to the reader to judge the persuasiveness of Stove’s
claims of the importance of content in the evaluation of deductive
arguments.
In any case, understanding the difference between logical validity and
empirical truth is critical to effective evaluation of formal arguments,
because such arguments can be evaluated from both perspectives. While
such an understanding is not enough to guarantee that one will always
avoid reasoning errors that stem from insensitivity to this distinction, it
should make such errors less likely and increase one’s ability to modify
one’s thinking appropriately when they are pointed out.
The great advantage of formal logic is its lack of ambiguity, for the most
part. It is a system of rules of inference that can be stated in relatively
straightforward ways. Whether a specific argument is consistent with the
rules, and is therefore valid, or violates them, and is therefore invalid, can
usually be determined, if not unequivocally at least adequately for practical
purposes.
A limitation of formal logic is that it works unambiguously only at the
level of form, which can best be represented by abstract symbols. As soon as
one attempts to replace the abstract symbols of a formal argument with
words, one necessarily must deal with semantics and things become much
less crisp. In order to evaluate a substantive argument – in the sense of
deciding whether to believe its conclusion – one must consider not only the

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 82 Evaluating Arguments
validity of the argument’s structure but what the premises mean. If a more
than passing acquaintance with the rules of formal logic is essential to good
reasoning, it must be considered only a necessary condition and not
a sufficient one.
Ensuring that the words used in an argument have the same meaning for
both, or all, of the arguers is a formidable undertaking. Most words have
many meanings, and even when the dictionary gives only a single meaning
for a word, how that word is interpreted may differ subtly with the context
in which it is used, and with the intentions of its user. The futility of trying
to ensure that there is no possibility that two evaluators of the same
statement (e.g., a premise of an argument) will interpret it in precisely
the same way was noted by Sidgwick (1884) this way.
It is obvious, further, that if there is to be an absolutely clear mutual
understanding between speaker and audience as to the meaning of every
separate word employed, and also of the assertion as a whole, a large part
of every speaker’s life would be occupied in the process of defining, or
explaining his statements, than could well be spared from his other
occupations; and that long before the meaning of most assertions
could be settled, their value as practical information would have passed
away (p.187).
This dilemma is generally avoided by simply assuming that most of the
time different people will give sufficiently close interpretations to the same
words to let meaningful communication occur, and insisting on, and
bickering about, definitions only in relatively rare cases.
On the other hand, there is the view that the importance of definitions is
generally not appreciated, and to our detriment. F. Max Müller (1887)
made the point: “Definition is the only panacea for all our philosophical
misery” (Appendix, p. 24). Müller (1887) maintained that all the words in
the English lexicon can be traced to about 800 roots, which, in turn, can be
reduced to about 120 concepts. “These one hundred and twenty concepts
are really the rivers that feed the whole ocean of thought and speech. There
is no thought that passes through our mind, or that has passed through the
minds of the greatest poets and prophets of old, that cannot directly or
indirectly be derived from one of these fundamental concepts” (p. 32).
Müller insisted that most of the misunderstandings among people are the
direct result of careless use of language: “All difficulties which visit us in the
various spheres of thought, whether scientific, historical, philosophical, or
religious, vanish as soon as we carefully examine the words in which we
think” (Müller, p. 78).

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 The Role of Content 83
As to the question of whether human reasoning is consistent with
the rules of deductive logic, one can only say that sometimes it is
and sometimes it isn’t. Johnson-Laird and Byrne (1991) put it this way:
“Our view of logical competence is that people are rational in principle,
but fallible in practice. They are able to make valid deductions,
and moreover they sometimes know that they have made a valid
deduction. They also make invalid deductions in certain circumstances”
(p. 19).
We know from a considerable body of research that when arguments
have semantic content, people often find it difficult to ignore that content
in judging whether they are formally valid (Evans, Barston, & Pollard,
1983; Evans, Newstead, & Byrne, 1993; Marcus & Rips, 1979; Markovits &
Nantel, 1989; Newstead et al., 1992; Oakhill & Johnson-Laird, 1985;
Quayle & Ball, 2000; Staudenmayer, 1975; Wason & Johnson-Laird,
1972). Judging an argument to be valid simply because its conclusion is
believed to be true is sometimes called the belief bias effect (Brisson et al.,
2014; Klauer, Musch, & Naumer, 2000) and has been defined as
a tendency of people “to rate an argument as valid if they think that the
conclusion is empirically true, and vice versa” (Markovits & Nantel, 1989,
p. 11), or “to accept or reject a conclusion on the basis of its consistency
with everyday knowledge, regardless of its logical status” (Dube, Rotello, &
Heit, 2010).
Although discussions of reasoning in argumentation are likely to
emphasize the importance of the concept of logical validity – it is easy to
show that whether an argument is valid is not the sole determinant of
whether it is persuasive. Nickerson et al. (in press) had college students
evaluate arguments with respect to both validity and persuasiveness.
Invalid arguments that were judged to be persuasive were almost three
times as likely to be considered to be valid as were invalid arguments that
were considered not to be persuasive. Here is an example of an invalid
argument that was considered by 85 percent of the subjects to be persuasive:
If John is in France, he is in Paris.
John is in Paris.
Therefore, he is in France.
The argument is invalid by virtue of affirming the consequent. The
following argument was considered by 80 percent of the subjects to be
persuasive, despite being invalid, in this case by virtue of denying the
antecedent:

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 84 Evaluating Arguments
If Jane has any pets, she has a dog.
Jane doesn’t have any pets.
Therefore, she doesn’t have a dog.
The degree to which formal logic is descriptive of normal untutored
human thought is a matter of long-standing controversy within psychology
and philosophy. Investigators have compiled long lists of how thinking
often is illogical. Some theorists have argued that thought is not greatly
constrained by principles of logic (Cheng & Holyoak, 1985; Evans, 1989;
Harman, 1986) or that even when it produces results that are consistent
with logic, those results may be based on operations other than logical ones
(Johnson-Laird, 1983). Others have taken the position that thinking is
basically logical and that what appears to be evidence of illogicality can
usually be traced to linguistic confusions or misinterpretations (e.g.,
Cohen, 1981; Henle, 1962).

Evaluating Informal Arguments

[U]nfortunately, there is no precise agreed use for the expression
“inductive argument,” nor any pair of adjectives similar to “valid”
and “invalid” for assessing inductive arguments
(Swinburne, 1974, p. 2).

In establishing the structure of an argument, we must interpret the

words of the speaker, supply the missing links, which is always very
risky
(Perelman & Olbrechts-Tyteca, 1969, p. 187).

Induction can only be defended by appeal to it

(Polkinghorne, 1989, p. 30).

As Alvin Goldman (1986) points out, not all the rules of belief justification
can be derived from deductive logic. “Where nondeductive inferences are
appropriate – and such cases are certainly legion – truths of deductive logic
are of little or no help” (p. 67). Whether the breach might be filled by
truths of inductive logic, Goldman sees as “very dubious.” “Many episte-
mologists and philosophers of science now question whether there is
a body of truths we can call ‘inductive logic’” (p. 67). However, inductive
arguments are ubiquitous, and we have to deal with them.
Despite their formality and the rules that determine their validity,
formal arguments can be difficult to evaluate. Informal arguments can be

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 Evaluating Informal Arguments 85
doubly so. The time-honored binary distinction between valid and invalid
arguments is widely considered to be too course to be very helpful in
evaluating informal arguments (Grennan, 1986; Rescher, 1976). One is
more likely to encounter argument evaluations in terms of their strength
(Hoeken, Š orm, & Schellens, 2014; Hoeken, Timmers, & Schellens, 2012),
quality (Hahn & Hornikx, 2016; Park et al., 2007) persuasiveness (O’Keefe,
2013), or some other property that admits off degrees. But, as Hahn and
Hornikx (2016) point out, “Despite its importance, and considerable
research effort aimed at the issue, a comprehensive theory of argument
quality has remained elusive” (p. 1834).
Stephen Thomas (1973) proposed that arguments be evaluated in terms of
levels of strength – weak, moderate, strong – marked at the extremes by nil and
deductively valid. Others have recognized, at least implicitly, that inductive
arguments seldom prove their conclusions to be true beyond doubt. Scriven
(1986) makes the point this way: “Argument will rarely result in the discovery
of the absolute truth; this is clear in science, and it should be clearer in non-
scientific discourse” (p. 13). From this perspective, he promotes the need to
seek a process that will produce the best conclusions. “To this end, it is
important to look beyond the content of the arguments to the process
whereby arguments are generated, developed and evaluated” (p. 13).
Rieke and Sillars (1975), who offer advice on how to compose, evaluate,
and refute arguments in a variety of contexts, describe how what they refer
to as an “audience-centered” theory of argumentation would be evaluated:
“The ultimate test of argument is whether or not people give their adher-
ence to claims made by the arguer by stating concurrence or behaving
accordingly. This understanding of why we advance claims and give
reasons for them dictates a distinctive understanding of argumentation”
(p. 2). The focus of this kind of theory is on “those arguments that win
adherents, and not on those that are ‘correct’ or ‘true’ or that ought to have
been selected” (p. 24).
Richard Swinburne (1974) suggests that although the terms “valid” and
“invalid” are not applicable to informal, or inductive, arguments, the
qualifications “correct” and “incorrect” are. In his view, a correct inductive
argument is one in which the premises “make it reasonable” for us to accept
the conclusion whereas an incorrect inductive argument is one in which
the premises do not make it reasonable for us to accept the conclusion, but
it is falsely claimed that they do. Stated in probabilistic terms, a correct
inductive argument is said to be one in which the premises “make the
conclusion more probable than any equally detailed rival, although the
argument is not deductively valid” (p. 5).

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 86 Evaluating Arguments
Max Black (1974/1958) draws a similar contrast between deductive and
inductive arguments, and contends that inductive arguments vary in
strength, whereas deductive arguments do not. “If a deductive argument
is not valid it must be invalid, no intermediate cases being conceivable . . ..
Appraisal of an inductive argument admits of degrees” (p. 129).
Unfortunately, if one accepts this position, one does not thereby free
oneself of conceptual difficulties, because philosophers can find much to
argue about around such concepts as reasonableness and probability.
Most “everyday” arguments are neither exclusively deductive nor
expressed in a canonical logical form. However, they typically do have
certain identifiable components. These include a conclusion and one or
more assertions that are offered, directly or indirectly, as support for it – as
reasons for believing it to be true. Informal arguments can be complex and
confusing, especially when several conclusions and supporting assertions
are run together in a random fashion. A complex argument may contain
assertions that are intended to support certain conclusions and that are
themselves conclusions supported by other parts of the argument.
Sometimes a distinction is made between an argument and the language
in which it is expressed. Ronald Fisher (2004/1988), for example, gives his
objective in The Logic of Real Arguments, as describing and demonstrating
“a systematic method for extracting an argument from its written context
and for evaluating it” (p. 128). Whether one finds this distinction
a compelling one, there can be little doubt that often the most challenging
aspect of evaluating an informal argument is figuring out exactly what its
various components are and what they are intended to mean.
The following, excerpted from Nickerson (1986a), are two informal
arguments taken from The Boston Globe published in 1982 in anticipation
of a referendum on a Massachusetts law requiring that a refundable deposit
be paid for certain beverage containers sold in the state. (A bottle bill – the
Beverage Container Recovery Law – had been enacted by the legislature
overriding the governor’s veto in 1981.) The first argument is from
a proponent of the bill; the second is from an opponent. A “yes” vote on
the referendum would keep the law in place; a “no” vote would do away
with it. The arguments were restricted in length.
Argument For:
The bottle bill was passed by two-thirds vote of the Legislature last year in
order to clean up the litter cluttering our lawns, streets, and parks. This
sensible legislation will put a stop to an enormous waste of money and
resources.

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 Evaluating Informal Arguments 87
Similar laws have proven popular and successful in achieving those aims
in states much as Maine, Vermont, Connecticut, and Michigan. New York
has passed a bottle bill which will take effect next July.
Among the benefits of the Massachusetts bottle bill:
An 80 percent decrease in beverage container litter, the most prevalent
and dangerous type;
Lower prices: A 5 percent reduction of beverage prices; Tax savings:
Millions of tax dollars saved due to reduced garbage collection and disposal
costs;
More jobs: A net increase of more than 2,000 skilled and unskilled jobs
for Massachusetts residents;
Less waste: 33 percent less energy consumed by the beverage industry.
Argument Against:
In this age of limited resources, it is understandable that Massachusetts
voters said “no” to forced deposit containers the only time they had an
opportunity to vote on the issue. They knew that requiring return of cans,
plastic bottles, and other containers would create many problems, and were
concerned about the need to:
Save town and private recycling operations; Save the state’s water supply
during a declared emergency;
Save local separation programs because landfill space is rapidly
disappearing;
But most importantly, save money for already over-burdened consumers.
Further, our position is that the will of the electorate is better served by
people voting in free elections than legislators acting in a highly political
environment. A “no” vote is a vote for an industry-funded litter control and
recycling law – which will give money to communities for cleanup and
recycling projects and anti-litter education programs.
The referendum was on the ballot in 1982 and the bottle bill survived by
a 60 percent to 40 percent margin. It would be interesting to know which,
if any, of the predictions offered in the pro and con arguments proved to be
true, but inasmuch as the bill has been modified more than once over the
years, it is unlikely that could be determined. It seems more likely that each
side would be able to argue that its position had been justified.
Here is another example of an informal argument, this one in support of
the further development and use of nuclear power in the United States:
That there is nothing inherently unworkable about nuclear power is borne
out by the success of nuclear projects overseas and indeed of many projects
in the U.S. Moreover, although a variety of alternative sources of energy are
available to utilities, and several promising new non-nuclear technologies
are under development, abandoning nuclear power might well render the
nation’s electricity supply substantially less efficient and environmentally

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 88 Evaluating Arguments
benign. Oil and gas, albeit currently plentiful, will eventually grow scarcer
and costlier. Coal is difficult to burn cleanly: acid rain and other by-
products of coal combustion are causing serious damage to the environment
and will be expensive to control, and in the future, the carbon dioxide
released by the combustion of fossil fuels may have a severe effect on global
climate. The potential of solar electric technology to compete economically
outside a fairly narrow range of favorable sites or specialized uses has not yet
been demonstrated (Lester, 1986, p. 31).

While informal, these arguments are still much neater than many of those
one encounters in daily life. The people who constructed them were moti-
vated to make them concise and compelling. Even so, they are not nearly as
easy to evaluate as are formal deductive arguments; and, as already noted, there
are no widely agreed-upon rules for doing the evaluation. To assess such
arguments one must, at least, recognize the claims that are being made, decide
how much credence to give to each of these claims, judge how much support
each of them – if true – gives to the conclusion or position the argument is
intended to substantiate, and somehow aggregate the results of these con-
siderations into an overall assessment of the persuasiveness of the argument as
a whole. In short, evaluation of informal arguments of any substance and
complexity involves at least the following components: analysis, judgments of
relevance and weight, and aggregation or synthesis.

Analysis
Figuring out what the essence of the argument is often requires analysis.
Informal natural-language arguments can be buried in an excess of verbiage
and the first challenge in evaluating them is to determine what they are – to
discover precisely what the argument is (Copi, Cohen, & McMahon,
2010). This is made the more difficult – and the more necessary – when
the argument is expressed poorly or its originator’s intentions are camou-
flaged by a superfluity of words, but in order to evaluate an argument one
must know, or make an assumption about, what the argument is. This
means, at least, identifying the conclusion(s) one is intended to draw and
what is being asserted in its (their) support.
Figuring out the essence of an argument, even when it is well stated, can
be a considerable challenge. This observation holds for short deductive
arguments as well as for long inductive ones. Consider the argument that
the second law of thermodynamics (essentially that in any closed system
randomness increases) implies a beginning for the universe. One statement
of it is as follows: “the universe cannot have existed forever, otherwise it

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 Analysis 89
would have reached its equilibrium endstate an infinite time ago.
Conclusion: The universe did not always exist” (Davies, 1983, p. 11).
Is the argument sound? Is the claim that if the universe had always
existed it would have reached its endstate (equilibrium, total randomness)
an infinite time ago true? Does the claim imply that the universe never
could have been in a state of disequilibrium (less-than-maximum-
randomness)? If it does imply that, then the second law of thermodynamics
does seem to entail a beginning. On the other hand, if it does not have that
implication – if the notion that the equilibrium endstate would have been
reached an infinite time ago is compatible with the idea that at some time
the universe contained some structure, it is not clear what rules out the
possibility that we are now living in that period. And so on.
Cassius Keyser (1926) notes the importance of “postulate detection” to
good thinking, not only by scientists or philosophers but by all people in
everyday life. This is because much of daily thinking has to do with the
acceptance or rejection of propositions. Arguments in defense of, or in
opposition to, a given proposition invariably make use of other proposi-
tions, some of which may be taken for granted and given the status of
postulates. Only by recognizing explicitly what are being taken as postu-
lates do we have the opportunity to decide whether we consider them to be
true or false. Unfortunately, the identification of arguments’ postulates is
often complicated by the existence of ambiguities and unstated assump-
tions (Nickerson, 1996).
Fisher (2004/1988) suggests that analyzing informal arguments can be
facilitated by learning to recognize certain words or phrases that typically
signal a conclusion (therefore, so, hence, thus, . . .) or a reason given in
support of a conclusion (because, for, since, follows from, . . .). He refers to
the first group as “conclusion indicators,” to the second as “reason indica-
tors,” and to the combination as “inference indicators” or “argument
indicators.” Fisher describes a procedure for evaluating written formal
arguments that begins with the identification of such indicator words
and includes identifying any conclusions that are drawn (perhaps flagged
by indicator words), identifying “reasons” that are given why one should
believe those conclusions, and so on.
So, there is much advice on record regarding how to go about analyzing
inductive arguments. Undoubtedly some, perhaps most, of that advice is
helpful; at the least it provides food for thought. But whereas there appears
to be general agreement that analysis is an important aspect of argument
evaluation, there is no algorithm that is universally – or nearly universally –
accepted as the best way to do it.

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 90 Evaluating Arguments
Judgments of Relevance and Weight
To determine how much credence to give to the conclusion of an argu-
ment, one must be able to judge the assertions that are made in its support
as to their relevance and, given that an assertion is considered relevant, one
must decide how much weight to attach to it (Campbell, 1986; Pham, 1998;
Skovgaard-Olsen, Singmann, & Klaur, 2016). How much weight to attach
to an assertion depends in part on what one believes to be the likelihood
that it is true; if one believes the assertion to be false, one will give it no
weight. In both cases the judgment is one of degree, relevance can vary
continuously as can weight. The same types of considerations must be
given also to counterarguments that one may construct, inasmuch as the
assertions comprising them can also be more or less relevant and can vary in
the degree to which they increase the credibility of the counter conclusion.

Judgments of Truth or Falsity

A claim may be judged to be relevant to a conclusion and to give the
conclusion considerable support – assuming it is true. The assumption
here is critical. A false claim provides no support. So, a major task in
evaluating an informal argument is to judge whether to accept the claims
made in its support as true. In many cases that is likely to be the most
difficult of the evaluations that must be made. Often it will be impossible
to judge with certainty either that a claim is true or that it is false; the best
one can do is to make a judgment of the amount of credence to give to
a claim that is somewhere between certainly true and certainly false. This is
where probability theory becomes a useful tool in argument evaluation.

Aggregation or Synthesis
Having considered the argument’s parts, one must assess its persuasiveness
as a whole. One must decide how convincing it is relative to the most
compelling counterarguments one is able to construct, and on this basis
accept or reject its conclusion(s), perhaps with qualifications or provisos,
and probably at some level of surety less than absolute.
In some instances, one may feel justified in being very confident that
a conclusion is true on the grounds that one can imagine no way that the
supporting statements (which one believes to be true) could be true if the
conclusion were false. This comes very close to the situation one has with
a formal deductive argument, except that in the informal case, what is

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 Aggregation or Synthesis 91
lacking is the authority of the formal rules of inference. More often, the
conclusion is likely to be accepted at something less than certainty. (Of
course, this is true also when formal logic is applied to real-life situations.
In this case, one can be certain the conclusion of a valid inference is true if
the premises that imply it are true, but if the premises are meaningful
assertions about the real world, one’s certainty of their truth is likely often
to be less than complete.)
I do not mean to suggest that one always does each of the noted aspects
of evaluation in a conscious and deliberate fashion, but the effective
evaluation of informal arguments involves them, at least implicitly.
Further, these activities draw upon other capabilities such as the following.
• Assessment of own knowledge. If one knows a lot about the topic of an
argument, one is in a better position to evaluate the argument than if
one knows only a little. Equally as important as one’s knowledge of
a topic, however, is one’s awareness of the extent and limitations of that
knowledge. Especially vulnerable is the person who believes his/her
knowledge to be extensive when, in reality, it is very limited. This
consideration becomes important in certain types of plausibility judg-
ments. Suppose, for example, that I judge a particular assertion to be
highly implausible on the grounds that it declares to be a fact something
of which I am unaware. My tacit assumption must be that if that which
is declared to be a fact really were a fact, I would be aware of it, and since
I am not aware of it, it must not be a fact. Such a basis for judging
plausibility is justified only to the degree that I am highly knowledge-
able with respect to the subject. If my knowledge of the subject is very
limited and I am aware that it is, then the fact that I do not know some
assertion pertaining to that subject to be true should give me very little
reason to conclude that it is false.
• Information seeking and selection. It may be desirable to obtain addi-
tional information to supplement what one has in one’s head, especially
if one’s knowledge of the domain is limited. Skill in the finding of
information is a useful one not only in the context of argument
evaluation, but much more generally. “Poor arguments are not neces-
sarily poor because they are illogical, but because they ignore some
relevant possibility or piece of evidence” (Garnham & Oakhill, 1994,
p. 262). At least some aspects of this skill should be relatively easy to
address through training, namely those pertaining to the use of formal
information-finding resources such as encyclopedias, atlases, almanacs,
indexes, and a variety of computer-based search engines. Instruction in

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 92 Evaluating Arguments
the effective use of such resources should, I believe, receive considerable
emphasis throughout the educational system. This is not to suggest that
students should be led to adopt the idea that answers to all questions
can be found or should be sought in books or other repositories of
information, but it is to their advantage to know how to find those that
are to be found there.
• Estimation and approximation. Many arguments have quantitative com-
ponents: assertions about populations, incidence, costs, distances, rates
of change, probabilities. In order to evaluate such arguments, it is
necessary to judge the plausibility of their component assertions, and
inasmuch as the quantities involved are often – perhaps typically –
unknown to the evaluator, such judgments must be based on the ability
to estimate or approximate them. Skills of estimation and approxima-
tion appear to be grossly undervalued by society and their development
is strangely neglected by many educational systems.
• Detection of inconsistencies. Inconsistencies can occur within an argu-
ment or between an argument’s assertions and known or assumed facts.
Such inconsistencies, when recognized, weaken an argument, so the
ability to detect them is important. This is a knowledge-based ability;
one cannot detect inconsistencies between assertions and facts unless
one knows the facts.
To assess an individual’s competence with respect to argumentation
probably requires explicit attention to each of these aspects and several
more. It is not safe to assume that an individual who is competent at
detecting fallacies in deductive arguments posed in syllogistic form will be
equally competent at judging the amount of credence that should be given
to assertions offered in support of informal inductions.
An important aspect of the evaluation of informal arguments involves
going beyond the information comprising the argument itself. The most
compelling reasons for rejecting an argument are often found in what
could have been said but was not. A general strategy that is useful in
evaluating arguments is to attempt to make explicit those relevant facts
that were left unstated. It may be especially useful to attempt to explicate
any facts that are inconsistent with the conclusion drawn. That is to say, it
may be especially useful to attempt to construct a counterargument. The
ability to do so will depend heavily, of course, on one’s knowledge of the
domain.
It is perhaps here, in the realm of counterargument construction, that
reasoning in argument evaluation is most heavily knowledge dependent. It

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 Enthymemes 93
is here too that one’s metaknowledge – one’s awareness of the extent and
limitations of one’s own knowledge of a domain – is especially important.
If I know a lot about a domain, and realize it, and am unable to construct
a compelling counter to a given argument, I am likely to give more
credence to the argument’s conclusion than if I am aware that I know
too little about a topic to be able to construct a counterargument even if
there were a simple but compelling one to be constructed.

Enthymemes
In everyday reasoning arguments are often stated incompletely; such
arguments are known as enthymemes.
The contract is signed by both parties, so it is binding.
The contract is not signed by both parties, so it is not binding.
What is missing in the first case is:
If the contract is signed by both parties, it is binding,
and in the second:
If the contract is not signed by both parties, it is not binding,
Some might want to claim that if one concludes that a contract is not
binding upon being told that it was not signed by both parties may not be
reasoning at all, but simply recalling from memory the fact that to be
binding a contract must be signed by both parties. However, even if
memory is implicated in this fashion, it is a missing assertion that justifies
the conclusion; and what memory can supply is the knowledge that for any
contract to be binding it must be signed by both parties, and to get from
contracts in general to the particular contract in question requires an
inference.
Other enthymemes:
She does not speak French, so she did not grow up in Paris.
Missing premise:
If she does not speak French, she did not grow up in Paris,
or
If she grew up in Paris, she speaks French.
The percolator light is not on, so the coffee is not ready.

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 94 Evaluating Arguments
Missing premise:
If the percolator light is not on, the coffee is not ready,

or
If the coffee is ready, the percolator light is on.
Enthymemes are very common in everyday discourse (Hitchcock, 1986);
undoubtedly, they are more common than fully expressed arguments in
which all premises and the conclusion are explicitly identified. Baird (1950)
makes the point in contending that arguments are usually stated as enthy-
memes: “One premise at least is usually taken for granted or may be
considered too obvious for statement as a formal proposition” (p. 155).
Baird recognizes three orders of enthymeme depending on whether what is
absent is the major premise, the minor premise, or the conclusion. To
illustrate the expression of an argument as an enthymeme, the missing
middle term of which may be supplied by the context, Baird uses the
following: “The United States will go bankrupt, for it is headed toward
a debt of more than 500 billion dollars” (p. 156). (As of January 2019, the
US national debt stood at approximately 22 trillion dollars, and was
increasing at the rate of about 3.8 billion dollars a day. What a difference
sixty-six years can make!)
The observation that people often state arguments incompletely is not
a criticism. Sometimes stating a premise may amount to wasting words
because it can be safely assumed that the information that would have been
conveyed by the missing premise is well known to the argument’s partici-
pants. Sometimes what is missing is one or more premises; sometimes it is
a conclusion. (A conclusion may go unstated because the originator of the
argument considers it too obvious to require being stated.) Of course,
when participants in an enthymematic argument are having difficulty
agreeing on whether it is persuasive – that its conclusion should be
accepted – they may find it helpful to make the missing part(s) of the
argument explicit.

Models of Argument Evaluation

Attempts to provide prescriptive models of how arguments – especially
informal arguments – should be evaluated typically state a set of principles
by which the process should be guided. The following case in point is from
John Hoagland (1986).

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 Models of Argument Evaluation 95
First the structure of the argument must be established: the conclusion must
be identified, the premises including tacitly assumed ones must be stated,
and the argument scanned for formal patterns like modus ponens. Then
ambiguity and equivocation must be removed. Internal criticism must focus
on the relation of premises to conclusion: do the premises, if true, establish
the conclusion? External criticism asks whether the premises are true, then
tests the strength of the argument with counterarguments. Finally, an
overall evaluation must be given: is the argument strong or weak? Some
writers recommend diagramming the premises relative to the conclusion to
reveal argument structure (p. 392).
Here is another set of rules for evaluating arguments (Table 3.1), this one
offered by Evert Vedung (1986). (Rules 2a and 2b have been modified to
correct typing errors in Vedung’s table).

Table 3.1 Rules of assessment in argumentative inquiry (from Vedung,

1986, p. 357)

1. The rule of support: Claims should be supported by reasons.

2. The rule of clarity: The content should be clear.
a. Claims and reasons must not be tacit and so give rise to obscurity.
b. Words must not be used in an unclear way.
c. The presentation should be arranged so as not to cause obscurity.
3. The rule of relevance: The content should be relevant.
a. Pronouncement should be relevant.
b. Relevant aspects of the case should be acknowledged and presented.
4. The rule of consistency: The content should be consistent.
a. Inferences from reasons to claims should be derived in a logically valid way.
b. Claims should be consistent with other claims, reasons with other reasons.
5. The rule of truth: The content should be true.

William Benoit (1986) contends that there are two important phases to
argument evaluation, the first of which is to identify the argument in its
entirety. “Since most arguments are enthymematic, the argument critic
must identify the premises that the auditor is willing to supply in order to
complete it . . .. Once the entire argument is displayed, the critic can
proceed to the second phase, evaluation” (p. 302).
Ennis (2001) offers an approach to argument evaluation that recognizes
five “phases” (as paraphrased):
(1) Identification of argument parts (conclusions and supporting reasons).

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 96 Evaluating Arguments
(2) Deciding to what extent (unstated) assumptions should be added.
(3) Abandonment of attempts to classify arguments into types.
(4) Successive application of sets of argument standards “usually in
conjunction with assumption ascription, stopping only when the
argument satisfies a set of standards (in which case the argument is
deemed a good argument), or when the set of sets of standards is
exhausted without any set being satisfied (in which case it is deemed
a weak or poor argument)” (p. 120).
(5) Qualified reasoning tempered by flexibility toward the degree of
precision needed in the situation. (Listed on pages 120, 121, and
again, slightly modified, on page 134.)
Ennis (2001) expands on each of these phases in a thoughtful and
thought-provoking way. Using actual court proceedings in a murder trial
to illustrate how they may play out, he judges an argument to be induc-
tively valid by virtue of satisfying the following “best-explanation” stan-
dards, the first three of which, he contends, are necessary conditions, while
the fourth is desirable.
(1) The conclusion should explain facts.
(2) Plausible alternative hypotheses should be ruled out.
(3) The conclusion should be consistent with known facts, and
a reasonable effort should have been made to uncover facts incon-
sistent with the conclusion.
(4) The conclusion should be plausible (p. 123).
Ennis treats this set of standards as one among others that might be used,
and the selection of an appropriate set as itself a decision that an argument
evaluator must make.
There are different sets of standards, including, in my view, sound-
ness standards, inductive standards (including best-explanation stan-
dards and standards for generalizing from instances), and value
judging standards, all of which are subject to qualified-reasoning
adjustments. Using the sets-of-standards approach, we are not
required to classify an argument before evaluating it. Instead we
successively apply sets of standards. We then accept the conclusion
if, in the context, it satisfies any set of standards – remembering the
set of standards that it satisfies, and showing this, if need be, in
appropriate qualifications to the acceptance of the argument or the
conclusion. If the argument satisfies no set of standards, the argument
is deemed defective (p. 125).

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 Models of Argument Evaluation 97
How one is to judge the adequacy of any particular set of standards to
apply to any particular argument is not clear. Failure to get agreement on
the answer to this question leaves open the possibility of different evalua-
tors coming to different opinions regarding the quality (validity, strength,
soundness) of any specific argument. Ennis effectively acknowledges this in
noting that the standards in terms of which an evaluator will assess an
argument will depend on his or her goals.
In a highly influential book (Uses of Argument), cited in Chapter 2,
Stephen Toulmin (1958) likens an argument – an extended argument
I should say – to an organism and notes that its structure can be examined
at more than one level. He considers the traditional distinction between
deductive (conclusive, demonstrative) and inductive (inconclusive, non-
demonstrative) arguments to be an unfortunate one and one that has led to
an increasingly wide gap between the work of logicians and the reasoning
required by everyday life. However, the distinction between deductive and
inductive arguments, for better or worse, is a widely accepted one, and in
my view we do well to pay it some mind.
Toulmin contends that logicians have focused on deductive reasoning,
and in particular on the analytic syllogism, because it is tractable, but it is
not very usefully applied to problems of reasoning with practical import.
The distinction between deductive and inductive reasoning, as it has been
made historically, Toulmin suggests, conflates all of the following distinc-
tions: arguments with necessary conclusions versus arguments with prob-
able conclusions, formally valid arguments versus arguments that cannot
be formally valid, arguments that rely on a warrant the adequacy and
applicability of which have already been established versus arguments
that are intended to establish the adequacy of a warrant, arguments that
are expressed in terms of logical connectives or quantifiers (all, some, or)
versus those expressed in other terms (most, few, but), and analytic
(abstract) arguments versus substantive ones.
Toulmin presents a structured approach to argument evaluation that
distinguishes several components of an argument other than the traditional
premises and conclusion. There is first the claim (C) that is being made and
that the entire argument is intended to support. Then there are the data
(D), the facts that provide the foundation for the claim. Warrants (W) are
the kinds of propositions that justify arguing from data to claims. These are
the basic components that all arguments contain, implicitly if not expli-
citly. Consider, for example, the argument: Harry is a British subject,
because he was born in Bermuda. The claim is that Harry is a British
subject; the datum is that Harry was born in Bermuda; the warrant,

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 98 Evaluating Arguments
unstated, is that anyone born in Bermuda is a British subject. In this simple
case, we could easily have analyzed the argument in terms of
a (supplemented) major premise (Anyone born in Bermuda is a British
subject), minor premise (Harry was born in Bermuda), and conclusion
(Harry is a British subject).
In this example, the warrant was not explicitly stated in the argument. In
Toulmin’s conceptualization, this is a distinguishing difference between
data and warrants: data are stated explicitly, warrants are not. Analyzed in
traditional terms as a deductive argument involving premises and
a conclusion, the example would be considered an enthymeme and, in
particular, an enthymeme of the first order, which is to say an argument
missing the major premise.
Toulmin’s approach is intended to be able to deal with arguments much
more complex than the above example, but additional concepts are neces-
sary for the more complex cases. Modal qualifiers (Q), such as “necessarily,”
“probably,” and “presumably,” and conditions of exception or rebuttal (R)
(unless . . .) are used to indicate the degree of force that a warrant carries
and conditions under which it may not hold. The force of the argument in
ensuring the truth of the claim, even as qualified and delimited by provisos,
depends on the truth of the datum and the warrant, both of which are
claims themselves. The situation is analogous to that of a conventionally
structured deductive argument, in which the guarantee of the truth of the
conclusion rests on the truth of the argument’s premises. The question of
the truth of the data in such an argument is likely to be less problematic
than the question of the truth of the warrant, because the former will tend
to involve particular verifiable facts, whereas the latter are more likely to be
general claims that may not be verifiable empirically in practice or even in
principle.
Toulmin recognizes the need for support for warrants and provides for it
in his representation of an argument with a component that he calls
backing (B). “Standing behind our warrants . . . there will normally be
other assurances, without which the backing themselves would possess
neither authority nor currency” (p. 103).
We see here the possibility of an infinite regress – an endless stack of
turtles standing on the backs of turtles – but this does not distinguish
Toulmin’s approach to argument evaluation from any other serious
approach of which I am aware. All involve justifying certain claims in
terms of certain other claims. In all cases, the claims that are used for
justification require justification themselves. Invariably one can be pushed
back to a point at which one must say, I take thus-and-so as given, and that

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 Models of Argument Evaluation 99
is the end of it. Toulmin recognizes this problem and acknowledges the
need to take something for granted.
Some warrants must be accepted provisionally without further challenge, if
argument is to be open to us in the field in question; we should not even
know what sort of data were of the slightest relevance to a conclusion, if we
had not at least a provisional idea of the warrants acceptable in the situation
confronting us. The existence of considerations such as would establish the
acceptability of the most reliable warrants, is something we are entitled to
take for granted (p. 106).
Toulmin emphasizes the distinction between a warrant and its backing.
It is critically important, he argues, to keep clear the difference between the
force of warrants and the backing on which their authority depends. The
force of a statement of a given type, say a universal statement, when used as
a warrant is the same in all fields of argument, in Toulmin’s view, but the
backing for a warrant of a given sort is likely to vary from field to field. In
this sense the criteria for argument evaluation are field-dependent.
Toulmin sees the fact that his representation of the structure of arguments
makes the distinction between warrant and backing explicit as one of its
major benefits and what most sharply differentiates his representation of
the structure of arguments from the more traditional representation in
terms of premises and conclusions.
[I]f we pay closer attention to the differences between warrants and backing,
and between different sorts of backing for one and the same warrant, and
between the backing for warrants of different sorts, and if we refuse to focus
our attention hypnotically on the traditional form “All A’s are B’s,” we can
not only come to see that sometimes “All A’s are B’s” does have existential
implications and sometimes not, but furthermore begin to understand why
this should be so (p. 116).
Again, “[o]nce we bring into the open the backing on which (in the last
resort) the soundness of our arguments depends, the suggestion that
validity is to be explained in terms of ‘formal properties,’ in any geome-
trical sense, loses its plausibility” (p. 120).
In addition to the distinction between analytical and substantial argu-
ments, a key aspect of Toulmin’s approach is his insistence that substantial
arguments are field- or context-dependent and the criteria by which argu-
ments should be judged can differ across fields. “Arguments within any
field can be judged by standards appropriate within that field, and some
will fall short; but it must be expected that the standards will be field-
dependent, and that the merits to be demanded of an argument in one field

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 100 Evaluating Arguments
will be found to be absent (in the nature of things) from entirely meritor-
ious arguments in another” (p. 255). “If we ask about the validity, necessity,
rigour, or impossibility of arguments or conclusions, we must ask these
questions within the limits of a given field, and avoid, as it were, con-
demning an ape for not being a man or a pig for not being a porcupine”
(p. 256).
The field-dependence of argumentation is reflected in the fact that
different sorts of problems require different types of warrants, backing
and criteria of necessity, and probability. It is critical, in Toulmin’s view,
that people who wish to make progress on the evaluation of arguments in
a given field agree on how to test the adequacy of warrants for arguments
pertaining to that field. “Two people who accept common procedures for
testing warrants in any field can begin comparing the merits of arguments
in that field: only where this condition is lacking, so that they have no
common ground on which to argue, will rational assessment no longer be
open to them” (p. 176).
It is the field- or context-dependency of substantive arguments that
ensures the limited applicability of abstract logic – which does not recog-
nize inter-field differences – to them. Toulmin’s assessment of the rele-
vance of symbolic logic to practical arguments is severe: “By now,
mathematical logic has become a frozen calculus, having no functional
connection with the canons for assessing the strength and cogency of
arguments” (p. 186). “In logic, as in morals, the real problem of rational
assessment – telling sound arguments from untrustworthy ones, rather
than consistent from inconsistent ones – requires experience, insight and
judgment, and mathematical calculations (in the form of statistics and the
like) can never be more than one tool among others of use in this task”
(p. 188).
Toulmin’s Uses of Argument was (is) a very influential (and accessible)
book – in effect an extended argument regarding the inadequacy of formal
logic for understanding and evaluating the kinds of arguments encoun-
tered in daily life. First published more than a half century ago, it (an
updated edition published in 2003) is still in print, and much cited. In my
view, no treatment of argumentation would be adequate that did not give it
substantial space. However, despite my admiration for Toulmin’s work,
I have elected to stay with the conventional distinction between deductive
and inductive (formal and informal) arguments. The several new concepts
that Toulmin introduced in his work on argument evaluation – warrant,
backing, field- or context-dependency are of unquestioned interest and
importance to researchers who study argumentation; how much they add

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press

 Models of Argument Evaluation 101
to our understanding of how people actually evaluate arguments in daily
life is debatable.
Evaluating arguments involves two kinds of judgment: (1) whether the
argument’s conclusion or key assertion follows logically from, or is made
highly likely by, its premises or supporting assertions, and (2) whether to
accept the premises or supporting assertions as true or very probably so.
Reflection prompts two further questions: What makes an argument
persuasive and what makes an assertion plausible?
These questions are of considerable interest in many contexts: educa-
tion, debating, diplomacy, marketing/advertising, diagnostic reasoning,
military planning, crime investigations, judicial proceedings, among
numerous others. They are considered in Chapters 4 and 5.

https://doi.org/10.1017/9781108892032.004 Published online by Cambridge University Press