Annual Reviews

www.annualreviews.org/aronline

Ann. Rev. Comput.Sci. 1986. 1:255-87

Copyright©1986by AnnualReviewsInc. All rights reserved

KNOWLEDGEREPRESENTATION
1AND REASONING

eHector J. Levesque

Department of Computer Science, University of Toronto, Toronto, Ontario M5S IA4

Canada

Thenotionof a representationof knowledge is at hearteasyto understand.It

simplyhas to do with writing down,in somelanguageor communicative
medium,descriptionsor picturesthat correspond in somesalient wayto the
worldor a state of the world.In artificial intelligence(AI)weare concerned
withwritingdowndescriptionsof the worldin whichan intelligent machine
might be embeddedin such a way that the machine can cometo new
conclusions about its world by manipulating these symbolic representations.
David Israel characterizes this "representation problem" as follows;

All parties to the debate agree that a central goal of research is that computers must
somehowcometo "know" a good deal of what every humanbeing knowsabout the world
and about the organisms, natural or artificial, that inhabit it. This bodyof knowledge--
indefinite, no doubt, in its boundaries---goes by the name"common sense." The problem
weface is howto impart such knowledgeto a robot. That is, howdo wedesign a robot with
a reasoning capacity sufficiently powerful and fruitful that whenprovided with some
subbody of this knowledge, the robot will be able to generate enough of the rest to
intelligently adapt to and exploit its environment?(68, p. 37)

Despite the apparent simplicity of this goal, the research area of knowledge
3representation (KR) has a long, complex, and as yet nonconvergent history.

~This review draws heavily from work done with Ron Brachman. I also thank Jim des
Rivi6res, Bart Selman, and John Tsotsos, for helping with an earlier version of the document.
Financial support was received from the Natural Sciences and Engineering Research Council of
Canada.
2Fellow of the CanadianInstitute for AdvancedResearch.
3Research in KRoriginated with a single paper written by John McCarthyin 1958, and
republished as (89).

255
8756-7016/86/1115-0255502.00
Annual Reviews
www.annualreviews.org/aronline
256 LEVESQUE

Even though manycurrent AI programshave a knowledgebase (KB) contain-

ing symbolic descriptions in some "representation scheme"along the lines
described above, there is still significant disagreement amongresearchers
about manyof the most fundamentalissues. Part of the problemis that KRhas
evolved (and absorbed material) from a numberof research areas with quite
different goals and methods,such as psychology(in terms of realistic human
memorymodels), linguistics (representations of word senses), philosophy
(the nature of concepts and propositions), logic (varieties of formal reason-
ing), and computerscience (information systems). For a closer look at this
amazing diversity, see (131a), a very revealing questionnaire about
answeredby the researchers themselves.
But a more coherent interpretation based primarily on logic and computer
science appears to be emerging.So rather than recapitulate the history of KR,
this article takes a definite stand on the nature of the research. KRis
intimately connected with reasoning, because an AI system will almost
alwaysneed to generate explicitly at least someof what has been implicitly
represented. Moreover,for sufficiently expressive representation languages,
calculating these implications maybe too demandingcomputationally, and so
compromisesare necessary. Knowledgerepresentation, then, can be thought
of as the study of what options are available in the use of a representation
schemeto ensure the computational tractability of reasoning.
The idea of constructing systems that performtheir tasks by reasoning with
explicitly represented knowledgeis just a workinghypothesis about howto
achieve generality, flexibility, modularity, and the like in complexsystems.
In the next section, we examinethis hypothesis in more detail, and we show
howit leads to a view of~(-R that is groundedin logic, on the one hand, and
computationalcomplexity,on the other. In fact, the mainthemeof this article
is that we can understand much of KRresearch as attempts to reconcile
simultaneously the demandsof logical coherence and computationaltractabil-
ity.
Perhaps the most interesting type of KRresearch that is not well covered by
this emphasis on the demandsof tractable reasoning has been the attempt to
formalize commonsenseknowledgeof the world. Indeed, Hayes argues (61)
that the most pressing task for KRresearch is to analyze what kinds of
knowledgeneed to be represented in specific subject areas, without too much
concern (initially, at least) for howthe knowledgewill be represented
whetherit can be effectively used by any system. This leads to a very different
view of KR--aview that concentrates on formal representational theories of
domainsof general interest such as the properties of time (3, 95, 134), space
(75), and liquids (60). Twocollections of research papers in this style
"applied" KRresearch have been published (10, 66). For more com-
prehensive surveys of KR, see (6) and (105). Anexcellent general introduc-
Annual Reviews
www.annualreviews.org/aronline
KNOWLEDGE
REPRESENTATIONAND REASONING 257

tion to KR(and much of AI) is found in (107). Finally, an annotated

collection of thirty of the most influential papers in the area has been
published (22).

1. THE KNOWLEDGEREPRESENTATION HYPOTHESIS

The basic assumption underlying KR(and muchof AI) is that thinking can
usefully understood as mechanicaloperations over symbolic representations.
This hypothesis is, in fact, quite old, mucholder than computers, and seems
to have originated with the philosopher Leibniz (1646-1716):
Thereis little doubt,however, that Leibniz’ideas,whichfar outstrippedin detail and
understandinganyearlier hints, werehis ownspontaneous
creation."WhileI wasyet a boy
witha knowledge onlyof common logic, and withoutinstructionin mathematics, the
thoughtcame to me,I knownotbywhatinstinct,that ananalysisof ideascouldbedevised,
whence in somecombinatory way,truths couldarise andbe estimatedas thoughby
numbers" (Elementa Rationis).Hewasthereafterconstantlyoccupied withsuchnotions
andattempted to contrivean alphabetof thought,or characteristica
universalis,which
wouldrepresentideas in a logical way,not thingsin a pictorial way,andwouldbe
mechanicalin operation,unambiguous, and nonquantitative.(The Encyclopedia of
Philosophy, Vol. 4, p. 538)
Just as there is a calculus of arithmetic, where numerical expressions are
formally manipulatedin a value-preserving way, so might there be a calculus
of thought, wherepropositional expressions could be formally manipulatedin
a truth-preserving way.
This agrees remarkably well with most current views of KRwithin AI.
ManyAI systems represent knowledgeas explicitly as possible, and (relative-
ly) declaratively, in someformal language. The term knowledge-basedsystem
(or KBS)is currently popular in AI. An AI system can be described
knowledge-based not because knowledgeis required to build it, nor just that it
behavesas if it had knowledge,but rather that its architecture includes explicit
KBsthat are moreor less direct symbolic encodings of the knowledgeof the
system. This is certainly true of Expert Systems[see (135), for example],
currently the most visible and plentiful type of AI system.
This is not to say that all AI systems exhibiting knowledgeare knowledge
based in this sense. A typical game-playingprogram,for instance, might act
as if it believed that it should bring its knight out early in the game.Yet this
could be the result of applying a complexevaluation function to the leaves of
a generated gametree. Specifically, there need not be any symbolicstructure
anywherein the programcorresponding to that belief. But according to the
Knowledge Representationhypothesis (131), intelligence (or at least a certain
kind of generality, versatility, etc.) is best servedby explicitly representingin
the data structures of a programas muchas possible of what a systemneeds to
know.
Annual Reviews
www.annualreviews.org/aronline
258 LEVESQUE

Implicit in this hypothesis4 are two major properties that the structures
forming a KBmust satisfy:
¯ For the structures to represent knowledge,it must be possible to interpret
them propositionally, that is, as expressions in a language with a truth
theory. Weshould be able to point to one of them and say what the world
wouldhave to be like for it to be true.
¯ The system should act the way it does because of the presence of these
structures. Clearly, the hypothesis wouldnot be satisfied in a systemwhere
the KBwas completely ignored (like commentsin a program, for ex-
ample).
The key point is that an account of cognitive activity in terms of com-
putational operations over propositionally interpreted structures imposescon-
straints on howa KBScan be realized. First of all, it rules out data structure
operations that do not respect (in somesense) the propositional interpretation
of a KB(e.g. reversing the words of a sentence). Secondly, because of the
causal role of a KB, it rules out operations that are not computationally
manageable.In other words, the operations on a KBneed to be semantically
coherent without demandingmore than what any computercan be expected to
do. To better understand these constraints, we need to examinewhat it means
to operate on structures in a waythat respects their semantic interpretation.

2. KNOWLEDGE REPRESENTATION AND LOGIC

Wantingto deal with truth-preserving operations over symbolic structures

PUtSus into the domainof logic. Indeed, standard formal logic is a perfect
example of a system that deals with symbolic structures (its well-formed
formulas) and operations over them that, on the one hand, can be defined
purely formally (as specified by a proof theory) and, on the other hand, can
shownto preserve semantic properties (as specified by a truth theory).
For a detailed explanation of formal logic see, for example, (97). Recall,
however,that there are two major relationships of inter~t-drfined over the
sentences of a logical language.
¯ logical consequence A sentence tx is a logical consequence of a set of
sentences S (written S ~ ct) iff any interpretation of the logic that makes
every sentence in S true also makesa true. A sentence is valid if it is a
logical consequenceof the empty set.

4It is worthkeepingin mindthat this is onlya workinghypothesis.Perhaps its strongest

defenseis thatit seems
to be"theonlygame in town";buteventhere,alternative
trendsappearto
be emerging(123).Fora criticismof this hypothesis onmorephilosophicalgrounds,see(39).
Annual Reviews
www.annualreviews.org/aronline
KNOWLEDGE
REPRESENTATIONAND REASONING 259

¯ derivability A sentence a is derivable from a set of sentences S (written

~- or) iff there is a sequenceof sentencesal ..... otn whereotn is t~, such
that each t~i either is an elementof S, or a logical axiom, or follows from
earlier aj’s by a rule of inference. A sentenceis a theoremif it is derivable
from the empty set.
Oncea language has been fixed, the logic must specify the allowable in-
terpretations, on the one hand, and the axiomsand rules of inference, on the
other.
The mainpoint is that derivability in formal logic is defined in a waythat
does not mention what the formulas are intended to mean. Yet soundness and
completenessproperties can establish links betweenthe two: A sentence will
be derivable from a set of sentences precisely whenit is a logical consequence
of that set. For logics that are soundand complete,this is the sense in which
purely formal operations can be shownto preserve semantic properties, as
required by the KRhypothesis.
Logic however, does not guarantee the computational tractability of the
operations of its proof theory. The derivability of a formula dependson the
existence of a certain sequence of formulas, but not on whether or not that
sequence can be found purely mechanically, or in a reasonable amount of
time. So it is not immediately obvious howlogic can be made to play the
causal role required by the KRhypothesis.
Indeed, the role of formal logic in KRhas been hotly contested from the
very beginning.5 Part of the problemconcerns the history and goals of KRand
their differences from those of symbolic logic. After Leibniz, the next big
advancein formal logic was the workof Frege at the turn of the century who,
along with Russell, Peano, and others, gave logic muchof the flavor it has
today. The goal of this early work was to put mathematicsand mathematical
reasoning on a sound theoretical footing. Indeed the major application of
symbolic logic until recently, and one of the success stories of twentieth
century mathematics,was the analysis of formal theories of sets and numbers.
The goals of KR, however, were always quite different, and perhaps much
more in line with Leibniz’ original dream. On the one hand, KRschemes
were being used to represent the semantic con~ent of natural language con-
cepts (e.g. 125,128), and on the other, to represent psychologicallyplausible
memorymodels(e.g. 109, 112). In neither case was there a clear relationship
to formal languages of any kind [as Woods,amongothers, has pointed out
(141)]. Gradually, however, KRschemesbegan to be used as a very flexible
and modular way to represent the facts that a system needed to know to

5Toget an ideaof the rangeof opinions

involved
at variouspoints,see(58,99, 102,111).
Annual Reviews
www.annualreviews.org/aronline
260 LEVESQUE

behave intelligently in a complexenvironment[an early exampleis (137)].

This view of KR,involving propositional representations of the beliefs of a
system, along the lines of the KRhypothesis, has slowly cometo dominate
the field. Withit has comean emergingconsensus about at least someof the
issues relating KRand logic. For example, the need for a truth theory for
representation languages is generally acknowledged.Without someconcrete
specification of the meaningof a notational convention, what is implied by an
expression in that language is unclear, and comparisonsto other notational
systems are impossible (57). Also, it is generally accepted that someform
deductivereasoningwill be necessaryto extract whatis implicit in the explicit
beliefs to which a system has access. Whena KBSneeds to knowsomething
about its domain, it must use the symbolic structures representing what it
knowsin order to decide what is true or false about the world. In general,
there is no guaranteethat what it needsto knowwill be represented directly in
these data structures, but it mayvery well be-logically implicit in these
structures.
Suppose, for example, that a medical diagnosis system needs to know
whether or not a patient P is allergic to medication Mbefore prescribing a
treatment. It maybe the case that the KBin question has a structure represent-
ing the fact that
Patient P is allergic to medication M.
For instance, there maybe a table in a database relating patients to the
medicinesto whichthey are allergic. In this case, the systemcan determineby
retrieval alone whether or not it can use M. On the other hand, suppose the
KBhas explicitly represented only the following facts:
Patient P is allergic to medication M*.
and
Anybodyallergic to M*is also allergic to M.
In this case, it will be necessaryto determinefacts that are only implicitly
represented in order to be able to decide what medication can be prescribed.
In general, a KBSwill be concernednot with what is explicitly represented in
.a KBbut, rather, with what these structures taken together tell about its
application domain. In other words, the main concern of a KBSis what
information is contained either implicitly or explicitly in a collection of
propositionally interpreted data structures.
To a ftrst approximationthen, if we restrict our attention to the yes-no
questions about the world that a system might be interested in, a KRsystem
has to be able to determinethe logical consequencesof what is present in the
KB. In other words, given a yes-no quesdon a, and a KBunderstood as a
Annual Reviews
www.annualreviews.org/aronline
KNOWLEDGE
REPRESENTATIONAND REASONING 261

collection of sentences, a KRsystemmust be able to determinewhetheror not

KB~ or. 6 So if, as a starting point, wenowfix the representation languageto
be the first-order predicate calculus (or FOL) 7, we can give a very precise
definition of KB~ c~ (and thus, the reasoning service to be provided by the
KRsystem). Moreover, if we assume that the KBcan be understood as
finite set of sentences and let KBstand for their conjunction, then becauseof
soundness, completeness, and other properties of FOL, KB~ ot iff the
sentence (KBD oz) is derivable from no premises, that is, it is a theorem
FOL. Thus, the question-answering operation becomesone of theorem prov-
ing in FOL.

2.1 The Problem

The good news in reducing the KRservice to theoremproving is that we now
have a very clear, very specific notion of what the KRsystem should do; the
bad newsis that it is also clear that this service cannotbe provided.Thefact is
that deciding whether or not a sentence of FOLis a theorem is unsolvable
[again, see (97)]. Moreover,evenif werestrict the languagepractically to the
point of triviality by eliminating the quantifiers, the decision problem,though
nowsolvable, does not appear to be solvable in anywherenear reasonable
time. s It is important to realize that this is a property not of particular
algorithms that have been discovered so far, but of the problemitself: There
cannot be an algorithm that does the test for theoremhoodcorrectly in a
reasonable amountof time. This bodes poorly, to say the least, for a service
that is supposed to be only a part of a larger KBS,
Oneaspect of these intractability results is that they deal with the worst-
case behavior of algorithms. In practice, a given theorem-provingalgorithm
maywork quite well. In other words, a given programmight behave properly
for a very wide range of questions, eventhoughthere will alwaysbe questions
whoseanswerswill not be returned for a very long time, if at all. Howserious
is the problem,then? To a large extent this dependson the kind of question
asked of a KRsubsystem. The worst-case prospect might be perfectly toler-
able in a mathematical application where a question is an open problem in
mathematics. Provided that progress is being made, a user might be quite

6In this viewof a KRsystem,the serviceit provides to a knowledge-basedsystemdepends

onlyonthe truth theoryof the language of representation. Nocommitment is necessaryabout
howsucha servicecanberealized,if indeedit can.Thisis in keeping withwhatwehavecalled
elsewhere a functionalviewof KR(see 18, 79), wherethe serviceperformed bya KRsystem
definedseparatelyfromthe techniquesa systemmightuseto realizethat service.
7See (97)for detailsonthis language.
In whatfollows,wewill havereasonto consider other
languagesas well, but mainlyas departuresfromFOL.
STheproblem is nowco-NP-hard, meaningthat it is stronglybelievedto be computationally
intractable(see52).
Annual Reviews
www.annualreviews.org/aronline

262 LEVESQUE

willing to stop and redirect a theoremprover after a few months,if it seemsto

be thrashing. Worst-casebehavior is irrelevant; this mightbe the only case of
interest.
But imagine, on the other hand, a robot that needs to knowabout its
external world(such as whetheror not it is raining outside, whereits umbrella
is, and so on) before it can act. If this robot has to repeatedly invoke a KR
utility as a subroutine on a wide range of questions, the worst case prospect is
muchmore serious. To be bogged downon a logically difficult but low-level
subgoal and be unable to continue without humanintervention is clearly an
unreasonable form of behavior for somethingaspiring to intelligence.
But "on the average" the robot might do alright. The trouble is that nobody
seems to be able to characterize what an "average" case might be like. As
responsible computerscientists, we should not be providing a general inferen-
tial service if all that can be said about it is that by and large it will probably
worksatisfactorily. If the KRservice is to be a modulein a larger systemand
if it is not available for introspection or control, then it had better be
dependable both in terms of its correctness and the resources it consumes.
Unfortunately, this rules out a service based on theorem proving (in full
first-order logic).

2.2 Some Pseudo-Solutions

There are at least two fairly obvious ways to minimizethe intractability
problem.Thefirst is to pushthe computationalbarrier as far back as possible.
An entire subarea of AI called Automatic Theorem Proving (ATP) studies
techniques for avoiding redundancies and speeding up certain operations in
theoremprovers. 9 Significant progress has been achieved here, which allows
open questions in mathematics to be answered (136, 143). Along similar
lines, VLSIand parallel architectural support no doubt will improve the
performance of theorem provers (85).
The second way to maketheoremprovers more usable is to relax our notion
of correctness. A very simple wayof doing this is to makea theorem-proving
programalwaysreturn an answerafter a certain amountof time. If it has been
unable to proveeither that a sentence or its negation is implicit in the KB,it
could assume that it was independent of the KBand answer "unknown"(or
maybereassess the importance of the question and try again). This form of
error (i.e. one introduced by a resource-limited theoremprover), is not nearly
as serious as returning a "yes" for a "no", and is obviously preferrable to an
answerthat never arrives. 1o

9Auseful introduction to ATPis (86). Anoverviewof the research in the area can be found
(142).
1°S¢¢ (108, 139) for a fuller discussion of resource-limited processing. Manyof these ideas
were eventually incorporated into the KRL(11) representation language.
Annual Reviews
www.annualreviews.org/aronline
KNOWLEDGE
REPRESENTATIONAND REASONING 263

However, from the point of view of KR, both of these are only pseudo-
solutions. Clearly, the first one alone does not help us guarantee anything
about an inferential service. The secondone, on the other hand, might allow
us to guarantee an answer within certain time boundsbut wouldmakeit very
hard for us to specify what that answer would be. If we think of the KR
service as reasoningaccordingto a certain logic, then the logic being followed
is immenselycomplicated (comparedto that of FOL)when resource limita-
tions are present. Indeed, the wholenotion of the KRsystemcalculating what
is implicit in the KB(whichwas our original goal) wouldhave to be replaced
by someother notion that went beyondthe truth theory of the representation
language to include the inferential power of a particular theorem-proving
program. In a nutshell, we can guarantee getting an answer, but not the one
we wanted.

2.3 Knowledge Representation as Compromise

So there are serious problemswith a KRservice that attempts to calculate in a
reasonable amountof time what is logically implicit in a KB.In this review
we demonstrate that muchof the research in KRcan be construed as trading
off a certain generality offered by FOLfor a moretractable formof inference.
Tosee wherethis trade-off betweenexpressivenessand tractability originates,
we should first look at the use of the expressive powerof FOLin KRand how
it differs from its use in mathematics.
In the study of mathematical foundations, the main use of FOLis in the
formalization of infinite collections of entities. So, for example, we have
first-order theories of numbersand sets that use quantifiers to range over these
classes, and conditionalsto state the properties that these entities have. This is
exactly howFrege intended his formalism to be used.
In KR, on the other hand, the domains being characterized are usually
finite. The powerof FOLis used not so muchto deal with infinities but to deal
with incompleteknowledge(77, 78, 102). t~ Considerthe kind of facts 12 that
might be represented using FOL:
1. -~Student(john).
Sentence 1 says that John is not a student, without saying what he is.
2. Parent(sue,bill) ~/Parent(sue,george).
Sentence2 says that either Bill or Georgeis a parent of Sue, but it does not
specify which one is the parent.

~IAKBis saidto beincompleteif it tells usthatoneof a (possibly

infinite)set of sentences
true withouttelling us which.
~2Theuseof FOL to captureterminology or lawsis somewhat
different[see(20)for details].
Annual Reviews
www.annualreviews.org/aronline
264 LEVESQUE

3. Bx Cousin(bill,x)/k Male(x).
Sentence3 says that Bill has at least one male cousin, but it does not say who
that cousinis.
4. ~’x Friend(george,x) D By Child(x,y).
Sentence4 says that all of George’sfriends have children, without saying who
those friends or their children are, or even if there are any.
The mainfeature of these examplesis that FOLis used not to capture complex
details about the domainbut to avoid having to represent details that maynot
be known.The expressive power of FOLdetermines not so muchwhat can be
said but what can be left unsaid.
For a system that has to be able to acquire arbitrary knowledgein a
piecemealfashion, there maybe no alternative to full logical reasoningwith a
language as expressive as FOL.But we maybe able to get by with muchless.
In what follows, we examineKRresearch by considering three distinct ways
of reducing the computational demandson a general KRreasoning service.

1. special-purpose languages By limiting what can be expressed in a KR

language, we can rule out certain forms of weakstatements allowed by full
FOL. Certain types of incomplete knowledgemay not be representable,
but full logical reasoning maynowbe within reach. The result, then, is a
KBthat is forced to be morecomplete because of the inexpressiveness of
its representation language.
2. models of limited reasoning Another possibility is to leave the ex-
pressiveness unchanged,but to give up trying to ferret out all the im-
plications of an incompleteKBby considering a form of implication that is
weakerthan full logical consequence.Becauseonly someof the possibili-
ties admitted by an incomplete KBwill be considered during reasoning, an
incomplete KBends up being treated as if it were more complete.
3. defaults and assumptions Finally, we can leave the language and its
logic unchangedbut can transformthe incompleteKBitself into one that is
more complete. By making assumptions based on the absence of contra-
dicting evidence, a KRsystem maycome to conclusions not necessarily
implied logically by the original KB.Theresult of this type of reasoningis
that an incomplete KBwill be mademore complete by filling in the holes
in what it knows.

So the view of KRwe are considering here is certainly based on logic, but not
necessarily classical logic. Rather than dealing with a very expressive lan-
guage (i.e. FOL),and inferences that are logically complete and sound,
are concernedwith inexpressive languagesand with logics that are classically
incomplete and unsound. First, we will look at some special-purpose KR
Annual Reviews
www.annualreviews.org/aronline
KNOWLEDGE
REPRESENTATIONAND REASONING 265

languages, focussing on their expressive limitations. Next, we will look at

logics of knowledge and belief and how these can be made to provide a
weakernotion of implication than that of standard logic. Finally, we will
examinesome of the research in nonmonotonicreasoning in an attempt to
characterize the use of defaults and assumptions.In all cases, the mainpoint
is that there is muchto be gained by looking at the logical form of what is
being represented and howthe necessary reasoningcan be kept (or at least has
a hope of being kept) computationally tractable.
As for other sources on this topic outside of AI, there is unfortunately
almost nothing in philosophy on managingthe dual concerns of logic and
computational complexity [a notable exception is (27)]. There have been
results in theoretical computerscience and mathematicallogic in special cases
of logics that are computationallymanageable(e.g. see 83), but none of these
results focus on whetherthese special cases are representationally significant.
Althoughthe issues addressed here provide a very useful perspective on KR
research, they are only beginning to be examinedin depth.

3. SPECIAL-PURPOSE LANGUAGES

A KRlanguage with restricted expressive power is simply a language whose

expressions can be understood propositionally, but without being closed
underthe normallogical operations of disjunction, existential quantification,
and so on. Given that various types of incomplete knowledgewill no longer
be representable in such a language, the logical consequencesof a KBcan
often be calculated using efficient special-purpose techniques, for example,
avoiding, time-consuming(or infinite) case analyses and their equivalents.
Consider, for instance, information that we mayhave about two positive
integral quantities mand n expressedin a first-order logical languagethat has
symbols for the arithmetic operations and relations. So, for example, we
might knowthat 2m- n = 6 and that either mn-- 80 or m2 < 5n. If we call
these two facts E and add to them Q, the single axiomof Robinson’s system
of arithmetic (97), then what follows logically is a numberof properties of
and n, such as that 2m+ n > 20. This meansthat this property (call it P)
implicit in a KBthat contains E and Q. Moreover,standard theorem proving
(among other methods) can be used to discover this, since ((E/~ Q)
must be a theoremof first-order logic.
But suppose that instead of E, we knowE’, which says that 2m- n = 6
and rn + n = 15. Again, P follows from E’ and Q, and again, we could
determine this using first-order theorem proving. But since our knowledge
about mand n is expressible as a pair of linear equations, there is a much
better wayto do this: Solve the set of equations (using Gaussianelimination
Annual Reviews
www.annualreviews.org/aronline
266 LEVESQUE

with back substitution) and calculate directly whetheror not P is true of these
values. In general, k equations can be solved in roughly k3 operations, and
assuming a unique solution, P can be checked in a linear number of op-
erations. 13 Specialized techniques like this are not possible in general, and
sets of linear equations are only a small subset of whatcan be expressedin the
full languageof first-order arithmetic. But it is nonetheless a very rich and
useful subset. So it makes sense to build systems whoseinput language is
restricted to lie in this subset.
Muchof the work in KRhas involved inventing new KRformalisms and
embeddingthese within KRsystems. In retrospect, a good proportion of this
research can be seen as the search for useful compromisesbetweenexpressive
power, on the one hand, and tractability of reasoning, on the other (80).
fact, with the possible exception of nonmonotonicfacilities (see Section
below), these formalisms can almost inevitably be understood as subsets of
classical first-order logic.

3.1 Databases
The most obvious restriction to the form of a KBis what might be called
database form. The idea is to restrict a KBto contain only the kind of
information that can be represented in a simple standard database. Consider,
for example,a very trivial database that deals with university courses. If we
had to characterize in FOLthe information contained in the database, we
might use a collection of function-free atomic sentences like
Course(csc248)Dept(csc373,ComputerScience) Enrollment(psy400,42)
. .
Course(matl00)Dept(his 100,History) ...

In other words, a standard tabular database characterizes exactly the positive

instances of the various predicates. But moreto the point, since we neve.r end
up with sentences like
Dept(matl00,Mathematics) ~/ Dept(matl00,History),
certain kinds of incomplete knowledgecannot be represented.
There is, moreover, additional hidden information in a standard database
that makes it even more complete. Consider the question
Howmany courses are offered by the Computer Science Department?
The knowledge expressed by the above FOLsentences is insufficient to
answer the question: Nothing says that ComputerScience has at least two
courses (since "csc373" and "csc248" could be namesof the same individual),
and nothing says that it has at most two courses (since there could be courses

13Thiswill be true of any P that can be evaluated directly oncethe values of its variables are
known.So, for example, any P that does not use quantifiers has this property.
Annual Reviews
www.annualreviews.org/aronline
KNOWLEDGE
REPRESENTATIONAND REASONING 267

other than those mentionedin the list of sentences). But a database system
could answerthe question successfully by interpreting it as (somethinglike)
Howmanytuples in the COURSE
relation have ComputerSciencein their
Dept field?
This is a questionnot about the world being modelledbut about the data itself.
To be able to reinterpret it as the intuitive question about courses and
departments(rather than as one about tuples and fields), we need to account
for additional informationtaking us beyondthe stored data itself. In particu-
lar, we need FOLsentences of the form

ci 4: cj, for distinct constants ci and cj,

stating that each constant represents a uniqueindividual. In addition, for each
predicate, we need a sentence similar in form to
~’x[Course(x) D x = csc248 ~/ . . . ~/x = matl00],
saying that the only instances of the predicate are the ones namedexplicitly. 14
If we nowconsider a KBconsisting of all of these sentences, a KRsystem
wouldindeed conclude (just like its database management counterpart) that
there were exactly two ComputerScience courses.
The main observation here is that answering questions with a KBin
database form is mucheasier than in the general case. Becausethere is no
incompleteness in our knowledge,inference reduces to simple calculation.
Wedo not, for instance, have to reason by cases or by contradiction, as we
might in a more general setting, and we can represent symbolically what is
knownabout the world using sets of tuples, exactly like a standard database
system. Then,to infer the numberof courses of a certain type, all we have to
do is count howmanyappropriate tuples appear in the COURSE relation.
Fromthis perspective, a database is a knowledgebase whose limited form
permits a very special form of inference.
Althoughdatabase languages are almost always too weakfor AI purposes,
the limitations they imposeon the logical form of a KBare found in manyKR
languages. There is very close structural correspondencebetweensomepart
of the KBand the domainof interest: For each entity in the domain,there is a
uniquerepresentational object that stands for it; for each relationship that it
participates in, there is somesort of connectionin the KBthat correspondsto
it. In a very real sense, that part of the KBis an analogueof the domainof
interest, not so different from other analogues such as maps or physical
models. The mainadvantage of having such an analogue is that it can be used
directly to answer questions about the domain. Calculations on the model

i’ll’his is one form of what has been called the closed world assumption (118).
Annual Reviews
www.annualreviews.org/aronline
268 LEVESQUE

itself play the role of more general reasoning procedures (much the way
arithmetic can replace reasoning with Peano’s axioms). The disadvantage of
an analogue, however,should also be clear: There are certain kinds of facts
about the domainthat it cannot leave unsaid.~5 In this sense, an analogue
representation can be viewedas a special case of a propositional one, where
the information it contains is relatively complete.

3.2 Logic Programs

The second restriction on the form of a KBthat we will consider is a
generalization of the previous one found in programs written in PROLOG
(73), PLANNER (62), and related languages. 16 A KBin logic program
contains a collection of first-order sentences (called Hornsentences) of the
form
Vx! . . . xn[P1/~¯ ¯ ¯/~ Pm~ Pro+1] wherern -> 0 and each Pi is atomic.
In the case wherem= 0 and the argumentsto the predicates are all constants,
the logic programform coincides with the database form. As in the database
case, since we are interested in morethan just the universe of terms, we again
haveto include additional facts in the KB.In general, the resulting KBwill be
infinite, since it must contain all sentences of the form (s 4 t), for any two
distinct terms formedusing function and constant symbols appearing in the
KB. As before, the KBmust also contain a version of the closed world
assumption, which is nowthe negation of every ground atomic sentence not
implied by the Horn sentences in the original KB.
The net result is a KBthat once again has complete knowledgeof the world
(within a given language),17 but this time it requires nontrivial inference to
answer questions. The necessary reasoning is the execution of the logic
program. For example, given a KBin logic program form consisting of
parent(bill,mary)
parent(bill,sam)
parent(X,Y),female(Y) D mother(X,Y).
female( mary).

l~Thesameis true for the standardanalogues. Amapdoesnotallowyouto say,for example,

that a river passesthroughoneof twowidelyseparatedtowns,withoutspecifying whichtown.
Similarly,a plasticmodelof a shipcannottell usthat the shipit represents doesnothavetwo
smokestacks,withoutalso telling us howmany it doeshave.Thisis notto saythat thereis no
uncertaintyassociatedwithan analogue,butthat this uncertaintyis limitedin variousways.See
(82, 129,130)for moreinformation andsee (51)for the useof analogues in problemsolving.
16SeeSection5.2 for anotherviewof the procedural schoolof KR.
’TNoticethatit is impossible
to statein, a KBof this formthat (Pk/Q)is true without
stating
whichoneis true, or that 3xP(x)is true withoutsayingwhatx is.
Annual Reviews
www.annualreviews.org/aronline
KNOWLEDGE
REPRESENTATIONAND REASONING 269

we knowexactly whothe motherof Bill is, but only after having executed the
program.
In somesense, the logic programform does not provide any computational
advantage to an FOLreasoning system, since determining if a ground atomic
sentence is implied by a collection of Horn sentences (containing function
symbols) is undecidable. 18 On the other hand, the form is much more
manageablethan in the general case, since the necessaryinference can be split
very nicely into two components:a retrieval componentthat extracts (atomic)
facts from a database by pattern-matching, and a search componentthat tries
to use the nonatomic Horn sentences to complete the inference. In actual
systems like PROLOG and PLANNER, moreover, the search component is
(partially) under user control, giving the user the ability to incorporate
domain-specific control knowledge.The only purely automatic inference is
the retrieval component.
This suggests a different wayof looking at the inferential service provided
by a KRsystem(without even taking into account the logical form of the KB).
Instead of automatically performing the full deduction necessary to answer
questions, a KRsystemcould managea limited form of inference and leave to
the rest of the knowledge-based system (or to the user) the responsibility
intelligently completing it. As suggested in (50), the idea is to take the
"muscle" out of the automatic componentand leave the difficult part of
reasoning as a problem that the overall system can (meta-)reason about and
plan to solve (54). 19 It is clear that one of the majorattractions of PROLOG
its programmability, and its potential for integrating procedural and de-
2°
clarative concerns, rather than its poweras a KRlanguage.

3.3 Semantic Networks

Whereasdatabases and logic programshave been studied quite independently
of KR, semantic networks are muchmore of a pure KRphenomenon.21 The
first observation about a KBin this form is that it only contains unary and
binary predicates. For example,instead of representing the fact that John’s
grade in csl00 was 85 by

rain manysimpleExpertSystem applications,however, the problemis solvablebecauseno

functionsymbols (exceptfor constants)are used.
19Thistopicis discussedfurtherin Section4.
2°Formoreinformationon PROLOG as a programming language,see (29, 74).
2~Odginally, semanticnetworks wereproposed as modelsof associativememory (e.g. 112).
Graduallythey cameto be usedfor generalKRpurposes,but not withoutsomeconfusion
regardinghowtheywereto be interpreted(15, 16, 141).Fora collectionof researchpapers
demonstratingsomeof the varietyin semantic networkrepresentations,see (47).
Annual Reviews
www.annualreviews.org/aronline

270 LEVESQUE

Grade(john, csl00, 85),

we might postulate the existence of objects called "grade-assignments" and
represent the fact about John in terms of a particular grade-assignmentgl as
Grade-assignment(g0/~ Student(g~,john)
Mark(g~,85).
This pal~ of a KBin semanticnet formis also in database form: a collection of
function-free ground atoms, along with sentences stating the uniqueness of
constants and closed world assumptions.
The mainfeature of a semantic net (and of frames, below), however,is not
howindividuals are handled, but the treatment of the unary predicates (which
we will call types) and the binary ones (whichwe will call attributes). First,
types are organized into a taxonomy,which, for our purposes, can be treated
as a set of sentences of the 22
form
VxiB(x) A(x)].
Finally, there are sentences that place constraints on an attribute as it applies
to instances of a type:
~¢x[B(x) ~ 3y(R(x,y) V(y))] or ~/x[B(x) ~ R (x,c)].
Oneproperty of a KBin this form is that it can be represented by a labelled
directed graph (and displayed in the usual way). The nodes are either con-
stants or types, and the edgesare labelled either with an attribute or with the
special label is-a. Thesignificance of this graphical representation is that it
allows certain kinds of inference to be performedby simple graph-searching
techniques. For example, to find out if a particular individual has a certain
attribute, it is sufficient to search fromthe constant representingthat individu-
al, up is-a links, for a node having an edge labelled with the attribute. By
placing the attribute as high as possible in the taxonomy,all individuals below
it can inherit the property. Computationally,any improvement in the efficien-
cy of graph-searching will improvethe performance of inference in a KBof
this form (although in general, of course, graph-theoretic operations can be as
intractable as any other operations).
In addition, for better or for worse, the graph representation suggests
different kinds of inference based more directly on the structure of the KB
than on its logical content. For example, we can ask for the connection
23
betweentwo nodes and answer by finding a path in the graph betweenthem.
Another exampleis to use the is-a taxonomyfor default reasoning. 24 For

22See(16, 59)for a discussionof someof the subtletiesinvolvedhere.

23Quillian(113)proposeda "semanticintersection"approach to answering
questionsin his
originalworkonsemantic nets. Seealso (30) for later workonthe sametopic.
24Defanlt
reasoningis discussedin detail in Section5.
Annual Reviews
www.annualreviews.org/aronline

KNOWLEDGE
REPRESENTATIONAND REASONING 271

instance, the elephant node can have a color link to the value gray, but
anythingbelowelephant (such as albino-elephant) can be linked to a different
color value. To infer the color of an individual only requires searching up the
taxonomyfor a value, and stopping whenthe first one is found, preempting
any higher values.25 The trouble with this type of reasoningis that, with only
a proceduralaccount like the one above, it is very easy to lose the import of
exactly what is being represented (17).

3.4 Frames
The final form we consider--frame descriptions--is mainly an elaboration of
the semanticnetworkform.26 The emphasis,in this case, is on the structure of
the types themselves(usually called frames) in terms of their attributes (now
called slots). Typically, the kind of detail associated with the slots of a frame
includes
1. values, stating exactly whatthe attribute of an instance should be. Alterna-
tively, the value maybe just a default, in whichcase an individual inherits
the value, provided it does not override it.
2. restrictions, stating what constraints mustbe satisfied by attribute values.
Thesecan be value restrictions, specified by a type that attribute values
should be instances of, or numberrestrictions, specified in terms of a
minimumand a maximumnumber of attribute values.
3. attached procedures, providing procedural advice on howthe attribute
should be used. Anif-needed procedureexplains howto calculate attribute
values if none have been specified; an if-added procedure explains what
should be done whena new value is discovered.
Like semantic networks, frame languages tend to take liberties with logical
form, and the developers of these languages have been notoriously lax in
characterizing their truth theories (17, 59). Restricting ourselves to a noncon-
troversial subset of a frame language, we might have descriptions like
(Student
with adept is computer-science and
with -> 3 enrolled-course is a
(Graduate-Course with adept is a Engineering-Department)).
This is intended to be a structured type that describes ComputerScience
students taking at least three graduate courses in departmentswithin Engineer-

25Formorecomplex examples
of reasoningbaseddirectly onthe formof semantic networks,
see(121)and(140).
26Thetheoryof framesin KRwasfirst presentedin (99), althoughmany of the ideaswere
already"in the air". KRL(11-13)andKL-ONE (14,23) are perhapsthe mostrepresentative
languagesbasedonthese ideas.
Annual Reviews
www.annualreviews.org/aronline

272 LEVESQUE

ing. If this type had a name(say A), we could express the type in FOLby
"meaning postulate" of the-form
~’xA(x)-= [Student(x)/~ dept(x, computer-science) /~
~YlY2Y3 (Yl -if= Y2 /’k Yl -¢ Y3 /~ Y2 -~ Y3
enrolled-course(x,yl) /~ Graduate-Course(y0/~
3z(dept(y~,z)/~ Engineering-Department(z))/~
enrolled-course(x,y2)/~ GradUate-Course(y2)/~
3z(dept(y2,z)/~ Engineering-Department(z))/~
enrolled-course(x,y3)/~ Graduate-Course(y3)/~
3z(dept(y3,z)/~ Engineering-Department(z)))].
Similarly, it should be clear howto state equally clumsily27 in FOLthat an
individual is an instance of this type.
Oneinteresting property of these structured types is that wedo not have to
explicitly assert that one of them is below another in the taxonomy. The
descriptions themselves implicitly define a taxonomyof subsumption, where
type A is subsumedby type B if, by virtue of the form of A and B, every
instance of A must be an instance of B. For example, without any word
knowledgeat all, we can determine that the type
(Person with every male friend is a Doctor)
subsumes
(Person with every friend is a (Doctor with a specialty is surgery)).
Analytic relationships like subsumptionare useful properties of structured
types that are not available in a semantic networkwhereall of the types are
essentially atomic (20).. In the KRYPTON KRlanguage (18, 19), a
first-order KBis used to represent facts about the world, but subsumption
information is also available. The reason for this is that while subsumption
can be defined in terms of logical, implication, 28 there can be very good
special-purpose "description matching"algorithms for calculating these rela-
tionships (21). Again,becausethe logical formis sufficiently constrained, the
required inference can be muchmore tractable.

3.5 Other Possibilities

Onefinal class of special-purpose language worth mentioning are those that
are geared to specific subject matters. For example, the representation of

27These sentences
are especiallyawkwardin FOLbecauseof the number reslrictions. For
example,the sentence"Thereare a hundredbillion stars in the MilkyWayGalaxy" wouldbe
translatedinto an FOLsentencewithonthe orderof lO22 conjuncts.
28Specifically,
typeAis subsumedbytypeB iff themeaning postulatesfor AandB logically
imply’#x[A(x)~ B(x)].
Annual Reviews
www.annualreviews.org/aronline

KNOWLEDGE
REPRESENTATIONAND REASONING 273

knowledgeabout time proposed by Allen (3) is clearly limited in what kinds

of knowledgeit can represent. The payoff is that there is a special-purpose
algorithmfor deciding if knowledgeabout a collection of temporalintervals is
consistent. In a similar vein, Schubert(126, 127) discusses a limited represen-
tation for knowledgeabout parts, where questions about whether or not one
object is an ultimate subpart of another can be answeredin constant time. Of
course, in somesense, the input languagesto linear equation solvers, circuit
simulators, and programcompilers can all be thought of as restricted repre-
sentation languages that admit very specialized forms of reasoning. Addition-
al domainindependentbut special-purpose reasoning situations are described
in (25).
One final aspect of this approach involves combining more than one
special-purpose language in what is called a hybrid system. Examplesof such
systems are the above-mentioned KRYPTON, CAKE(120), and Oppen’s
co-operating decision procedures (106). The issue in these systems is how
makemultiple reasoners co-operate, that is, solve complementary parts of a
reasoning problemby exchangingthe right sort of information, with a mini-
mumof redundancy.29 The promise of such systems (whichis only starting to
be realized) is the possibility of efficiently handlingrepresentation languages
that are more expressive than those of the subreasoners.

4. MODELS OF LIMITED REASONING

So far, we have considered keeping the reasoning task tractable by limiting

the KRlanguageand thus restricting the set of implications to be computedby
a KRservice. But there are other possibilities. In this section, we consider
another approach that leaves the language intact but uses a weaker form of
implication.
Technically, there are at least two waysto achieve this. Theobvious one is
to use (or develop)a nonstandardlogic with a different semanticsleading to
weakernotion of logical consequence. However,this wouldforce us to give
up all of the usual properties of classical logic. Anotherapproach, and one
that has received moreattention in KR,is to preserve the standard notion of
logical consequence, but to augmentthe language with a knowledgeor belief
3°
operator, so that the sentence Bc~is to be read as "the KBbelieves that a".
The idea is to take seriously the concept of belief built into the term "know-

29Hybrid systemsshouldbe distinguishedfromthosewithmultiplerepresentation languages

that donotattemptto avoidredundancy or to passinformation
amongmultiplereasoners,suchas
the systemsdescribedin (1, 5, 26).
3°Strictlyspeaking,
the term"knowledge"is inappropriate,
sincewhatis represerttedin a KB
is typicallynotrequired
to betrue;at theveryleast,weshouldbetalkingaboutbelief.Becauseof
this, andto be consistentwithmostphilosophers, weusethe term"belief’throughout.
Annual Reviews
www.annualreviews.org/aronline

274 LEVESQUE

ledge representation" and to formalize explicitly a weakersense of belief in

terms of the semantics of Bt~. So, instead of basing a KRservice on whether
or not a is a logical consequenceof a set of sentences S, we consider whether
or not believing a is a logical consequenceof believing the elements of S
31
according to the augmentedlogic.

4.1 Logical Omniscience

Thefirst formal logic of belief along these lines originated with Hintikka(64)
and was based on a possible-world semantics. 32 The idea, roughly, is to
acknowledgein the semantics that things might have happened differently
from the way they did in fact occur, and possible worlds are theoretical
entities that index into this space of possibilities. The importantpoint is that
within a given interpretation, certain sentences can be true in somepossible
worlds and false in others. A belief state, then, is characterized by a set of
possible worlds, and what is considered to be believed in a belief state is
precisely what is true in all of those words. So having no opinion on some
sentence simply meansthat it will comeout true in some of these possible
worlds and false in others.
As is, however, the possible-world model does not formalize a type of
belief that is morerestricted than the KRservice described so far. It suffers
from logical omniscience (65): At any given point, the set of sentences
consideredto be believed in this modelis closed under logical consequence.It
is built into possible-world semantics that if a is believed and ct logically
implies/3, then/3 is believed as well. A numberof attempts have recently
been madeto formalize a more limited notion of belief. Before looking into
them, it should be noted that the main emphasisin this research has been to
accurately modelthe beliefs of an agent, and none of these modelshave yet
been applied in a direct way to the design of new KRarchitectures.

4.2 The Syntactic Approach

Whentalking about what an agent actually believes, we want to be able to
distinguish between believing only ~ and (c~ D /3) on the one hand, and
believing t~, (t~ ~/3) and fl, on the other. Whilethe picture of the worldis the
samein both cases, only the secondinvolves realizing that/3 is true. This is
somewhatof a problemsemantically, since the two sets of beliefs are true in
precisely the same possible worlds and so are, in somesense, semantically
indistinguishable. Thus, in the possible-worlds approachto belief (as in 77,

31Another
reasonfor preferringthis approach
is that it quitenaturallygeneralizes
to dealwith
beliefsaboutotherbeliefs(byusinga Boperatorwithinthe scopeof another).Thisallowsfacts
aboutself-knowledge to be expressed,whichhas applicationsdiscussedin Section5.
32See (67) for an introductionto possibleworldsandthe modallogicsbasedonthem.
Annual Reviews
www.annualreviews.org/aronline
KNOWLEDGE
REPRESENTATIONAND REASONING 275

100, 101), it is impossibleto distinguish betweenthe twocases. But if the two

belief sets are semanticallyindistinguishable, they are certainly syntactically
distinguishable. This might suggest that any realistic modelof belief will have
to include (somethingisomorphicto) an actual set of sentences to distinguish
between the two belief states above. In other words, part of what will
constitute a belief state in this syntactic approachto belief are the actual
sentences that are believed.
A numberof variants of the syntactic modelof belief have been proposed
(32, 40, 43, 91, 104). Perhaps the most sophisticated one is by Konolige
(70-72). In his case, a belief state is characterized by an initial set
sentences (representing a base set of beliefs) and a set of logically sound
deduction rules for obtaining new derived beliefs. Logical omniscience is
avoided by allowing the deduction rules to be logically incomplete. Among
other things, Konolige is able to demonstrate how manyvariants of the
possible-world modelcan be seen as special cases of his model, where the
deduction rules are required to satisfy certain properties. Moreover,Kono-
lige’s model allows introspection to be characterized very naturally and
concretely using a submodel(with different base beliefs and deductionrules),
whichis the system’s modelof itself.
With or without deduction rules, the syntactic approach suffers from a
serious defect that is, in somesense, the oppositeof the problemwith possible
worlds. A possible-world semantics is too coarse-grained to modelbelief in
that it cannot distinguish belief sets that logically imply the same set of
sentences. The syntactic approach, on the other hand, is too fine-grained in
that it considers any two sets of sentences as distinct semantic entities and,
consequently, different belief sets. In general, Bct will logically imply
only if a and/3 are identical. So, for example,a systemthat believes that (ct
/3) is true is not even required to believe that (/3 k/ a) must also be true.
Of course, the syntactic approach can be embellished by requiring that
belief sets satisfy certain additional properties, for example,respecting the
commutativityof disjunction. The trouble with this kind of stipulation is that
it does not showthe source of this property, namelythat a disjunction is true
exactly whenone of the disjuncts is. Clearly, it wouldbe preferable to have a
modelof belief whererestrictions on belief sets followed from the definition
of belief, that is, a modelbasedin somewayon a conceptof truth rather than
on a collection of ad hoc restrictions to sets of sentences.

4.3 Impossible and Partial Worlds

Recent research in KRhas attempted to deal with the problem of logical
omniscience by using a generalized notion of possible world. Logical om-
niscience (and the attendant computationaldifficulty) is a direct result of the
characterization of belief states as sets of possible worlds. If the agent only
Annual Reviews
www.annualreviews.org/aronline

276 LEVESQUE

believes that a is true, the set of worlds will be all those wheret~ is true:
some, for example, where /3 is true, others, where /3 is false. However,
because valid sentences will also be true in all of these possible worlds, the
agent is thought of as believing themjust as if they were amonghis actual
beliefs. In terms of the possible worlds, there is no wayto distinguish t~ from
a valid sentence.
Onesolution is to makethis notion of what an agent thinks the world is like
be morerelevant to what he actually believes. This can be done by replacing
the possible worlds by a different kind of semantic entity that does not
necessarily deal with the truth of all sentences. In particular, sentences not
relevant to what an agent actually believes (including somevalid ones) need
not get a truth value in one of these partial possible worlds [whichwe call
situations, following (7)]. In fact, we can think of possible worlds as those
limiting cases whereevery sentence has a truth value. Indeed, the concept of a
possible world being compatible with a situation is intuitively clear: Every
sentence whose truth is supported by the situation should be true in that
possible world, and every sentence whosefalsity is supported should be false.
Also, we can allow for situations that have no compatible possible worlds.
Theseare situations that support both the truth and falsity of somesentence.
Althoughthey can never be real, such impossible situations can be imagined
and are very useful, since they allow an agent to havean incoherent picture of
33
the world.
The "trick," then, that underlies the modelsof belief is to identify belief
with a set of situations rather than with a set of possible worlds. This has the
following effect (roughly): Not all valid sentences have to be believed, since
situations can fail to support themby being partial. Also, becauseof impossi-
ble situations, beliefs need not be closed under logical consequence. For
example,a situation can support both a and (~ct ~//3), without supporting/3,
by supporting both a and its negation.
Althoughthere were certainly antecedents in the philosophical literature
(e.g. 114), the first proposal for using this kind of logic in a computersystem
was by Belnap (8). He sought to use relevance logic (4), which embodies
these situations (in terms of four truth values), as a meansof dealing with the
partiality and potential inconsistency of information presented to a machine.
The relation of logical consequencein this type of relevance logic (called
tautological entailment) is a strict subset of the relation of logical conse-
quence in classical logic. Levesque(81) showedhowto use these ideas
establish a formal modelof belief parallel to the one based on possible worlds.
More importantly, Levesque described a KRservice based on this weaker
notion of implication and proved that the required inference was indeed more

33Althoughthey are used here, impossible situations are not strictly necessary to model
inconsistent beliefs, as is discussed later.
Annual Reviews
www.annualreviews.org/aronline
KNOWLEDGE
REPRESENTATIONAND REASONING 277

computationally tractable (for a language without quantifiers) than the one

based on classical logic.
As the foundation for a KRservice, relevance logic does have serious
drawbacks,however.As mentionedabove, it is the impossible situations that
allow beliefs to be open under logical consequence:Anyagent whosebelief
state is characterizedby a set of possiblesituations alwaysbelieves fl if c~ and
(~c~ ~/ /3) are believed. Intuitively, one wouldlike to say that failing
believe/3 results fromfailing to put the other twobeliefs together, rather than
allowing for the possibility that both a and its negation might be true. The
idea is to have a modelwherebelieving a conjunction is not the samething as
believing both of the conjuncts; it should require putting the two conjuncts
together. This type of modelis presented in (43) and illustrates howlogical
omnisciencecan be avoided without requiring impossible situations.
Anotherproblemwith the taulogical entailment relation of relevance logic
is that the obvious first-order version of it is undecidable(110). This means
that a computationally manageablemodelof belief dealing with full first-
order logic cannot be derived simply from propositional versions. Three
(related) proposalsfor dealing with full first-order logic are foundin (49,
110). Finally, the relevance logic approach by itself gives no indication of
howto deal with beliefs about beliefs, for suggestions in this regard see (43).

5. DEFAULTS AND ASSUMPTIONS

As discussed earlier, avoiding incomplete knowledgeis an important way to
keep the reasoning of a KRservice tractable. In fact, in manycases where
knowledgeis incomplete, a useful strategy is to makeassumptions to com-
plete this knowledge,to reason with the enlarged (tractable) belief set, noting
whatassumptionsare in effect, and perhaps later to retract those conclusions
derived from assumptions invalidated by new information. For example,
given only that Clydeis an elephant, one mayassume(until there are reasons
to believe otherwise) that Clydeis gray. Anyreasoning that then dependson
this color wouldnot need to be duplicated for the gray and nongraycases. The
hypothesisis that it will be sufficiently advantageousto deal with relatively
completeKBseven if it is occasionally necessary to retract assumptions(and
conclusions drawnfrom them) in the face of conflicting evidence. In other
words, provided that assumptionsare chosen judiciously, there will be real
computationaladvantage to building systems that do not allow for all of the
possibilities admitted by the information they possess.
5.1 The Source of Assumptions
This way of completing a KBpresumes that there will be principles for
selecting assumptionsthat are most likely to be correct and least likely to
require (costly) retraction. Fortunately, this seemsto be the case. The most
Annual Reviews
www.annualreviews.org/aronline
278 LEVESQUE

obvious source of assumptions are defaults, which comein a numberof forms

(117):
the closed world assumption (see Section 3.1) Given a collection of facts
about some situation, a commonassumption is that any relationship that
cannot be inferred to hold amongthe entities being discussed in fact does not
hold (116). Amongother things, this assumption allows one to avoid having
to specify relationships that do not hold. 34 A similar assumptionappears in
someis-a hierarchies in semantic networks (16), where concepts are assumed
to be disjoint unless they share descendants. Other related assumptionsare
domain closure (90), where entities whose existence is not implied are
assumednot to exist, and the unique nameassumption (116), where distinct
namesare taken to represent distinct entities.
the frame assumption Any system that has to reason about the effects of
actions must deal with what exactly is not changedby the occurrence of an
action. Since any action will tend to have a very localized effect, a common
assumptionis that nothing is changedby an action except for what is knownto
be changed. So, for example, movingan object normally does not change its
color and, even less likely, the color of other distant objects. This type of
assumption was built into someAI planning systems such as STRIPS(46).
However,while the problem of circumscribing the effects of actions is one
that was recognized very early in AI [see (94) or (115)], it remains to
solved satisfactorily [see (35) and (48) for suggestions as to why].
prototypes Much of what we know about various categories of objects
involves properties that almost alwaysapply. So, for example,polar bears are
normally white, birds normally fly, and so on. Also, there seemsto be fairly
clear evidence that people understand the world based very muchon notions
of normality and abnormality (122). While there have been proposals in the
literature for dealing with these probabilistically (53, 55, 144), another
possibility is to understandthese probabilities as (meta-theoretic) evidencefor
whythese defaults are reasonable assumptions to make. That is, if there is
sufficiently high probability that a polar bear will be white, then there is an
equally goodchance that the default assumptionwill not have to be revised.
locality assumptions Another powerful source of default assumptions is
based on aspects of a "current" context. In communication,one can often fill
in details in an elliptical utterance based on using "here," "now," and "me"
whenthe place, time, or person involved is not mentioned. Moregenerally,
the context of conversationvcill introduce entities that can be used as defaults
in later discourse (e.g. see 125). Similarly, whenplanning to achieve
effect, appropriate instruments might default to those readily at hand.
34This assumption
is certainlyusedin standarddatabases whereonlypositiveinformation
is
stored.Thisis also relatedto the "negationas failure" of PROLOG
[28, 119].
Annual Reviews
www.annualreviews.org/aronline

KNOWLEDGE
REPRESENTATIONAND REASONING 279

Overall, because there are so manysources of default assumptions that do

indeed seem to work in a majority of cases, reasoning in this mannerseems
perfectly justifiable. In fact, Reiter claims(117, p. 218) that "Defaultreason-
ing mayvery well be the rule rather than the exception since normallywe act
in the presence of incomplete knowledge."
Finally, as pointed out in (82), there seemto be cases whereassumptions
can be madewithoutthe benefit of the statistical justification of defaults. If it
turns out that the subject of a potential assumptionis not likely to be the
subject of later information, it maystill be worth makingsomeassumptionfor
computational reasons, as it will be unlikely later to give rise to a conflict. So,
for example, given information that a dog chased a cat around a tree, one
might assume that the direction of the chase was clockwise, even though
statistically, it could go either way.35 Of course, the real trick here will be to
find principles for makingassumptionsthat take into account not only possi-
ble computationalgains but also the likelihood of errors and their severity.

5.2 Truth Maintenance

One branch of KRresearch has focussed on appropriate mechanisms for
managingfamilies of assumptionsand keeping track of justifications to allow
a graceful backing out. Thesehave been called truth maintenancesystems (or
sometimesreason maintenancesystems) (36, 37, 88). Of particular interest
this area are those systems that are explicit about the assumptions being
carded or being excluded, such as (33) and (87). 36 Most of this work,
however, deals with efficient mechanismsfor actually implementingtruth
maintenance. This in turn derives from earlier work on procedural repre-
sentations of knowledge,where a major focus of attention was the control of
reasoning, especially during the kind of backtrackingthat arises as a result of
defeated assumptions. This procedural school began more or less with the
PLANNER representation language (62, 63) and received perhaps a last push
in the AMORD system (34). A general discussion of the issues involved
this approachto representation can be foundin (57, 102, 138). Ultimately, the
problemof truth maintenanceis that of belief revision and theory evolution:
what beliefs should be discarded in the face of conflicting evidence [e.g. see
(2) and also (38) for a survey of workin this area].

35Indeed,in "visualizing"
a describedsituation,weare filling in a largenumber of visually
significantdetails that neednot be impliedbyanythingin the originaldescriptionnor by
statisticallyrelevantdefaults.So,for example,it is hardto thinkaboutRonald Reagan standing
besideMargaret Thatcherwithoutthinkingof themfroma specificpointof viewwithoneof them
ontheleft andthe otheronthe right.
36Interestinglyenough,ShapiroandMartinsarguethat relevancelogicis the appropriate
framework for maintaining
a networkof assumptions, quiteindependently of its role in limiting
reasoning(as discussedin Section4.3).
Annual Reviews
www.annualreviews.org/aronline

280 LEVESQUE

5.3 Nonmonotonic Reasoning

The major conceptual issue faced by KRsystems that must reason with
assumptionsis what kinds of assumptionsthere are and what is the appropri-
ate logic for reasoning with them. This has led to the developmentof a branch
of AI called nonmonotonicreasoning. The idea, roughly, is this. Ordinary
reasoning in a KBSis monotonic: The set of conclusions that can be drawn
from a body of information grows as the body of information grows (since
anything logically implied by a set of sentences is also implied by a superset
of that set). But when the body of information in question also contains
potential assumptionsto be made(barring informationto the contrary), the set
of conclusions need not grow monotonically, since new information mayvery
well invalidate previous assumptions.
The area of formal nonmonotonicreasoning has recently becomea very
active and fruitful one. Although no KRsystems have been designed to
incorporate the general principles, and very little is currently knownabout the
impact of nonmonotonicreasoning on the tractability of the reasoning prob-
lem, a numberof formalisms capturing different aspects of the problemhave
been proposed and analyzed.
Anearly collection of workin this area appears in (9). In this volume,three
formalisms for nonmonotonicreasoning are proposed. All of them are based
on modificationsto classical first-order logics, but unfortunately noneof them
enjoy the benefit of a semanticaccount. The first proposal, by Reiter (118),
called default logic. Theoriesin this logic consist of two parts: a base set of
standardfirst-order sentences, and a set of default rules specified by triples of
formulas. The proof-theoretic understandingof a default rule is that given (an
instance of) the first formula, the third formulacan be inferred provided that
the secondone has not been. Anextension of a default theory is any consistent
minimalset of sentences containing the base set, which is both deductively
closed and closed under the default rules. Reiter’s interpretation is that any
such extension constitutes a reasonable set of beliefs given the base set and
defaults, and he showshowcertain default theories mayhave none or more
than one such extension. The nonmonotonicityof default logic arises from the
fact that an extendedbase set mayprohibit the application of certain default
rules.
The second nonmonotonic formalism, called nonmonotonic logic, was
proposed by McDermott& Doyle (96). In this case, a standard first-order
language is augmentedwith a unary sentential operator M, whereMt~is to be
read as "a is consistent." Their version of an extension of a base set of
sentences is any minimal superset that is deductively closed and has the
property that Mais an element of the extension whenever -~a is not an
element. Unlike Reiter, the authors identify belief with the set of sentences
Annual Reviews
www.annualreviews.org/aronline

KNOWLEDGE
REPRESENTATIONAND REASONING 281

that are membersof all such extensions, even though such a belief set
need not itself be an extension. The nonmonotonicityarises here from the
fact that as the base set grows, fewer Mot sentences will be added to an
extension.
The third formalismis called circumscription and is due to John McCarthy
(92). Theidea is to take any finite set of first-order sentencesas the base and
"circumscribe"a predicate in it by addingto the base an infinite collection of
sentences (as specified by a circumscription schema). The intent of these
sentences is to makethe extendedtheory state that the predicate is as small
as possible, given the base set. The way this is done for a predicate P is
to include in the collection a sentencefor each openfirst-order formulath with
the same number of arguments as P. What this sentence says is that if
the base sentences with P replaced by ~b are true and th is a subset of P, then
t h and P are equal. The net effect is to prohibit any property (specifiable by
an open formula) from simultaneously satisfying the base sentences and be-
ing a proper subset of the circumscribed predicate. In other words, the ex-
tended theory makes the circumscribed predicate minimal. This is nonmon-
otonic, because any change to the base set (additive or not) will lead to
different extendedtheory, since every sentence in that theory uses the base
set directly.
Since the publication of these three formalisms, there has been extensive
research on their properties. It is generally agreed that the first two have
serious drawbacks. In Reiter’s model, domain-dependentknowledgehas to
be encoded in inference rules, more or less ruling out a semantic account
for the logic (103); McDermott& Doyle’s proposal appears to be based
on a notion of consistency that is inappropriate for the job but is best un-
derstood in terms of a logic of knowledgeor belief (103). Circumscription
remains the most successful of the three and certainly the one that has
received the most attention (e.g. 41, 84, 119). It does have expressive draw-
backs, however,and new versions of it are under active investigation (e.g.
93, 98).
Gainingin popularity is an attempt to understandnonmonotonicityin terms
of explicit knowledgeand belief. Thus, in (56, 69, 77, 103) a general form
nonmonotonicity is explained in terms of the inherent nonmonotonicity of
self-knowledge. The idea is that certain kinds of assumptionsare based on a
lack of other beliefs (31). For example,one might be willing to believe that
there is no city in NewZealandlarger than Los Angelesbased on a belief that
one wouldknowif there were such a city. So belief in the lack of a certain
belief is sufficient to make the assumption. But this is a nonmonotonic
process, since beliefs about what one does not believe must surely be revised
as new beliefs are acquired. On the other hand, it is not clear that all
Annual Reviews
www.annualreviews.org/aronline

282 LEVESQUE

nonmonotonicreasoning behaves this way. For example, the fact that birds
generally fly seemsto have nothing to do with what is or is not believed at any
given point.
At this stage of research, the computationalrelevance of this workremains
somewhatquestionable. For the three formalisms described above, it appears
(counter-intuitively) that reasoning with assumptions and defaults is even
more difficult than reasoning without them.37 From a computational stand-
point, the formalizations seem to be movingin the wrongdirection. Not that
systems have been unable to take computational advantage of defaults. As
discussedin Section 3.3, the use of defaults goes back to Quillian and the very
early semantic networks (112), and it remains a major feature of inheritance
hierarchies (16, 44, 132, 133). Defaults were also a major part of the frame
concept (99) as can be seen in representation languageslike KRL(11). But
has often been much easier to construct systems that reason in certain
(nonmonotonic)waysthan to justify the correctness of that reasoning. Indeed,
systems whose behavior seemedappropriate at first glance were later shown
to exhibit reasoning anomalies (42, 45). So it remains to be seen howthe
computational promise of nonmonotonicreasoning can be correctly realized.

6. CONCLUSION

Accordingto the KnowledgeRepresentation hypothesis, intelligent behavior

ultimately depends on explicitly represented knowledge. No other design
strategy as yet seems plausible to explain behavior, or have it dependin a
flexible wayon whatone is told, or to isolate the assumptionsthat governit,
or to achieve any numberof other desirable traits. But KRby itself does not
solve anything unless a systemis able to reason effectively with what it has
explicitly represented. Specifically, if reasoning needsto be performedauto-
matically as part of a larger task, it must be dependableboth in terms of what
it calculates and howlong it takes. Thus, KR,as it has beenpresented here, is
the study of what information can be extracted, in a computationallydepend-
able way, from what forms of represented knowledge. In other words, it
investigates the area within the confines of the KRhypothesis. In somesense,
theorem proving with classical first-order logic is the base campfor this
investigation, but as this reviewattempts to show,there is muchto be learned
from excursions into nearby regions.

37In fact, in the first two cases, applying an assumptiondepends on that assumption being
consistent with sometheory of the world. But for sufficiently expressive languageslike that of
FOL,the consistent sentences are not even recursively enumerable.
Annual Reviews
www.annualreviews.org/aronline

KNOWLEDGE REPRESENTATION AND REASONING 283

Literature Cited

1. Aikins, J. 1983. Prototypical knowledge 15. Brachman, R. 1979. On the epistemo-

for expert systems. Artif. lntell. 20(2): logical status of semantic networks. See
163-210 Ref. 47, pp. 3-50
2. Alchouron, C., Gardenfors, P., Makin- 16. Brachman, R. 1983. What is-a is and
son, D. 1985. On the logic of theory isn’t: an analysis of taxonomiclinks in
change: partial meet contraction and semantic networks. IEEE Comput.
revision functions. J. Symbol. Logic 16(10):30-36
50(2):510-30 17. Brachman, R. 1985. I lied about the
3. Allen, J. 1983. Maintaining knowledge trees (or, defaults and definitions in
about temporal intervals. Commun. knowledge representation). A1 Mag.
ACM26(11):832-43 6(3):80-93
4. Anderson, A., Belnap, N. 1975. Entail- 18. Brachman,R., Fikes, R., Levesque, H.
ment: The Logic of Relevance and 1983. KRYPTON: a functional
Necessity. Princeton, NJ: Princeton Uni- approach to knowledgerepresentation.
versity Press IEEE Comput. 16(10):67-73
5. Attardi, G., Simi, M. 1981. Consistency 19. Braehman,R., Gilbert, V., Levesque,
and completeness of OMEGA, a logic H. 1985. Anessential hybrid reasoning
for knowledgerepresentation. In Proc. system: knowledge and symbol level
Int. Jt. Conf. Artif. Intell., Vancouver, accounts of KRYPTON. In Proc. Int. Jt.
BC, pp. 504-10 Conf. Artif. Intell., Los Angeles, Calif.
6. Ban’, A., Davidson, J. 1981. Represen- 20. Brachman, R., Levesque, H. 1982.
tation of knowledge. In The Handbook Competence in knowledge representa-
ofArtificiallntelligence, ed. A. Ban., E. tion. In Proc. Natl. Conf. Am. Assoc.
Feigenbaum, pp. 141-222. Los Altos, Artif. lntell., Pittsburgh, Pa., pp. 189-
Calif: W. Kaufmann 92
7. Barwise, J., Perry, J. 1983. Situations 21. Brachman, R., Levesque, H. 1984. The
and Attitudes. Cambridge, Mass: Brad- tractability of subsumption in frame-
ford Books, MIT Press based description languages. In Proc.
8. Belnap, N. 1977. A useful four-valued Natl. Conf. Am. Assoc. Artif. lntell.,
logic. In Modern Uses of Multiple- Austin, Texas, pp. 34-37
Valued Logic, ed. J. Dunn, G. Epstein, 22. Brachman, R., Levesque, H., eds.
pp. 8-37. The Netherlands: Reidel 1985. Readings in KnowledgeRepresen-
9. Bobrow, D. 1980. Special Volume on tation. Los Altos, Calif: MorganKauf-
Non-MonotonicReasoning. Artif. lntell. mann
13(1, 2) 23. Brachman, R., Schmolze, J. 1985. An
10. Bobrow, D. 1984. Special Volume on overview of the KL-ONEknowledge
Qualitative Reasoning about Physical representation system. Cognitive Sci.
Systems. Artif. Intell. 24(1-3) 9(2):171-216
10a. Bobrow, D., Collins, A., eds. 1975. 24. Deleted in proof
Representation and Understanding: 25. Bundy, A., Byrd, L., Mellish, C. 1985.
Studies in Cognitive Science. NewYork: Special-purpose, but domain-indepen-
Academic dent, inference mechanisms. In Pro-
11. Bobrow, D., Winograd, T. 1977. An gress in Artificial Intelligence, ed. L.
overview of KRL, a knowledge repre- Steels, J. Campbell, pp. 93-111. Lon-
sentation language. Cognitive Sci. don: Ellis Horwood
1(1):3-46 26. Charniak, E. 1981. A commonrepresen-
12. Bobrow, D., Winograd, T. 1979 KRL: tation for problem-solvingand language-
another perspective. Cognitive Sci. comprehensioninformation. Artif. In-
3(1):29-42 tell. 16(3):225-55
13. Bobrow, D., Winograd, T., the KRL 27. Cheruiak, C. 1984. Computational com-
Research Group. 1977. Experience with plexity and the universal acceptance of
KRL-0:one cycle of a knowledgerepre- logic. J. Philos., 81(12):739-58
sentation language. In Proc. Int. Jt. 28. Clark, K. 1978. Negationas failure. See
Conf. Artif. lntell. Cambridge,Mass., Ref. 51a, pp. 293-322
pp. 213-22 29. Clocksin, W., Mellish, C. 1981. Pro-
14. Brachman,R. 1978. A structural para- gramming in PROLOG. New York:
digm for representing knowledge. BBN Springer-Verlag
Rep. 3605, Bolt Beranek & Newman, 30. Collins, A., Loftus, E. 1975. A
Cambridge, Mass. spreading-activation theory of semantic
Annual Reviews
www.annualreviews.org/aronline

284 LEVESQUE

processing. Psychol. Rev. 82(6):407- semantic network. In Proc. Int. Jr.

28 Conf. Artif. Intell., Vancouver,BC,pp.
31. Collins, A., Warnock,E., Nelleke, A., 257-63
Miller, M. 1975. Reasoning from in- 46. Fikes, R., Nilsson, N. 1971. STRIPS:a
complete knowledge. See Ref. 10a, pp. new approach to the application of
383-416 theorem proving to problem solving.
32. Creary, L. 1979. Propositional attitudes: Artif. Intell. 2(3-4):189-208
Fregean representation and simulative 47. Fil~dler, N., ed. 1979. Associative Net-
reasoning. In Proc. Int. Jr. Conf. Artif. works: Representation and Use of
lntell., Tokyo, pp. 176-81 Knowledge by Computers. New York:
33. de Kleer, J. 1986. Anassumption-based Academic
TMS.Artif. lntell. 28(2):127-62 48. Fodor, J. 1983. The Modularity of Mind.
34. de Kleer, J., Doyle, J., Steele, G., Suss- Cambridge, Mass: Bradford Books,
man, G., 1977. AMORD: explicit con- MIT Press
trol of reasoning. In Symp.Artif. Intell. 49. Frisch, A. 1985. Using model theory to
Program. Lang., Rochester, NY, pp. specify AI programs. In Proc. Int. Jt.
116-25 Conf. Artif. lntell., Los Angeles, pp.
35. Dennett, D. 1986. Cognitive wheels: the 148-54
frame problemof artificial intelligence. 50. Frisch, A., Allen, J. 1982. Knowledge
In Minds, Machines, and Evolution, ed. representation and retrieval for natural
C. Hookway. Cambridge, England: language processing. Tech. Rep. TR
Cambridge Univ. Press 104, Dep. Comput. Sci., Univ. Roches-
36. Doyle, J. 1982. A glimpse of truth- ter, NY
maintenance.In Artificial Intelligence: 51. Funt, B. 1980. Problem-solvingwith di-
An MITPerspective, ed. P. Winston, R. agrammaticrepresentations. Artif. In-
Brown, pp. 119-35. Cambridge, Mass: tell. 13(3):201-30
MIT Press 51a. Gallaire, H., Minker, J., eds. 1978.
37. Doyle, J. 1983. The ins and outs of Logic and Databases. New York:
reason maintenance. In eroc. Int. Jt. Plenum
Conf. Artif. lntell. Karlsruhe, FRG,pp. 52. Garey, M., Johnson, D. 1979. Comput-
349-51 ers and Intractability: A Guide to the
38. Doyle, J., London,P. 1980. A selected Theory of NP-Completeness. San Fran-
descriptor-based bibliography to the cisco: W. H. Freeman
literature on belief revision. SIGART 53. Garvey, T., Lowrance,J., Fischler, M.
Newsl. 71:7-23 1981. An inference technique for in-
39. Dreyfus, H. 1981. From micro-worlds tegrating knowledge from disparate
to knowledgerepresentation: AI at an sources. In Proc. Int. Jt. Conf. Artif.
impasse. In MindDesign, ed. J. Hauge- lntell., Vancouver, BC, pp. 319-25
land, pp. 161-204. Cambridge, Mass: 54. Genesereth, M. 1983. An overview of
MIT Press meta-level architecture. In Proc. Natl.
40. Eberle, R. 1974. A logic of believing, Conf. Am. Assoc. Artif. Intell., Wash-
knowing and inferring. Synthese 26: ington, DC, pp. 119-24
356-82 55. Ginsberg, M. 1984. Non-monotonicrea-
41. Etherington, D., Mercer, R., Reiter, R. soning using Dempster’srule. In Proc.
1984. On the adequacy of predicate Natl. Conf. Am. Assoc. Artif. lntell.,
circumscription for closed-worldreason- Austin, Texas, pp. 126-29
ing. In The Non-Monotonic Reasoning 56. Halpern, J., Moses, Y. 1984. Towardsa
Workshop, New Paltz, NY, pp. 70-81 theory of knowledgeand ignorance: pre-
42. Etherington, D., Reiter, R. 1983. On liminary report. In The Non-Monotonic
inheritance hierarchies with exceptions. Reasoning Workshop, New Paltz, NY,
In Proc. Natl. Conf. Am. Assoc. Artif. pp. 125-43
lntell., Washington DC, pp. 104-8 57. Hayes, P. 1974. Some problems and
43. Fagin, R., Halpern, J. 1985. Belief, non-problems in representation theory.
awareness and limited reasoning. In In A1SB Summer Conf. Univ. Sussex,
Proc. Int. Jt. Conf. Artif. Intell., Los pp. 63-79
Angeles, Calif., pp. 491-501 58. Hayes, P. 1977. In defence of logic. In
44. Fahlman, S. 1979. NETL:A System for Proc. Int. Jt. Conf. Artif. lntell., Cam-
Representing and Using Real-WorM bridge, Mass., pp. 559-65
Knowledge. Cambridge, Mass: MIT 59. Hayes, P. 1979. The logic of frames. In
Press Frame Conceptions and Text Un-
45. Fahlman, S., Touretzky, D., van Rog- derstanding, ed.- D. Metzing, pp. 46-
gen, W. 1981. Cancellation in a parallel 61. Berlin: de Gruyter
Annual Reviews
www.annualreviews.org/aronline

KNOWLEDGE REPRESENTATION AND REASONING 285

60. Hayes, P. 1985. Naive physics I: ontolo- of incomplete knowledge bases. PhD
gy for liquids. See Hobbs & Moore thesis, Dep. Comput.Sci., Univ. Tor-
1985, pp. 71-107 onto, Ontario
61. Hayes, P. 1985. The second naive phys- 78. Levesque, H. 1983. The logic of in-
ics manifesto. See Hobbs & Moore complete knowledge bases. See Ref.
1985, pp. 1-36 105, pp. 165-86
62. Hewitt, C. 1969. PLANNER:a lan- 79. Levesque, H. 1984. Foundations of a
guagefor proving theorems in robots. In functional approach to knowledgerepre-
Proc. Int. Jt. Conf. Artif. lntell., Wash- sentation. Artif. lntell. 23(2):155-212
ington, DC, pp. 295-301 80. Levesque, H. 1984. A fundamental
63. Hewitt, C. 1972. Description and tradeoff in knowledgerepresentation and
theoretical analysis (using schemata) reasoning. In Proc. Bienn. Conf. Can.
PLANNER,a language for proving Soc. Comput. Stud. lntell., London,
theorems and manipulating models in a Ontario, pp. 141-52
robot. Tech. Rep. TR-258, AI Lab., 81. Levesque, H. 1984. A logic of implicit
MIT, Cambridge, Mass. and explicit belief. In Proc. Natl. Conf.
64. Hintikka, J. 1962. Knowledgeand Be- Am.Assoc. Artif. Intell., Austin, Texas,.
lief: An Introduction to the Logic of the pp. 198-202
TwoNotions. Ithaca, NY:Cornell Univ, 82. Levesque, H. 1986. Making believers
Press out of computers.Artif. lntell. In press
65. Hintikka, J. 1975. Impossible possible 83. Lewis, H. 1978. Complexityof solvable
worlds vindicated. J. Philos. 4:475-84 cases of the decision problem for the
66. Hobbs, J., Moore, R., eds. 1985. For- predicate calculus. In Proc. 19th IEEE
mal Theories of the Commonsense Syrup. Found. Comput. Sci,, pp. 35-47
World. Norwood, NJ: Ablex 84. Lipschitz, V. 1985. Closed-world data-
67. Hughes, G., Cresswell, M. 1968. An bases and circumscription.Artif. lntell.,
Introduction to Modal Logic. London: 27(2):229-236
Mcthuen 85. Loganantharaj, R. 1985. Theoretical
68. Israel, D. 1983. The role of logic in and implementationalaspects of parallel
knowledge representation. IEEE Corn- link resolution in connection graphs.
put. 16(10):37--42 PhDthesis. Dep. Comput. Sci., Colo.
69. Konolige, K. 1982. Circumscriptive State Univ., Fort Collins
ignorance. In Proc. Natl. Conf. Am. 86. Loveland, D. 1978. Automated Theorem
Assoc. Artif. lntell., Pittsburgh, Pa., Proving: A Logical Basis. NewYork:
pp. 202-4 North-Holland
70. Konolige, K. 1983. A deductive model 87. Martins, J., Shapiro, S. 1983. Reason-
of belief. In Proc. Int. Jt. Conf. Artif. ing in multiple belief spaces. In Proc.
lntell., Karlsruhe, FRG,pp. 377-81 Int. Jt. Conf. Artif. lntell., Karlsruhe,
71. Konolige, K. 1984. A deduction model FRG, pp. 370-73
of belief and its logics. PhDthesis. Dep. 88. McAllester, D. 1980. The Use of Equal-
Comput. Sci., Stanford Univ., Palo ity in Deduction and KnowledgeRepre-
Alto, Calif. sentation. MSthesis. AI Lab., Mass.
72. Konolige, K. 1985. A computational Inst, Technol., Cambridge
theory of belief introspection. In Proc. 89. McCarthy, J. 1968. Programs with com-
Int. Jt. Conf. Artif. lntell., Los Angeles, mon sense. See Ref. 98a, pp. 403-18
pp. 502-8 90. McCarthy, J. 1977. Epistemological
73. Kowaiski, R. 1974. Predicate logic as a problemsin artificial intelligence. In
programming language. In IFIP Con- Proc. Int. Jt. Conf. Artif. Intell,, Cam-
gress, Stockholm, pp. 569-74 bridge, Mass., pp. 1038-44
74. Kowalski, R. 1979. Logic for Problem 91. McCarthy,J. 1979. First order theories
Solving. Amsterdam!Elsevier North- of individual concepts and propositions.
Holland In MachineIntelligence, ed. J. Hayes,
75. Kuipers, B. 1979. Onrepresenting com- D. Michie, L. Mikulich, 9:129-47.
monsense knowledge. See Ref. 47, pp. Chichester, England: Ellis Horwood
393-408 92. McCarthy, J. 1980. Circumscription--a
76. Lakemeyer, G. 1986. Steps towards a form of non-monotonicreasoning. Artif.
first order logic of implicit and explicit Intell. 13(1,2):27-39
belief. In Theoreti~al Aspects of Reason- 93. McCarthy, J. 1984. Applications of
ing about Knowledge: Proc. 1986 circumscription to formalizing com-
Conf., ed. J. Halpem, pp. 325-40. Los monsense knowledge. In The Non-
Altos, Calif: Morgan Kanfmann Monotonic Reasoning Workshop, New
77. Levesque, H. 1981. A formal treatment Paltz, NY, pp. 295-324
Annual Reviews
www.annualreviews.org/aronline
286 LEVESQUE

94. McCarthy, J., Hayes, P. 1969. Some 108. Norman, D., Bobrow,D. 1975. On data
philosophical problems from the stand- limited and resource limited processing.
point of artificial intelligence. In Ma- Cognitive Psychol. 7:44-64
chine Intelligence, ed. B. Meltzer, D. 109. Norman,D., Rumelhart, D., eds. 1975.
Michie, 4:463-502. Edinburgh: Edin- Explorations in Cognition. San Francis-
burgh Univ. Press co: W. H. Freeman
95. McDermott,D. 1982. A temporal logic 110. Patel-Schneider, P. 1985. A decidable
for reasoning about processes and plans. first-order logic for knowledgerepresen-
Cognitive Sci. 6(2):101-55 tation. In Proc. Int. Jr. Conf. Artif. In-
96. McDermott, D., Doyle, J. 1980. Non- tell., Los Angeles, pp. 455-58
monotoniclogic I. Artif. lntell. 13(1,2): Ill. Pentlandr A., Fischler, M. 1983. A
41-72 more rational view of logic. A/ Mag.
97. Mendelson, E. 1964. Introduction to 4(4):15-18
Mathematical Logic. New York: Van 112. Quillian, M. 1967. Wordconcepts: a
Nostrand Reinhold theory and simulation of some basic
98. Minker, J., Perlis, D. 1984. Protected semantic capabilities. Behav. Sci. 12:
circumscription. In Workshopon Non- 410-30
Monotonic Reasoning, New Paltz, NY, 113. Quillian, M. 1968. Semantic memory.
pp. 337-43 See Ref. 98a, pp. 227-70
98a. Minsky, M., ed. 1968. Semantic In- 114. Rantala, V. 1982. Impossible world
formation Processing. Cambridge, semantics and logical omniscience. Acta
Mass: MIT Press Philos. Fenni. 35:106-15
99. Minsky, M. 1981. A framework for 115. Raphael, B. 1971. The frame problem in
representing knowledge. In Mind De- problem solving systems. In Artificial
sign, ed. J. Haugeland, pp. 95-128. Intelligence and Heuristic Program-
Cambridge, Mass: MIT Press ming. NewYork: American Elsevier
100. Moore, R. 1977. Reasoning about 116. Reiter, R. 1978. Onclosed world data-
knowledgeand action. In Proc. Int. Jt. bases. See Ref. 51a, pp. 55-76
Conf. Artif. lntell., Cambridge,Mass., 117. Reiter, R. 1978. On reasoning by de-
pp. 223-27 fault. In Proc. Conf. Theor. Issues Nat.
101. Moore, R. 1980. Reasoning about Lang. Process, Univ. Ill. Urbana-
knowledgeand action. Tech. Note 191, Champaign
Artifi. lntell. Cent., SRI Int., Menlo 118. Reiter, R. 1980. A logic for default
Park, Calif. reasoning. Artif. Intell. 13(1,2):81-
102. Moore, R. 1982. The role of logic in 132
knowledge representation and com- 119. Reiter, R. 1982. Circumscription im-
monsense reasoning. In Proc. Natl. plies predicate completion (sometimes).
Conf. Am. Assoc. Artif. Intell., Pitts- In Proc. Natl. Conf. Am. Assoc. Artif.
burgh, Pa., pp. 428-33 Intell., Pittsburgh, Pa., pp. 418-20
103. Moore, R. 1983. semantical con- 120. Rich, C. 1980. Knowledgerepresenta-
siderations on nonmonotoniclogic. In tion languages and predicate calculus:
Proc. Int. Jt. Conf. Artif. Intell., Karl- howto have your cake and eat it too. In
sruhe, FRG, pp. 272-79 Proc. Natl. Conf. Am. Assoc. Artif. In-
104. Moore, R., Hendrix, G. 1979. Com- tell., Pittsburgh, Pa., pp. 193-96
putational models of belief and the 121. Rieger, C. 1976. An organization of
semantics of belief sentences. Tech. knowledgefor problem solving and lan-
Note 187, Artifi. lntell. Cent., SRI Int., guage comprehension. Artif . Intell.
Menlo Park, Calif. 7(2):89-127
105. Mylopoulos, J., Levesque, H. 1983. An 122. Rosch, E., Mervis, C. 1975. Family re-
overview of knowledgerepresentation. semblances:studies in the internal struc-
In On Conceptual Modelling: Per- ture of categories. Cognitive Psychol.
spectives from Artificial Intelligence, 7:573-605
Databases, and Programming Lan- 123. Rosenschein, S. 1986. Formal theories
guages, ed. M. Brodie, J..Mylopoulos, of knowledge in AI and robotics. New
J. Schmidt, pp. 3-17. New York: Generation Comput. 3(4): In press
Springer-Verlag 124. Schank, R. 1973. Identification of con-
106. Nelson, G., Oppen, D. 1979. Sim- ceptualizations underlying natural lan-
plification by cooperating decision pro- guage. In Computer Models of Thought
cedures. ACMTrans. Program. Lang. and Language, ed. R. Schank, K. Col-
Sys. 1(2):245-57 by, pp. 187-247. San Francisco: W. H.
107. Nilsson, N. 1980. Principles of Artifi- Freeman
cial Intelligence. Palo Alto, Calif: Tioga 125. Schank, R. 1975. Conceptual Informa-
Annual Reviews
www.annualreviews.org/aronline

KNOWLEDGE REPRESENTATION AND REASONING 287

tion Processing. Amsterdam: North- Assoc. Artif. Intell., Pittsburgh, Pa.,

Holland pp. 197-201
126. Schubert, L. 1979. Problemswith parts. 135. Waterman, D., Hayes-Roth, F., ed.
In Proc. Int. Jt. Conf. Artif. lntell., 1978. Pattern-Directed Inference Sys-
Tokyo, pp. 778-84 tems. NewYork: Academic
127. Schubert, L., Papalaskaris, M., 136. Winker, S. 1982. Generation and veri-
Taugher, J. 1983. Determining type, fication of finite models and counter-
part, color, and time relationships. IEEE examples using an automated theorem
Comput. 16(10):53-60 prover answeringtwo open questions. J.
128. Simmons, R. 1973. Semantic networks: ACM29(2):273-84
their computation and use for under- 137. Winograd, T. 1972. Understanding Nat-
standing English sentences. See Ref. ural Language. NewYork: Academic
124, pp. 63-113 138. Winograd, T. 1975. Frame representa-
129. Sloman, A. 1971. Interactions between ~ jtions and the declarative/proceduralcon-
philosophyand artificial intelligence: the troversy. See Ref. 10a, pp. 185-210
role of intuition and non-logical reason- 139. Winograd, T. 1980. Extended inference
ing in intelligence. Artif. Intell. 2:209- modes in reasoning by computer sys-
25 tems. Artif. Intell, 13(1-2):5-26
130. Sloman, A. 1975. Afterthoughts on an- 140. Winston, P. 1975. Learning structural
alogical representation. In Proc. Conf. descriptions from examples. In The Psy-
Theoretical Issues in Natural Language chology of Computer Vision, ed. P.
Processing, Cambridge, Mass, pp. 164- Winston, pp. 157-209. New York:
68 McGraw-Hill
131. Smith, B. 1982. Reflection and seman- 141. Woods, W. 1975. What’s in a link:
tics in a procedural language.Tech. Rep. foundations for semantic networks. See
MIT/LCS/TR-272,Mass. Inst. Technol., Ref. 10a, pp. 35-82
Cambridge 142. Wos, L., Pereira, F., Hong, R., Boyer,
131a. 1980. Special Issue on Knowledge R., Moore,J., et al. 1985. Anoverview
Representation. SIGARTNewsl., Vol. of automated reasoning and related
70 fields. J. Autom. Reason. 1(1):5-48
132. Touretzky, D. 1984. Implicit ordering of 143. Wos, L., Winker, S., Smith, B., Ver-
defaults in inheritance systems. In Proc. off, R., Henschen, L. 1984. A new use
Natl. Conf. Am. Assoc. Artif. lntell., of an automated reasoning :assistant:
Austin, Texas, pp. 322-25 open questions in equivalential calculus
133. Touretzky, D. 1984. The mathematics of and the study of infinite domains.Artif.
inheritance systems. PhDthesis. Dep. lntell. 22(3):303-56
Comput. Sci., Carnegie-Mellon Univ., 144. Zadeh, L. 1983. Commonsenseknowl-
Pittsburgh, Pa. edge representation based on fuzzy log-
134. Vilain, M. 1982. Asystem for reasoning ic. IEEE Comput. 16(10):61-66
about time. In Proc. Natl. Conf. Am.