Ruzena Bajcsy and Craig Reynolds

Computer Science: The

Science of and about Information
and Computation
New computing paradigms present serious challenges
for system architecture.
nformation is unique in that The History of ods, the first of which they call the

I it is simultaneously contex-
tual and arbitrary. Contextual
because it never exists in a vac-
Information and
Information Science
In their book Information Ages:
Classical Age. While physicists use
the term “information” to denote a
measure of order in physical sys-
uum, but always represents some Literacy, Numeracy and the Com- tems, and as such refer to an entity
computational process or func- puter Revolution, Michael Hobart dating back to the very beginnings
tion; arbitrary because it has and Zachary Schiffman com- of the universe, the Information
representational power only by pellingly relate the evolution of Age picks up the narrative starting
virtue of having some previously information throughout history. with prehistoric human language
agreed-upon meaningful con- The main message gleaned is and the oral tradition.
tent. We suggest the latter prop- simply this: As our knowledge We refer to Hobart and Schiff-
erty has given us a powerful tool and technology have grown in man for the details, but here we
for expanding and exploring the complexity and sophistication, will briefly explain the progression
space of scientific possibility, information—which initially rep- to alphabetic literacy. First,
and has made possible exciting resented the immediate flux and emblems or pictographs were used
new computing paradigms such flow of daily life and was inti- to stand for physical objects.
as optical, biomolecular, and mately connected to the medium Cuneiform, the earliest true form
quantum computing. of conveyance—has gradually of writing, developed as the
These new paradigms, how- grown more and more abstract in increasingly sophisticated
ever, present serious challenges such a way that we now find our- Mesopotamian civilizations began
for new system architectures. To selves in an age when informa- to break free from the constraints
this end, we further suggest that tion is represented by arbitrarily of accounting and to model writ-
taking into account the contex- assigned symbols conveyed by ing upon their language rather
tual nature of information will whatever means human ingenu- than the physical objects of every-
lead to better, more useful, and ity and nature provides. day life. During the Classical Age
PAUL WATSON

efficient computing systems. We More specifically, Hobart and the alphabet was developed, repre-
call this approach “information Schiffman divide the history of senting in the abstract the basic
engineering.” information into three distinct peri- units of speech, such as vowels and

94 March 2002/Vol. 45, No. 3 COMMUNICATIONS OF THE ACM

consonants. Here we see that, we still manipulate mathematical combined, modified, and trans-
through the alphabet, writing symbols by fixed rules, but do so formed. We are thus bounded
became a means of encoding any with the knowledge that the sym- only by our imaginations.
sort of information in piecemeal bols we use are in some deep sense
fashion; that is to say, writing was arbitrary, deriving their meaning The Analog/Digital
no longer restricted to a 1:1 refer- only through some previously Distinction
ence between language and con- agreed-upon referents. Given the preceding discussion,
crete experience. How, then, is the Contempo- it is a trivial observation that as
Around 1450 A.D., Johannes rary Age different in some mean- technology has progressed so has
Gutenberg invented the printing ingful way from the Modern Age? information processing. That is
press and the Modern Age began. Precisely in the power and oppor- to say, technological advances are
Descartes described it as “the ana- tunities for play enabled by the utilized as they become available
lytical vision of knowledge” (in combination of our open knowl- for better, faster, and more effi-
contrast to Aristotle’s classifying edge system and the modern cient computational processes.
vision of knowledge that was typi- equivalent of the printing press— From cuneiform on clay, to the
cal of his age), wherein physical the computer. The power of the printed word on paper, to digital
phenomena were ultimately computer, coupling as it does logi- bits in electronic circuits, the
reduced to mathematical equa- cal operations with electronic cir- transfer and processing of infor-
tions. The now-familiar litany of cuitry, is well known. However, mation is continually improving.
giants in the field of mathematics the computer-mediated play over Today, one has three general
characterizes this era: Descartes’ arbitrary symbols that typifies our choices when it comes to comput-
analytic geometry; the calculus of time is less clear and may be best ing and information representa-
Newton and Leibnitz; Riemann’s explained by example. tion—namely analog, digital, and
non-Euclidean geometry; and A case in point is a virtual real- hybrid systems. An analog com-
Hamilton’s dynamics. In short, ity application developed by a puter represents information as a
mathematics was a new and pow- team from the University of North continuous function of some para-
erful technology that replaced the Carolina-Chapel Hill called the meter, and the processing or com-
written word as the most efficient nanoManipulator (nM). The nM putation is performed
means of dealing with increasingly provides a VR interface to a scan- indiscriminately on the whole
abstract information. ning-probe microscope capable of function. A digital computer, on
Sadly (or happily, depending on imaging and manipulating materi- the other hand, segments the same
one’s point of view), the analytic als at the atomic level. informational content into discrete
vision of knowledge and the Information technology in the units, and the computational
agenda of scientific determinism Contemporary Age has expanded operations are selectively applied
were shown to be castles in the air. the space of possibility. It has on small portions of the data or a
Quantum mechanics, Russell’s opened a new realm of inquiry single digit. The tradeoff is that
Class Paradox, and Gödel’s allowing scientists from all disci- while analog computation may be
Incompleteness Theorem defined plines to explore the impossible faster, digital computation is more
this. And so, we come to what and to investigate synthetic and/or flexible and provides access to
Hobart and Schiffman call the previously unimagined worlds. It local portions of the data. Hybrid
Contemporary Age, an age when is information, hence it can be systems, as the name implies, are a

COMMUNICATIONS OF THE ACM March 2002/Vol. 45, No. 3 95

Technical Opinion

IT has opened a new realm of inquiry allowing scientists from

all disciplines to explore the impossible and to investigate synthetic
and/or previously unimagined worlds.
combination of digital and analog typically hinges on the relative controller for certain tasks. Or,
subsystems interconnected via benefits of speed versus precision. when there is a need for decision
converters. The lesson to be Although digital computers can- making of any sort (from the sim-
learned is that the freedom not calculate over the set of real plest case, such as producing an
allowed by processing over sym- numbers as analog computers can, inhibitory signal, to a more com-
bols with arbitrarily assigned con- they can effectively compute num- plex decision, such as enabling an
tent also requires us to make bers with as much precision as is ensemble of neural circuits for
principled choices between these practically necessary. When this pattern recognition) the system
three models of computation. fact is combined with system con- produces a discrete decision.
The expression “analog process” siderations, such as the greater These discrete decisions can be
or “analog computation” means a likelihood of analog computers modeled by continuous functions,
continuous process or computa- being prone to noise or error, then so-called logistic functions.
tion, as opposed to some discrete digital computing seems to be the The question, then, is: How
computation. The actual signal can clear winner. However, there are does one optimize this process? To
take such forms as an electrical cur- certain applications and classes of answer this question, we must first
rent or voltage as a function of problems for which analog com- make explicit that we are only
time, the waveform of an optical puting has definite advantages, interested in processes or functions
signal, a mechanical such as in robotic control systems. known to be computable. We
force/mass/velocity, or perhaps the leave to others the question of
heat and pressure of concentrations Information Engineering whether the universe is itself com-
of chemical compounds again as a The issue is this: If one thinks of putable. This being the case, we
function of time. computation not in the abstract assume:
Hence, we see that in analog but as a physically realized process
computation there is a circular rela- that connects some input • Computational processes or
tionship between the physical phe- information to some output functions can be abstractly
nomena or process being modeled, information, then perhaps the expressed by some mathemati-
the mathematical model of the computational process and how cal or algorithmic formalism;
physical process, and the physical the information is to be internally and
substrate implementing the mathe- represented may be optimized— • Mathematical or algorithmic
matical model (for example, along continuous/discrete lines— formalisms describe a computa-
mechanical gears and shafts, electri- in a way that is most efficient for tional process or function at
cal resistors and conductors, chemi- a given system architecture, task, a specific level of granularity or
cal compounds, optical lenses, and and the nature of the input/out- resolution.
mirrors). We shall return to this put information. Biology has
important point shortly. plenty of examples to this effect. From these assumptions it fol-
The familiar debate between Consider a visual sensor. It may lows that the granularity of the
analog and digital enthusiasts be directly connected to a motor computational process or function

96 March 2002/Vol. 45, No. 3 COMMUNICATIONS OF THE ACM

is a determining factor in whether or may be continuous electro- a given, of course, derived by the
the formalism ought to be in the magnetic waves.) task-oriented nature of computa-
continuous or discrete domain, or tion. It directly and indirectly
both. Said differently, the represen- When we take into account the affects the choice of physical sub-
tation should be appropriate to the real-world nature of computation, strate and formalism. Simultane-
computational process or function two things should be immediately ously, the formalism and physical
being described. Put even more clear. First, the controllable behav- substrate must be optimally
simply, the choice of mathematical ior of the physical substrate is a matched. These choices are made
model must take into account the determining factor in whether the because computation is physically
task or problem to be solved at the formalism ought to be in the con- realized in hardware. We depict
given scale of resolution. tinuous or discrete domain, or this model in Figure 2. (Note
Considering the granularity of both. In other words, the represen- again, however, the causal rela-
the process or function being tation should be appropriate to the tionship between physical sub-
modeled is not a sufficient condi- underlying physical substrate. This strate and computational process
tion by itself, however, for deter- is trivially true. Second, and or function no longer exists.)
mining the appropriate nature of equally important, the converse What, then, are the practical
the information representation. also holds: the formalism being implications of this model? Briefly,
This is because computation is implemented, and by extension we are suggesting that the tradi-
always implemented in hardware, the computational process or func- tional computer science subdisci-
either organic or synthetic. Let us tion being described, are deter- plines of architecture, software,
elaborate by adding two additional mining factors in whether the and theory need to recognize the
assumptions: physical substrate (system architec- contextual nature of information.
ture) ought to be in the analog That is to say, computer scientists
• Computation is ultimately domain, digital domain, or both. should recognize that information
physical, in the sense that the Keeping in mind the preceding does not exist in a vacuum, but
mathematical or algorithmic discussion, in Figure 1 we see that instead always represents some
formalisms describing compu- the physical process being for- computational process or function
tational processes or functions mally modeled and the physical (real-world data or cognitive con-
are executed upon, or imple- substrate implementing the structs) whenever it flows into a
mented in, a physical substrate; model both play an important computer, is acted upon via some
and role when choosing the suitable formalism, and is then output
• The physical substrate, and the mathematical model, or more back into the real world. If this is
controlled behavior thereof, generally the optimum informa- so, then each of the arrows in Fig-
may exhibit different levels of tion representation scheme. How- ure 2 represents a question of engi-
granularity or resolution. That ever, the relationship between the neering, the answer to which must
is, a physical substrate exhibits three is no longer circular. Why? be balanced against the others via
continuous, discrete, or hybrid Because the computer-mediated some cost functions:
behavior depending upon play over arbitrary symbols that is
which behavior is of interest. a defining feature of the Contem- A. Would a continuous, dis-
(Two familiar examples are porary Information Age breaks crete, or hybrid formalism best rep-
light and electricity, which may the chain of analogous processes. resent the information and task at
take the form of discrete pho- The computational process or hand?
tons and electrons, respectively, function being modeled remains B. Would an analog, digital, or

COMMUNICATIONS OF THE ACM March 2002/Vol. 45, No. 3 97

Technical Opinion

approach that more frequently brain is not a general-purpose

Figure 1. The circular only asks questions C and D. computer (like a universal Turing
nature of rudimentary
analog computing. Now, some may protest that machine) but a highly evolved
with all this talk of continuous special-purpose machine that by
functions and analog and hybrid sheer dint of complexity is capable
system architectures, we have given of a great many tasks. Depending
Physical Mathematical short shrift to digital general-pur- on the task and type of informa-
Process Model pose computing. One may point tion being transmitted, networks
out that considerations of econ- of neurons automatically employ
omy dictate use of a uniform rep- discrete, continuous, and hybrid
Physical resentation at an elementary level computation whenever necessary.
Substrate and a physical substrate usable for And perhaps this is the ultimate
the largest number of tasks and lesson to be learned from the most
models, which implies universality powerful computational machine
Figure 2. A model of of computation. (Computer scien- in nature with the given task of
computation in
the Information Age. tists have focused on this almost survival: it seems that the brain is
exclusively.) And haven’t digital capable of automatic architecture
Computational systems served us quite well for the customization for certain well-
Process or
Function last 50 years? We have no argu- defined computational tasks.
B A
ment here. Indeed, our point is Perhaps someday it will be
D that for many cases the informa- commonplace for computers to
Physical Mathematical tion engineering approach will likewise automatically tune their
Substrate or Algorithmic suggest that digital general-purpose discrete and continuous represen-
Formalism
C computing using discrete units for tations depending on the context
information representation is in of the information and the task.
hybrid architecture be best suited fact the optimal solution. How- In the near term, however, as new
for the information and task at ever, it should be obvious that dig- revolutionary computing para-
hand? ital computers are not best suited digms emerge, it is vital at the
C. Would a continuous, dis- for all tasks or representations. In very least to recognize and account
crete, or hybrid formalism be best certain special-purpose cases, ana- for this increasingly salient, symbi-
suited for the system architecture? log and hybrid systems may be the otic relationship between the con-
D. Would an analog, digital, or best solutions with regard to textual representation of
hybrid architecture be best suited robustness and efficiency, but they information and computation.
for the formalism used? clearly aren’t as flexible or easy to Information engineering is a good
program. These are the tradeoffs first step in that direction. c
Possible cost functions that may one must make, the balance one
be used to guide one’s decisions must strike, when recognizing the Ruzena Bajcsy (bajcsy@eecs.berkeley.
edu) is the director of the Institute for
include reliability, robustness, pro- contextual nature of information. Research in IT in service to society (CITRIS)
grammability, and flexibility in From an information engineer- at UC Berkeley.
range of tasks, energy consump- ing point of view, biological sys- Craig Reynolds (craig.reynolds@cmich.
tion, precision, and speed. This is tems have evolved the optimal edu) is a research and program officer at Cen-
tral Michigan University, Mt. Pleasant, MI.
an information engineering balance of answers to the ques-
approach, as opposed to the more tions for their specific purposes.
limited system engineering For example, we suggest that the © 2002 ACM 0002-0782/02/0300 $5.00

98 March 2002/Vol. 45, No. 3 COMMUNICATIONS OF THE ACM