Abstract
We formulate the design of discrete-state stochastic control systems as optimizing a performance objective specified in user-oriented terms, i.e. terms that need not be perceivable by the controller or agent being designed. This addresses a user acceptance issue: while agent designs (control algorithms) are limited to distinctions about state supported by artificial perception systems, end users want to evaluate performance using terms such as safety, opportunity, and throughput. We elucidate a feedback from evaluation to agent design via a sensitivity analysis, obtaining the gradient of a time-averaging objective function w.r.t. state transitions influenced by the agent. This gradient leads to a methodology for iteratively improving a system's performance, as perceived by others.
This work was supported in part by NASA contract 9-19040.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
J. Bryson. Agent architecture as object oriented design. In this volume.
D. Draper, S. Hanks & D. Weld. Probabilistic planning with information gathering and contingent execution. Proc Int'l Conf on AI Planning Systems 1994, 31–36.
L. Kaelbling, M. Littman & A. Moore. Reinforcement learning: a survey. JAIR 4, 1996, 237–285.
N. Kushmeric, S. Hanks & D. Weld. An algorithm for probabilistic planning. AI Journal 76:1, 1995, 239–286.
D. Luenberger. Introduction to Dynamic Systems — Theory, Models and Applications, chapter 7. Wiley & Sons, New York, 1979.
R.A. McCallum. Hidden state and reinforcement learning with instance-based state identification. IEEE Trans SMC (B) 26:3, 1996, 464–473.
S. Rosenschein & L. Kaelbling. The synthesis of digital machines with provable epistemic properties. Theoretical Aspects of Reasoning Conf 1986, 83–98.
E. Verharen, F. Dignum & S. Bos. Implementation of a cooperative agent architecture based on the language-action perspective. In this volume.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Schoppers, M., Shapiro, D. (1998). Designing embedded agents to optimize end-user objectives. In: Singh, M.P., Rao, A., Wooldridge, M.J. (eds) Intelligent Agents IV Agent Theories, Architectures, and Languages. ATAL 1997. Lecture Notes in Computer Science, vol 1365. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0026746
Download citation
DOI: https://doi.org/10.1007/BFb0026746
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64162-9
Online ISBN: 978-3-540-69696-4
eBook Packages: Springer Book Archive