electronics

Article
Applications of Multi-Agent Systems in Unmanned Surface Vessels
Lada Males 1 , Dean Sumic 2, * and Marko Rosic 3

1 Faculty of Humanities and Social Sciences, University of Split, 21000 Split, Croatia
2 Faculty of Maritime Studies, University of Split, 21000 Split, Croatia
3 Faculty of Science, University of Split, 21000 Split, Croatia
* Correspondence: dean.sumic@pfst.hr

Abstract: The comprehensive and safe application of unmanned surface vessels is certainly one of
the biggest challenges currently facing maritime science. Such vessels can be implemented within
a wide range of autonomy levels that goes from remote-controlled vessels to fully autonomous
vessels in which intelligent vessel systems completely perform all necessary operations. One of the
ways to achieve autonomous vessel systems is to implement multi-agent systems that take over all
functions performed by the crew in classical manned crew vessels. A vessel is a complex system that
conceptually can be considered as a set of interconnected subsystems. Theoretically, the functions
of these subsystems could be performed using appropriate multi-agent systems. In this paper we
analyzed 24 relevant papers. A review of the current state of implementation of multi-agent systems
for performing the functions of unmanned surface vessels is presented.

Keywords: unmanned surface vessels; autonomous surface vessels; agent-based systems; multi-agent systems

1. Introduction
Citation: Males, L.; Sumic, D.; Rosic, Unmanned vessels are one of today’s trends in maritime affairs. We are witnessing
M. Applications of Multi-Agent numerous projects of unmanned vessels for activities such as civil, military, and research
Systems in Unmanned Surface purposes. The history of attempts to achieve unmanned vessels is longer than 130 years.
Vessels. Electronics 2022, 11, 3182. Nikola Tesla created the first model of a miniature remote-controlled ship [1,2]. He pre-
https://doi.org/10.3390/ sented this model at Madison Square Garden in New York at the exposition in 1898. As
electronics11193182
with all his inventions, Tesla was ahead of his time. This prototype was actually the first
Academic Editor: Fernando De unmanned vessel, but the development of such vessels should have waited some time.
la Prieta This time was needed to develop appropriate supporting technologies that would make the
idea of unmanned vessels interesting to maritime industry. Today, the maritime industry is
Received: 30 August 2022
preparing for a time when human presence on ships will no longer be required. The reasons
Accepted: 28 September 2022
for this trend can be divided into two groups. The first group includes financial reasons,
Published: 4 October 2022
which can be seen in reducing operating costs and increasing the carrying capacity of ships
Publisher’s Note: MDPI stays neutral due to the absence of human crews. The second group relates to security reasons. A large
with regard to jurisdictional claims in number of maritime incidents are caused by human factors. Therefore, it can be anticipated
published maps and institutional affil- that the reduction in the number of people on board, and eventually the complete absence
iations. of human crews on board, will reduce the number of marine accidents. However, it is also
necessary to consider which classes of security challenges this new approach to maritime
will bring. When designing unmanned surface vessels, it is necessary to harmonize the
designed solutions with the official documents regulating this area. In this case, these are
Copyright: © 2022 by the authors.
conventions and standards such as the International Regulations for Preventing Collisions
Licensee MDPI, Basel, Switzerland.
This article is an open access article
at Sea (COLREG) and the International Convention for the Safety of Life at Sea (SOLAS)
distributed under the terms and
imposed by the International Maritime Organization (IMO), as well as all other relevant
conditions of the Creative Commons
official documents. Failure to comply with these rules is a frequent cause of a ship collisions.
Attribution (CC BY) license (https:// Prior to the introduction of modern navigation systems, 56% of ship collisions were caused
creativecommons.org/licenses/by/ by the failure to comply with COLREG [3].
4.0/).

Electronics 2022, 11, 3182. https://doi.org/10.3390/electronics11193182 https://www.mdpi.com/journal/electronics

Electronics 2022, 11, 3182 2 of 14

There are several degrees of vessel autonomy. At the lowest level, only individual
processes are automatized, and the vessel needs a human crew. At the two next levels,
autonomy is enabled by remote control. The autonomy of the vessel increases as the level
of remote control of the vessel increases. The highest level of unmanned vessel includes
vessels with control system that can make and implement decisions fully autonomously. A
vessel is a complex system that consists of different subsystems. Each of these subsystems
should be made autonomous in order to develop a fully autonomous vessel. The question is
how to conceptualize such a complex system, how to implement it, and how to evaluate it.
The use of intelligent agents in addressing these challenges is an increasingly frequent topic
in papers published by relevant authors in this field. Intelligent agents are autonomous
entities that perceive the state of their environment, interpret perceptions, and act within
the environment in accordance with their goals. The elements of an intelligent agent that
allow it to perceive the environment are called agent sensors. On the other hand, effectors
are those elements that enable an intelligent agent to operate within the environment.
During the implementation of intelligent agents into existing vessels, it is possible to use
parts of the existing ship equipment as the agent’s sensors or effectors. In order to increase
efficiency, intelligent agents are organized as multi-agent systems, i.e., the systems in which
agents act cooperatively to accomplish common goals.
There are numerous possibilities for the use of intelligent agents in areas such as
transportation and logistics [4], design of critical systems [5], manufacturing [6], tourism [7],
or education [8]. The aim of this paper is the review of the current state of research
in the implementation of multi-agent systems in unmanned vessels, i.e., the review of
advancement in unmanned vessels functionalities using multi-agent systems. Some of
research described in this review uses a single agent solution. However, we included them
because any use of a single agent solution in a group of vessels must be considered as a
multi-agent system, i.e., the group has all multi-agent system properties. Moreover, two
additional questions will be considered. The first concerns the answer to the question of
whether any research includes mechanisms for interoperability between different systems
of unmanned surface vessels. The second question concerns an existence of metrics that
would establish a relevant comparison system of different multi-agent ship systems of the
same purpose.
The paper is organized as follows. Sections 2 and 3 give brief descriptions of unmanned
surface vessels and intelligent agents, respectively. Section 4 gives review of 24 papers that
use multi-agent systems to achieve any functionality of the unmanned surface vessels. In
Section 5 discussion is given.

2. Unmanned Surface Vessels

The increasing maritime traffic is demanding a global increase in the number of
people employed on ships of any kind. This particularly applies to experienced officers
on board. Technologically advanced modern ships emphasize this problem even more.
Nowadays in shipping, we have on one side the lack of experienced crews, and on the
other side the growth of freight transport demands. A lack of manpower is not the only
problem of maritime affairs. It is often the human factor that causes many unwanted
situations at sea. The problem with human crews on ships is that they are the biggest
causes of potentially devastating situations that can occur due to wrong decisions, violation
of defined procedures, fatigue, or due to lack of practice [9]. Due to the development
of appropriate technologies, one of the ways to solve the above-mentioned problems is
the development and implementation of unmanned vessels. Such vessels are not yet
commercially used. It can be expected that the technologies of unmanned vessels first
will be used for military purposes. So far, the military industry is usually the first that
has practical applications of new technologies. The same thing happens in unmanned
vessel technologies. For example, the U.S. Navy announced 6 years ago that it has ready
technology for small unmanned ships that are ready for interoperability with ships with
crews [10]. The first commercial unmanned ships are not expected before 2035, but by
11, x FOR PEER REVIEW 3 of 15

Electronics 2022, 11, 3182 3 of 14

technologies. For example, the U.S. Navy announced 6 years ago that it has ready tech-
nology for small unmanned ships that are ready for interoperability with ships with
crews [10]. The first commercial unmanned ships are not expected before 2035, but by
introducing
introducing them, maritime trafficthem, maritime
will enter a newtraffic will enter
era where a new era ships
conventional wherewithconventional ships with
crews and unmannedcrewsships and
will unmanned
sail within ships willwaters
the same sail within the same waters
simultaneously [11]. simultaneously [11].
The most frequently The most frequently
referenced definition referenced
of an unmanned definition of an
vessel unmanned
is the definition vessel is the definition
used by the International Maritime Organization (IMO). According to this definition, an to this definition, an
used by the International Maritime Organization (IMO). According
unmanned vessel is anyunmanned vessel
vessel that canisoperate
any vessel
up tothat can operate
a certain up to a certain
level without level without the direct
the direct
participation of people [12]. The same organization also proposed a classification of the a classification of the
participation of people [12]. The same organization also proposed
autonomy
autonomy of vessels that containofatvessels that degree
least some containofatautonomy.
least someThisdegree of autonomy.
classification is This classification
shown in Figure 1. Atisthe
shown
lowestin level
Figure 1. At
there arethe lowest
those levelthat
vessels there areatthose
have least vessels
some au- that have at least some
automated
tomated decision-making systemsdecision-making systems and
and the crew operates theirthe crew operates
functions. At thetheir
nextfunctions.
lev- At the next level,
there are remote-controlled manned vessels, followed
el, there are remote-controlled manned vessels, followed by the level with re- by the level with remote-controlled
unmanned
mote-controlled unmanned vessels.
vessels. Athighest
At the the highest
levellevel of autonomy
of autonomy are vessels
are vessels whosewhose systems are fully
capable of operating the
systems are fully capable of operating the vessel. vessel.

the highest degree of

fully autonomous vessel s autonomy

remote-control unmanned vessels

remote-control manned vessels

manned v essels, automated decision- the lowest degree of

making sy stems operat ed by t he crew autonomy

Figure 1. IMO classification of the

Figure autonomy
1. IMO of vessels.
classification of the autonomy of vessels.

There are other

There are other classifications of thisclassifications of this area.
area. For example, For example,
according according to the Lloyd’s Reg-
to the Lloyd’s
Register definition, anister definition, an
autonomous autonomous
high-level shiphigh-level
should haveship should have thetocapability
the capability au- to autonomously
tonomously cross thecross
ocean, theperforming
ocean, performing
assessment assessment of navigational
of navigational states, identification of navigation
states, identification
of navigation risk, andrisk, and decision
decision makingmaking in real-time
in real-time [13]. Regardless
[13]. Regardless of the of the classification used, it is possi-
classification
used, it is possible toble to classify:
classify: fully fully
manned manned vessels,
vessels, remotely
remotely operated
operated vessels,
vessels, and and
au- autonomous vessels.
tonomous vessels. The remotely controlled vessels are therefore a phase between fully manned and
autonomous
The remotely controlled vessels.
vessels Very high
are therefore a requirements
phase betweenwill continue
fully manned to and
be set to a crew that will
control a vessel remotely [14]. These requirements
autonomous vessels. Very high requirements will continue to be set to a crew that will are expected because the vessels will
control a vessel remotely [14]. These requirements are expected because the vessels will shall be subject to
still have to operate their safely navigate high seas. These vessels
requirements
still have to operate their affecting
safely navigate hightheir
seas.ability
Thesetovessels
sail, maneuver, locate,tomonitor,
shall be subject re- and operate the
propulsion system, as well as to any other subsystems
quirements affecting their ability to sail, maneuver, locate, monitor, and operate the of the vessel necessary for the
propulsion system, assafe
wellconduct
as to any ofother
the voyage. Whenofconsidering
subsystems safety issues
the vessel necessary for safe
for the unmanned vessels, some
new issues, such as cyber-attacks or systemic
conduct of the voyage. When considering safety issues for unmanned vessels, some new errors resulting from unforeseen system
issues, such as cyber-attacks or systemic errors resulting from unforeseen system be- unmanned vessels is
behaviors, must also be raised [15]. An additional challenge facing
haviors, must also be to operate
raised [15].within a highly challenge
An additional dynamic environment
facing unmanned wherevessels
vesselsis with
to a human crew can
also be found. This is important because human
operate within a highly dynamic environment where vessels with a human crew can also reactions can sometimes leave the zone of
expected actions and the management systems
be found. This is important because human reactions can sometimes leave the zone of of unmanned vessels need to be prepared
for such situations.
expected actions and the management systems of unmanned vessels need to be prepared
for such situations. 3. The Concept of an Agent and Multi-Agent Systems

3. The Concept of an Agent Theandconcept of an agent

Multi-Agent was in artificial intelligence for approximately three decades.
Systems
An agent theory describes what an agent is, what its properties and internal structure are,
The concept of an
and agent
howwas theyinare
artificial intelligence
connected. In orderfortoapproximately three
do this, an agent dec- uses a mathematical
theory
ades. An agent theoryformalism
describes for what an agent is, what its properties and internal struc-
representing an agent, its behavior, and reasoning. There is no unique
ture are, and how they are connected.
definition In order
of an agent. to dodefinitions
Several this, an agent
are theory usesThree
proposed. a mathe-
definitions are described
matical formalism forbelow,
representing
i.e., definitions given by Wooldridge [16], Ferber [17], and no
an agent, its behavior, and reasoning. There is Russel and Norvig [18].
unique definition of an agent. Several[16]
Wooldridge definitions
describesarean proposed.
agent as a Three definitions
computer system arethat is situated in some
environment, and that is capable of autonomous action in this environment in order to meet
its design objectives. This definition does not include intelligence. An agent is intelligent
if it is capable of flexible autonomous action, where flexible mean that beside autonomy,
Electronics 2022, 11, 3182 4 of 14

an agent also needs to possess the following characteristics: reactivity, proactiveness, and
social ability. Reactivity means that an agent can perceive its environment and has a real-
time response to the observed changes. Proactiveness is an ability of an agent to exhibit
goal-directed behavior that leads to the achievement of its designed objectives by taking
the initiative. Social ability is a characteristic of an agent to interact with other agents (and
possibly humans) in order to satisfy its designed objectives.
Ferber [17] defines an agent as a physical or virtual entity which is capable of acting
in an environment and can communicate with other agents. An agent is endowed with
autonomy, i.e., it is not driven by commands of the user or other agents but by a set
of tendencies, which can take a form of individual objectives or of satisfaction/survival
functions. It possesses own resources. An agent is able to perceive its environment to a
limited extent and has only a partial representation of this environment. An agent has skills
and can offer services and may be able to reproduce itself. Ferber represents an agent as a
‘living organism’ that behaves in such way to attain objectives on the basis of all available
elements, i.e., perceptions, representations, actions, communications, resources, and skills.
There are two concepts of agents according to the how the knowledge is repre-
sented [17]. The first types are cognitive agents. They have symbolic and explicit rep-
resentation of the world on the basis of which they can reason. They have a knowledge base
with data, know how to complete tasks, and how to communicate with other agents and
environment. If they have goals and explicit plans on how to achieve their goals, they are
intentional agents. A well-known type of cognitive agents are the belief–desire–intention
(BDI) agents [19–21]. They have mental attitudes: beliefs, desires, and intentions. Beliefs
represent the informational state of an agent about itself, other agents, and the environment.
Desires describe what the agent’s motivations and goals are. Intentions are a result of
deliberation. They lead to action. The strength of this type of agents is that they are inspired
by the human concept of knowledge and deliberation. This property makes them simple
and intuitive for understanding their internal structure. The second types are reactive
agents. Their representation of knowledge is situated at a sub-symbolic level; they have no
planning mechanism or explanation of goals. An internal representation is numerical, so
they use optimization methods for exploring parameter space. This type of agent originates
from the study of animal behavior [22,23]. They emulate the type of animal behavior that
is simple and governed by simple rules, e.g., an ant colony, a flock of birds, or fish. The
strength of this type of agent is not an individual agent, but a large number of agents with
their property of self-organization that leads to emergent behavior that might be described
as intelligent, i.e., intelligence of the group. Ferber [17] also divides agents according to
their modes of conduct on teleonomic and reflexes agents. Teleonomic agents are directed
towards explicit goals expressed within the agents. A reflex agent behavior is regulated by
perceptions obtained by environment.
This classification is simplified and divides agents in two completely different types.
In multi-agent systems, these types can be combined in different ways depending on the
problem domain, i.e., hybrid agent architecture can be developed or a hybrid multi-agent
system can be created.
Russel and Norvig [18] define an agent as anything that perceives its environment
through sensors and acts on that environment through actuators. An agent behavior is
described by the agent function that is implemented by an agent program. The agent
function maps percept sequence to an action. There exist many types of agent programs
design and they depend on the nature of the environment.
Russel and Norvig [18] describe four basic kinds of agents: simple reflex agents (di-
rectly respond to percepts), model-based reflex agents (have an internal state that depends
on the percept history, combines current percept with internal state, updates current state,
and acts according to that), goal-based agents (act to achieve their goals), and utility-based
agents (try to maximize their own utility function). All four kinds of agents can be the foun-
dation for learning agents because they can improve their performance through learning.
 Russel and Norvig [18] describe four basic kinds of agents: simple reflex agents (di-
rectly respond to percepts), model-based reflex agents (have an internal state that de-
pends on the percept history, combines current percept with internal state, updates cur-
rent state, and acts according to that), goal-based agents (act to achieve their goals), and
utility-based agents (try to maximize their own utility function). All four kinds of agents
Electronics 2022, 11, 3182 5 of 14
can be the foundation for learning agents because they can improve their performance
through learning.
As we saw from the definitions above, the concept of agent is mostly generic but
gives the opportunityAs webroadly
to be saw from the definitions
applied above, domains.
to many different the concept of agent isitmostly
Additionally, is generic but
givesagent
obvious that a single the opportunity
cannot make tosignificant
be broadlychange
appliedintoanmany different in
environment domains.
which itAdditionally, it is
obviouscannot
acts and consequently that a besingle agent cannot
interesting make significant
for implementation. change inagents
Therefore, an environment
are in which
it acts andsystems
organized in multi-agent consequently
in ordercannot be interesting
to resolve problem for in implementation.
different domains. Therefore,
To agents are
achieve commonorganized
goals agents in multi-agent
in multi-agentsystems
systemin must
orderinteract
to resolvewithproblem in different
each other. The domains. To
achieve common goals agents in multi-agent system
interaction may be simple or complex, as well as may be achieved directly or through themust interact with each other. The
environment. Sointeraction
far, numerous maymulti-agent
be simple orsystems
complex, areasdeveloped.
well as may Inbe achieved
this paper we directly
are or through the
interested in the environment.
use of multi-agent So far, numerous
systems multi-agent
in unmanned systems
surface are developed. In this paper we are
vessels.
interested in the use of multi-agent systems in unmanned surface vessels.
4. The Use of Intelligent Agents in Unmanned Surface Vessels
4. The Use of Intelligent Agents in Unmanned Surface Vessels
When considering multi-agent unmanned vessels, the multi-agent system ideally
When considering multi-agent unmanned vessels, the multi-agent system ideally
should be able to fully assume the role of a human crew member. Regardless of the ship’s
should be able to fully assume the role of a human crew member. Regardless of the
functionality, the multi-agent system needs to take over each of the following four func-
ship’s functionality, the multi-agent system needs to take over each of the following four
tional classes: (i) information acquisition; (ii) information analysis; (iii) decision and ac-
functional classes: (i) information acquisition; (ii) information analysis; (iii) decision and
tion selection, and (iv) action implementation [24]. Figure 2 shows the structure of an
action selection, and (iv) action implementation [24]. Figure 2 shows the structure of an
agent through these functional classes. Performance of information acquisition is the task
agent through these functional classes. Performance of information acquisition is the task
of agent sensors. Agent knowledge shall contain the knowledge necessary to perform
of agent sensors. Agent knowledge shall contain the knowledge necessary to perform
information analysis as well as decision and action selection. Finally, agent effectors
information analysis as well as decision and action selection. Finally, agent effectors
perform actions. When considering the operation of the multi-agent system on board, it is
perform actions. When considering the operation of the multi-agent system on board, it
necessary to conceptualize,
is necessary asto
parts of the multi-agent
conceptualize, as partssystem,
of theall those partssystem,
multi-agent of the ship’s
all those parts of the
equipment over which the agent may
ship’s equipment have
over control.
which Parts may
the agent of thehave
ship’s equipment
control. Parts that canship’s equipment
of the
acquire differentthat
states of its environment, e.g., smoke detector, echo sounder, etc., may
can acquire different states of its environment, e.g., smoke detector, echo sounder, etc.,
serve as agent sensors.
may serve Furthermore, all thoseFurthermore,
as agent sensors. parts that can allperform actions
those parts thatcan
canbe con- actions can be
perform
ceptualized as agent effectors.
conceptualized as agent effectors.

AGENT

information
analysis

information act ion

acquisition implementation
deci sion and
act ion selection

sensors agent s knowledge effectors

Figure 2. The functional

Figureclasses
2. The of the agentclasses
functional structure.
of the agent structure.

The following isThe following

a review is a review
of research papersof research
from the papers
scope of from
usethe scope of use of multi-agent
of multi-agent
systems to achieve the functionality of unmanned surface
systems to achieve the functionality of unmanned surface vessels. The presented papers vessels. The presented papers
were obtained
were obtained using using
search tools search
with tools with
keywords keywords
unmanned (orunmanned (or autonomous),
autonomous), vessel (or vessel (or ship),
and
ship), and agent (or agent (or multiagent,
multiagent, multi-agent). multi-agent).
The agents wereThe agents
first usedwere first used
in marine in marine
research for research
differentfor different simulation
simulation purposes. purposes. Simu-
Simulations enable testing of various items which, in a real situation, would bewould
lations enable testing of various items which, in a real situation, be too dangerous or
too dan-
too expensive to perform, and sometimes impossible to perform. In the case of maritime
research, simulations made it possible to test scenarios to raise the level of safety and
functionality of ship transport. With further development of simulation platforms, such
environments are also used for design and evaluation of entire systems, including systems
of unmanned vessels.
Liu, Sun, and Du [25], in their work, propose the model of intelligent agent for simulat-
ing ship collision avoidance. Within this model, they defined seven basic components that
such an intelligent agent should have. Each one of these basic components is specialized in
Electronics 2022, 11, 3182 6 of 14

performing one function. Cooperative activities of these components result in the following
functions: perception, memory, communication, emotion, action, learning, and thinking.
By using these functions, a ship conceptualized through an agent model is able to achieve
functionalities, such as: detecting changes in the environment, gathering information about
other vessels in the environment, judging the degree of danger of the current situation,
communicating with other participants within the environment, making knowledge-based
decisions, and acting to avoid collisions with other vessels or obstacles. With this approach,
the target ship assumes the characteristics of an intelligent agent because this ship acts
autonomously and adaptably.
Han, Zhang, Wang, Luo, Ran, and Xu [26] propose the multi-agent model for training
unmanned surface vessels. In their model, agents are designed as the physical controllers
for each unmanned surface vehicle. Within the model, an agent cooperation method is
defined. The authors designed the multi-agent-based learning system to train unmanned
surface vehicles. A system simulation was carried out to verify its effectiveness. This
simulation showed that the proposed model is feasible and can be transformed into a policy
for selecting the actions of a team of unmanned surface vehicles.
The focus of work of Xiao, Fu, Zhang, Agarwal, and Goh [27] is on planning, modelling,
and testing autonomous vessels. Their model can be clearly divided into the following
layers: (i) modelling analytics (sea way capacity evaluation, situational awareness, scenario
evaluation, risk evaluation, and high-level decision making); (ii) multi-agent system mod-
elling (dynamic multi-agent modelling, monitor runtime intermediate data, and generate
modelling output) and (iii) data and knowledge (ENC data, AIS and radar data, regulation,
spatial environment data, and other maritime transport data). The authors consider the pro-
cess of autonomous maritime transport as a chain consisting of path planning, monitoring
the situation, and making decisions according to the current state of the environment.
Autonomous navigation is the functionality most often referred to as the functionality
taken over by the agent-based systems in autonomous ships. Navigation is a complex
process that within itself possesses many requirements that need to be accomplished. These
requirements represent functionalities such as avoiding collisions, path planning, detecting
anomalies, or classifying objects of the vessel environment. The accomplishment of these
functionalities by agents can be achieved through learning based on data on previous
reactions of vessels in different situations. For the purpose of the above-mentioned training
of agent-based systems, deep learning algorithms are most commonly used. The need
to achieve autonomous navigation resulted in a greater number of publications dealing
with appropriate deep learning methods. Thus, within the fifteen-year period (2003–2018),
the number of publications discussing deep learning-based navigation methods increased
9 times. In the same period, the number of publications processing deep learning in ship
collision avoidance increased 5 times and the number of publications researching deep
learning for anomaly detection increased 15 times [28].
Perera [3] proposes a collision avoidance model within the framework for the au-
tonomous navigation of ships. In the base of this model is an agent system based on deep
learning systems. Deep learning systems would learn from the behaviors of the crews
of nowadays ships, for example, they would learn vessel management by observing the
actions of the shipyards. Similar methods are already used in practice. This concerns in
particular the resolution of autonomy problems in related areas, such as the autonomy
of road vehicles and drones. Nowadays, these methods can be adapted to unmanned
surface vessels.
By analyzing COLREG rules, Perera finds potential regulatory problems related to the
COLREG application to autonomous vessels. This author also suggests possible solutions
to the discovered deficiencies of COLREG. A structured technology framework proposed
by Perera would possess decision support layer supported by Internet of Things (IoT)
elements that would provide the system with relevant data about the environment. He
also stated that procedures for checking such autonomous systems should be established.
Of course, all these checks must be coordinated with the appropriate public authorities.
Electronics 2022, 11, 3182 7 of 14

Perera, in collaboration with Murray, suggests the method for maritime situation control
based on the prediction of ship behavior via clustering data extracted from ship trajectory
in an observed geographical region [29,30]. The method uses machine learning techniques
to extrapolate those segments of the trajectory that are relevant for the current calculation.
Azimi, Salokannel, Lafond, Lilius, Salokorpi, and Porres [31], in their navigational
model, recommend a combination of reinforcement learning and imitation learning. Re-
inforcement learning models identify the main elements of the problem understudy by
analyzing how an agent interacts with its environment to reach a maximum reward while
pursuing an ultimate goal. The agent derives information from its environment in order to
perform actions which have impact based on a predefined reward. The imitation learning
is based on data collected from experienced human behavior in order to teach the system to
imitate an expert behavior. The authors combine reinforcement and imitation in their agent-
based simulation environment to investigate the safety of this approach and the advantages
of this model in relation to traditional search-based planning and optimization algorithms.
In their approach to collision avoidance methods, authors Chen Chen, Ma, Xu, Yuwang
Chen, and Wang [32] model the system as the multi-agent system. In their modelling
approach, they do not start from an individual ship but observe a group of ships in a
determined area. Each ship is joined by an agent. The avoidance of collision takes place
through cooperation between these agents, and the applied actions are mutually decided
(head-on, overtaking, or crossing). The method by which the system learns is multi-agent
deep reinforcement learning.
Koznowski and Lebkovski [33] defined the multi-agent system for the control of
unmanned port tugboats. This system achieves interoperability within the formation of
tugboats when performing joint port operations, such as assistance in ship maneuvers,
monitoring and patrolling port areas, conducting port inspections, and assistance in the
resolution of marine pollution. Here, decision making is centralized, where decision
making is not at the level of tugboat, but every tugboat is part of the team. Within the
multi-agent system, the functionality of the interoperable action of tugboats is achieved
using control algorithms and evolutionary algorithms. The authors also developed an
appropriate simulation environment, which explores possible scenarios of synergy of
tugboats in performing port operations.
Wu, Lei, He, Zhang, and Ji [34] also use deep reinforcement learning within their agent
model. They use this model in order to solve the optimization problem of unnamed ship
path planning. The waiting times at the defined port places are the values that are used
for the path optimization in the model. In accordance with the reinforcement learning
methodology, the learning agent interacts with the environment and gains knowledge of
the best possible action.
The application of deep reinforcement learning to unmanned vessels is also the subject
of research by Guo, Zhang, Zheng, and Du [35]. They modified the deep reinforcement
learning method to achieve an agent-based path planning model for unmanned ships.
Xiao, Ligteringen, Gulijk, and Ale [36] defined the model for predicting ship behavior,
which is based on previous behavior data of the observed ship. Data of ship movements
are obtained from the automatic identification system (AIS). Based on these data, a mathe-
matical model that provides information about the probability of future ship behaviors was
created. Using this data, an autonomous ship can adjust actions according to calculated
probability of other ship behavior that shares the same environment.
The next area of research of the multi-agent system application in unnamed surface
vessels is safety. The range of potential safety threats to the ship is broad. Kim, Perera,
Sollid, Batalden, and Sydnes [37] define challenges that are related to autonomous ship
safety. According to them, the safety of autonomous ships should be considered through
the following eight categories: navigational safety, ship system safety, ship structural
safety, personnel safety (due to different degrees of autonomy, it is also necessary to
analyze this category), equipment safety, security (piracy, cyberattacks, illegal boarding, and
robbery), cargo safety, and onboard emergency management (fire extinguishing, chemical
Electronics 2022, 11, 3182 8 of 14

and biological issues, and emergency evacuation). It is evident from this categorization
how wide the area of onboard safety is. The same goes for agent systems that support
safety on board.
Under conditions where manned vessels and unmanned vessels are in use, it is
necessary to further improve maritime search and rescue. It is necessary to have ready
methods and procedures enabling the prompt and efficient undertaking of necessary
actions in case of any need to search and rescue at sea. This is the focus of the Yu and
Xue research [38]. They introduced the architecture of the multi-agent-based intelligent
decision support system of maritime search and rescue. According to characteristics and
procedures of the search and rescue, the whole decision process is divided into a few
threads. Each thread is accomplished by one agent. These agents are: the main control
agent, distress information processing agent, accident situation evaluation agent, search
and rescue supporting information processing agent, search area determining agent, search
and rescue resources selecting and optimizing agent, and shipwreck data mining agent.
Testing of prototype systems shows that it is possible to effectively replace human decision
making in search and rescue processes. The system proved credibility for decision making
in the following tasks: (i) assessing disaster risk; (ii) operating in an environment where a
quick response in critical situations is needed; and (iii) assessing the necessary search and
rescue resources.
Jakob, Vanek, Hrstka, and Pechoucek [39] propose the multi-agent model, the main
task of which is to reduce the risk of pirate attacks at sea. Pirate attacks are a serious
security problem in some seas. When solving this problem, the authors not only address
unmanned ships, but suggest a model that is applicable to any ship, regardless of its
degree of autonomy. The model recognizes three different types of agents: merchant
ships, pirate ships, and navy warships. The reduction in risk of pirated attacks is achieved
by recommending the following actions: recommending transit corridors, group transit
(convoys), route randomization, and recommendations of patrol actions. Of course, this
model cannot be achieved without integrating a wide range of data from the real world.
Therefore, this emphasizes the need for the coordination of various systems, both in trade
and military navies.
Sumic, Males, and Rosic [40,41] introduced a topic of the multi-agent system-based
fire protection of autonomous ships. They developed the multi-agent ship firefighting
model, where the agents take over fire control of every room of a ship. Each room of a ship
was assigned its own agent. This agent has access to the firefighting elements contained in
his assigned room. These elements include smoke detectors, flame detectors, IR cameras,
fire foam generators, and similar elements. These elements serve as sensors and effectors
through which the agent receives information from its environment and takes actions that it
concludes are necessary to execute. An appropriate simulation environment was developed
and the defined model was tested.
The supervision of numerous onboard systems is an area where the application of
agents could be useful in unmanned vessels. Perera suggests using agents for condition
monitoring [3]. Condition monitoring implies monitoring of all ship functions to predict
the need for maintenance. The idea is that the agent initiates maintenance through the
condition-based maintenance system in case of the conclusion that a system requires
maintenance. In this approach, the question arises when an urgent repair of a system is
required while the vessel is, for example, in the middle of the ocean. One possible answer
to this question is to achieve interoperability with robotic systems that could be used in
situations such as the situation mentioned above.
When considering unmanned surface vessels, it is necessary to research an infras-
tructure that will support such vessels. Marine surveillance systems are an important
part of this infrastructure. These systems are particularly important in the areas of dense
traffic. Many vessels, and the different ways these vessels behave, create a dynamic and
complex system that needs to be carefully monitored. Multi-agent systems can also have a
significant application in this field.
Electronics 2022, 11, 3182 9 of 14

Mano, George, and Gleizes developed the adaptive multi-agent model for maritime
surveillance [42]. In this model, each vessel is represented by one agent. Each agent
monitors local anomalies to react appropriately in case it concludes that intervention
is necessary. The causes of ship behavior anomalies can vary from criminal intent to
various malfunctions.
A kind of continuation of the previous research is the research conducted by Brax,
Andonoff, and Gleizes. They proposed a multi-agent system designed to observe the
abnormal behavior of ships [43]. The developed system proposed by these authors is
not intended for independent decision making, but serves as a means of helping human
operators in early warnings about potential problems. The human operator shall decide
on the action to be taken. This information is feedback to the multi-agent system. Based
on feedback, the multi-agent system adapts its future alerts related to the potential ship’s
abnormal behaviors.
Singh, Nguyen, Kumar, and Lau also assign an agent to each ship within their multi-
agent model [44]. The purpose of their model is the management of maritime transport
in a busy waterway environment. In order to avoid congestions, the system based on this
model generates the speed recommendation for each vessel in the environment.
Some authors approach the problem of unmanned surface vessels through robotic tech-
nology. A certain number of such approaches can also be considered as an agent approach
if the robot is considered as a physical manifestation of an agent and artificial intelligence
methods are used. Often, these systems are operationalized through the cooperative activity
of a large number of robots, i.e., multiple unmanned surface vessels. Song and Chen [45]
conceptualize multiple unmanned surface vessels as cognitive agents who, besides sen-
sors and effectors, possess the following modules: (i) user interface and communication
module; (ii) learning module; (iii) module for modelling environmental information; and
(iv) decision-making module. Luo, Bae, Min, and Kim [46,47] model multiple unmanned
surface vessels for the purpose of applying teams of robotic vessels in environmental
operations, such as oil cleaning operations and other sea protection operations.
Finally, two more cases will be listed. Both cases are specific when compared to
previously described research. In the first one, the management of groups of fishing boats
is considered, while the second refers to the problem of avoiding the destruction of the
marine biosystem.
In their research, Vanhée, Borit, and Santos [48] consider a particular type of au-
tonomous vessel, namely autonomous fishing vessels. The focus of their work is the
application of multi-agent systems in autonomous fishing vessels. Their goal is the defini-
tion of autonomous fishing operation systems. This system is modeled as a multi-agent
system and its task is to perform actions necessary for integrated fishing operations, i.e.,
search for suitable fishing locations, operations with fishing nets, and the storage and
transport of the fish caught.
The multi-agent model 3MTSim is developed to simulate the spatiotemporal move-
ment of marine mammals and maritime traffic [49]. It is based on existing telemetry data
on fin, blue, and beluga whales, as well as on land-based tracking of humpback and minke
whales in the St. Lawrence Estuary in Canada. This is the decision–support tool to inform
management personnel in the estuary. This tool also can be used in autonomous ships
passing an ecologically sensitive area, i.e., an area populated by protected marine species.
The model represents the decision-making process as a function of environmental condi-
tions, the contextual setting, and objectives that are set. This paper also shows how wide
the planning process has to be when designing unmanned vessels. It is not expected that
each unmanned vessel must contain an implemented system enabling the avoidance of
disturbances of local marine species. This functionality can be provided as an external
service to vessels when passing through bio-sensitive areas.
Electronics 2022, 11, 3182 10 of 14

5. Discussion
A total of 24 papers examining multi-agent systems as facilitators of unmanned surface
vessel functions are presented. A summary of the main contribution of these papers is
given in Table 1.
It is shown that the most frequent application of agents in unmanned surface vessels
is in navigation, since a total of nine papers directly consider that purpose. The number of
papers engaged in the navigation of unmanned vessels is not exhausted. Two out of three
papers presented here that are classified in the field of simulations consider navigational
problems through simulations. By adding these two papers, the number of papers directly
or indirectly engaged in navigation of unmanned vessels almost reaches half of all papers
considered.
Safety is one of the essential elements of unmanned surface vessels. Among the
presented papers, three papers belong directly to the safety category. These are the papers
that consider reducing the risk of pirated attacks on the sea, fire protection of autonomous
vessels, and agent-based search and rescue operations at sea. All works categorized in the
field of navigation could also be categorized in the field of safety. This is understandable
because, for example, every system that covers navigation must have built-in methods to
avoid collisions, which is one of the functions that must be covered by vessel safety systems.
However, the presented papers show that there is enormous space for additional application
of intelligent agents in the field of unmanned surface vessel safety. For example, there is
no appropriate multi-agent system solution for avoiding, detecting, and responding to
events such as grounding, engine errors, propulsion system errors, hull damage, problems
with ship stability, failure of any ship subsystem, cargo problems, or cyber-attacks on the
ship’s systems. Intelligent agents could also be used in the monitoring and management
of ship systems such as the engine, propulsion system, electrical system, communication
system, and IT system. Only one paper was found to discuss the problem of vessel system
monitoring and control.
Three papers on maritime surveillance based on agents are also presented. Maritime
surveillance is not a direct function of a vessel, but this activity is essential for the sea
traffic. This importance is even more emphasized in the hybrid environment, where vessels
of different degrees of autonomy navigate in the same area, starting from vessels with a
human crew, through remote-controlled vessels, to fully autonomous unmanned vessels.
There are two papers in the group of elaborated papers that were singled out from the
previously analyzed topics. The first paper topic is about the coordination of unmanned
fishing boats, and the second covers the avoidance of sensitive marine species that may
inhabit areas of navigation. These papers are an indicator of the interdisciplinary potential
of the area, i.e., show how interesting it can be to extend the research of multi-agent system
applications in unnamed surface vessels to other related areas.
It should be noted that none of the presented papers discuss the interoperability
between several multi-agent systems on unmanned surface vessels. It is reasonable to
assume that the interoperability of such systems is necessary in order to achieve the
full functionality of autonomous unmanned surface vessels. This is difficult to achieve
without a common framework that would define common ontology, communication, and
cooperation procedures for the multi-agent systems installed on the vessel.
Electronics 2022, 11, 3182 11 of 14

Table 1. Main contributions of the presented papers.

Author(s) Publication Year Contributions

Liu, Sun, Du [25] 2007 Model for simulating the ship collision avoidance
Mano, George and Gleizes [42] 2010 Adaptive model for the maritime surveillance
Parrott, Chion, Martins, Lamontagne, Turgeon, Landry, Zhens, Marceau,
2010 Model of the marine mammals’ movements and the maritime traffic interaction
Michaud, Cantin, Menard, Dionne [49]
Yu, Xue [38] 2010 Intelligent decision support system of the maritime search and rescue
Brax, Andonoff and Gleizes [43] 2012 Model for observing the abnormal behavior of ships
Xiao, Ligteringen, Gulijk, Ale [36] 2012 Model for the ship behavior prediction
Jakob, Vanek, Hrstka, Pechoucek [39] 2012 Model for reducing the risk of piracy at sea
Vanhée, Borit, Santos [48] 2018 Model of the autonomous fishing vessels
Luo, Bae, Min, Kim [46,47] 2018, 2020 Environmental operations based on multi unmanned surface vessels
Singh, Nguyen, Kumar, Lau [44] 2019 Model for the management of maritime transport
Han, Zhang, Wang, Luo, Ran, Xu [26] 2019 Training model for the unmanned surface vessels
Xiao, Fu, Liye Zhang, Wanbing Zhang, Agarwal, Goh [27] 2019 Framework for design, planning, modelling, and evaluation of the autonomous shipping systems
Perera [3] 2020 Framework for the autonomous navigation of ships; collision avoidance
Perera [3] 2020 Model of monitoring all ship functions to predict the need for maintenance
Azimi, Salokannel, Lafond, Lilius, Salokorpi, Porres [31] 2020 Ship navigation model based on combination of reinforcement learning and imitation learning
Guo, Xiuguo Zhang, Yisong Zheng, Du [35] 2020 Model of the vessel path planning based on deep reinforcement learning
Murray, Perera [29,30] 2021, 2022 Model of the maritime situation control based on prediction of a ship trajectory
Koznowski, Lebkowski [33] 2021 Model for the control of unmanned port tugboats
Song, Chen [45] 2021 Conceptualization of multi unmanned surface vessels through the system of cognitive agents
Chen Chen, Ma, Xu, Yuwang Chen, Jin Wang [32] 2021 Multi-ship cooperative collision avoidance based on deep reinforcement learning
Sumic, Males, Rosic [40,41] 2021 Model for the autonomous ship firefighting
Wu, Lei, He, Zhang, Ji [34] 2022 Model for the path planning optimization based on deep reinforcement learning
Electronics 2022, 11, 3182 12 of 14

6. Conclusions
The analyzed papers show how wide the application area of multi-agent systems in
unmanned surface vessels is. An interoperability of different multi-agent ship systems is
not presented in any of the research.
Finally, it is reasonable to consider advantages and disadvantages of the methods,
models, or systems presented here. This is not possible now for the following reasons:
The first reason is the great heterogeneity of the analyzed area. It is unnecessary
to compare, for example, a multi-agent system designed to extinguish ship fires with a
multi-agent system designed to achieve autonomous navigation of a vessel. In order to
precisely detect advantages and disadvantages of a model, it is necessary to compare that
model with another model of the same functionality. Each of the approaches considered
here achieves some progress comparing with human-based solutions. Furthermore, in
analyzed papers, comparisons of the presented model with similar models are not given.
The second reason follows from the first, and that is the absence of appropriate
metrics. Different metrics should be introduced for each of the different scopes presented
here. For example, a special metric should be introduced to measure the efficiency of fire
extinguishing on vessels, and a special metric to measure navigation issues of vessels.
One way to achieve these metrics is to establish appropriate benchmark data bases. The
benchmark data bases should contain problem cases that proposed models should solve.
This requires the multidisciplinary approach and cooperation of various specialists.
It can be concluded that in the future we can expect more papers on the topic discussed
here. An increase in the number of published papers related to agent-based navigation is
to be expected, as well as the papers related to the less explored parts of this area. More
papers related to interdisciplinary topics should also be expected. All this research effort is
in order to achieve fully autonomous unmanned surface vessels in the near future.

Author Contributions: Conceptualization, L.M. and D.S.; investigation, L.M., D.S. and M.R.; method-
ology, M.R.; project administration, D.S.; resources, L.M.; writing—original draft, L.M. and D.S.;
writing—review and editing, M.R. All authors have read and agreed to the published version of
the manuscript.
Funding: This research received no external funding.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Marincic, A.; Budimir, D. Tesla’s multi-frequency wireless radio controlled vessel. In Proceedings of the 2008 IEEE History of
Telecommunications Conference, Paris, France, 11–12 September 2008; pp. 24–27.
2. Danilin, G.; Sokolov, S.; Knysh, T.; Singh, V. Unmanned Navigation Development Prospects Based on Structural Analysis of
Automated Vessel Control System. In Proceedings of the Journal of Physics: Conference Series, Coimbatore, India, 25–26 March
2021; p. 012185.
3. Perera, L.P. Deep Learning Toward Autonomous Ship Navigation and Possible COLREGs Failures. J. Offshore Mech. Arct. 2020,
142, 031102. [CrossRef]
4. Bernhardt, K. Agent-based modeling in transportation. Artif. Intell. Transp. 2007, 72, 72–78.
5. Wagner, N.; Agrawal, V. An agent-based simulation system for concert venue crowd evacuation modeling in the presence of a fire
disaster. Expert. Syst. Appl. 2014, 41, 2807–2815. [CrossRef]
6. Tang, H.; Li, D.; Wang, S.; Dong, Z. CASOA: An architecture for agent-based manufacturing system in the context of industry 4.0.
IEEE Access 2017, 6, 12746–12754. [CrossRef]
7. Nicholls, S.; Amelung, B.; Student, J. Agent-based modeling: A powerful tool for tourism researchers. J. Travel Res. 2017, 56, 3–15.
[CrossRef]
8. Rakic, K.; Rosic, M.; Boljat, I. A Survey of Agent-Based Modelling and Simulation Tools for Educational Purpose. Teh. Vjesn. 2020,
27, 1014–1020. [CrossRef]
9. Akyuz, E. A marine accident analysing model to evaluate potential operational causes in cargo ships. Safety Sci. 2017, 92, 17–25.
[CrossRef]
10. Smalley, D. The future is now: Navy’s autonomous swarmboats can overwhelm adversaries. US Off. Nav. Res. 2014, 5, 2014.
11. Wu, Q.; Wang, T.; Diaconeasa, M.A.; Mosleh, A.; Wang, Y. A Comparative Assessment of Collision Risk of Manned and Unmanned
Vessels. J. Mar. Sci. Eng. 2020, 8, 852. [CrossRef]
Electronics 2022, 11, 3182 13 of 14

12. International Maritime Organization. Outcome of the regulatory Scoping Exercise for the use of Maritime Autonomous Surface
Ships (MASS). In MSC.1/Circ.1638; IMO: London, UK, 2021; p. 105.
13. Lloyd’s Register. Cyber-Enabled Ships: ShipRight Procedure–Autonomous Ships; Lloyd’s: London, UK, 2016; p. 28.
14. Fan, S.; Yang, Z.; Blanco-Davis, E.; Zhang, J.; Yan, X. Analysis of maritime transport accidents using Bayesian networks. Proc. Inst.
Mech. Eng. Part O J. Risk Reliab. 2020, 234. [CrossRef]
15. Chang, C.H.; Kontovas, C.; Yu, Q.; Yang, Z.L. Risk assessment of the operations of maritime autonomous surface ships. Reliab.
Eng. Syst. Safe 2021, 207, 107324. [CrossRef]
16. Wooldridge, M. Intelligent Agents, Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence; New Ed edition ed.;
Weiss, G., Ed.; MIT Press: Cambridge, MA, USA, 1999; p. 648.
17. Ferber, J. Multi-Agent Systems: An Introduction to Distributed Artificial Intelligence; Addison-Wesley: Harlow, UK, 1999; p. 509.
18. Russell, S.J.; Norvig, P. Artificial Intelligence: A Modern Approach, Pearson, 2nd ed.; Prentice Hall: Upper Saddle River, NJ, USA,
2003; p. 1081.
19. Bratman, M. Intentions, Plans, and Practical Reason. CSLI Publication; New edition ed.; Center for the Study of Language and
Information: Stanford, CA, USA, 1987; p. 208.
20. Rao, A.S.; Georgeff, M.P. Modeling rational agents within a BDI-architecture. KR 1991, 91, 473–484. [CrossRef]
21. Rao, A.S.; Georgeff, M.P. BDI agents: From theory to practice. In Proceedings of the ICMAS, San Francisco, CA, USA, 12–14 June
1995; pp. 312–319.
22. Eberhart, R.C.; Shi, Y.; Kennedy, J. Swarm Intelligence; Elsevier: Amsterdam, The Netherlands, 2001.
23. Dorigo, M.; Stützle, T. Ant Colony Optimization: Overview and Recent Advances. In Handbook of Metaheuristics; International
Series in Operations Research & Management Science; Springer: Berlin/Heidelberg, Germany, 2010; Volume 146.
24. Parasuraman, R.; Sheridan, T.B.; Wickens, C.D. A model for types and levels of human interaction with automation. IEEE Trans.
Syst. Man Cybern. —Part A Syst. Hum. 2000, 30, 286–297. [CrossRef] [PubMed]
25. Liu, Y.H.; Sun, Z.Q.; Du, X.M. Modeling Ship Behaviors as an Intelligent Agent for Collision Avoidance Simulation. In Proceedings
of the 2007 International Conference on Machine Learning and Cybernetics, Hong Kong, China, 19–22 August 2007; pp. 88–93.
26. Han, W.; Zhang, B.; Wang, Q.; Luo, J.; Ran, W.; Xu, Y. A multi-agent based intelligent training system for unmanned surface
vehicles. Appl. Sci. 2019, 9, 1089. [CrossRef]
27. Xiao, Z.; Fu, X.; Zhang, L.; Zhang, W.; Agarwal, M.; Goh, R.S.M. MarineMAS: A multi-agent framework to aid design, modelling,
and evaluation of autonomous shipping systems. J. Int. Marit. Saf. Environ. Aff. Shipp. 2019, 2, 43–57. [CrossRef]
28. Noel, A.; Shreyanka, K.; Gowtham, K.; Satya, K. Autonomous ship navigation methods: A review. In Proceedings of the
Conference Proceedings of ICMET OMAN, Muscat, Oman, 5–7 November 2019.
29. Murray, B.; Perera, L.P. An AIS-based deep learning framework for regional ship behavior prediction. Reliab. Eng. Syst. Safe 2021,
215, 107819. [CrossRef]
30. Murray, B.; Perera, L.P. Ship behavior prediction via trajectory extraction-based clustering for maritime situation awareness.
J. Ocean. Eng. Sci. 2022, 7, 1–13. [CrossRef]
31. Azimi, S.; Salokannel, J.; Lafond, S.; Lilius, J.; Salokorpi, M.; Porres, I. A survey of machine learning approaches for surface
maritime navigation. In Proceedings of the Maritime Transport VIII: Proceedings of the 8th International Conference on Maritime
Transport: Technology, Innovation and Research: Maritime Transport’20, Barcelona, Spain, 17–18 September 2020; pp. 103–117.
32. Chen, C.; Ma, F.; Xu, X.; Chen, Y.; Wang, J. A novel ship collision avoidance awareness approach for cooperating ships using
multi-agent deep reinforcement learning. J. Mar. Sci. Eng. 2021, 9, 1056. [CrossRef]
33. Koznowski, W.; Łebkowski, A. Port Tugboat Formation Multi-Agent Control System. TransNav Int. J. Mar. Navig. Saf. Sea Transp.
2021, 15, 807–814. [CrossRef]
34. Wu, D.; Lei, Y.; He, M.; Zhang, C.; Ji, L. Deep reinforcement learning-based path control and optimization for unmanned ships.
Wirel. Commun. Mob. Comput. 2022, 2022, 7135043. [CrossRef]
35. Guo, S.; Zhang, X.; Zheng, Y.; Du, Y. An autonomous path planning model for unmanned ships based on deep reinforcement
learning. Sensors 2020, 20, 426. [CrossRef]
36. Xiao, F.; Ligteringen, H.; Van Gulijk, C.; Ale, B. Artificial force fields for multi-agent simulations of maritime traffic: A case study
of Chinese waterway. Procedia Eng. 2012, 45, 807–814. [CrossRef]
37. Kim, T.-E.; Perera, L.P.; Sollid, M.-P.; Batalden, B.-M.; Sydnes, A.K. Safety challenges related to autonomous ships in mixed
navigational environments. WMU J. Marit. Aff. 2022, 21, 141–159. [CrossRef]
38. Yu, W.; Xue, J. Intelligent decision support system of maritime search and rescue based on JADE. In Proceedings of the 2010
International Conference on E-Product E-Service and E-Entertainment, Henan, China, 7–9 November 2010; pp. 1–4.
39. Jakob, M.; Vanek, O.; Hrstka, O.; Pechoucek, M. Agents vs. pirates: Multi-agent simulation and optimization to fight maritime
piracy. In Proceedings of the AAMAS, Valencia, Spain, 4–8 June 2012; pp. 37–44.
40. Sumić, D.; Maleš, L.; Rosić, M. Agent Based Onboard Firefighting System. Trans. Marit. Sci. 2021, 10, 101–111. [CrossRef]
41. Sumic, D.; Males, L.; Rosic, M. An Agent-Based Ship Firefighting Model. J. Mar. Sci. Eng. 2021, 9, 902. [CrossRef]
42. Mano, J.-P.; Georgé, J.-P.; Gleizes, M.-P. Adaptive multi-agent system for multi-sensor maritime surveillance. In Advances in
Practical Applications of Agents and Multiagent Systems; Springer: Berlin/Heidelberg, Germany, 2010; pp. 285–290.
Electronics 2022, 11, 3182 14 of 14

43. Brax, N.; Andonoff, E.; Gleizes, M.-P. A self-adaptive multi-agent system for abnormal behavior detection in maritime surveillance.
In Proceedings of the KES International Symposium on Agent and Multi-Agent Systems: Technologies and Applications,
Dubrovnik, Croatia, 25–27 June 2012; pp. 174–185.
44. Singh, A.J.; Nguyen, D.T.; Kumar, A.; Lau, H.C. Multiagent decision making for maritime traffic management. In Proceedings of
the Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA, 27 January–1 February 2019; pp. 6171–6178.
45. Song, Y.; Chen, J. Agent-based multi-usv intelligent command and control cooperative system. In Proceedings of the Journal of
Physics: Conference Series, Coimbatore, India, 25–26 March 2021; p. 012035.
46. Luo, S.; Bae, J.H.; Min, B.-C. Pivot-based collective coverage control with a multi-robot team. In Proceedings of the 2018 IEEE
International Conference on Robotics and Biomimetics (ROBIO), Kuala Lumpur, Malaysia, 12–15 December 2018; pp. 2367–2372.
47. Luo, S.; Kim, J.; Min, B.-C. Asymptotic Boundary Shrink Control With Multirobot Systems. Ieee Trans. Syst. Man Cybern. Syst.
2020, 52, 591–605. [CrossRef]
48. Vanhée, L.; Borit, M.; Santos, J. Autonomous fishing vessels roving the seas: What multiagent systems have got to do with it. In
Proceedings of the Autonomous Agents and Multiagent Systems, Stockholm, Sweden, 10–15 July 2018.
49. Parrott, L.; Chion, C.; Martins, C.; Lamontagne, P.; Turgeon, P.; Landry, J.; Zhens, B.; Marceau, B.; Michaud, D.; Cantin, G.; et al.
3MTSim: An agent-based model of marine mammals and maritime traffic to assist management of human activities in the Saint
Lawrence Estuary, Canada. In Proceedings of the Report submitted to the Scientific Committee of the International Whaling
Commission 62nd Annual Meeting, Agadir, Morocco, 30 May–11 June 2010.