Journal of Experimental & Theoretical Artificial

Intelligence

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/teta20

A comprehensive survey on machine learning

approaches for dynamic spectrum access in
cognitive radio networks

Amandeep Kaur & Krishan Kumar

To cite this article: Amandeep Kaur & Krishan Kumar (2020): A comprehensive survey on
machine learning approaches for dynamic spectrum access in cognitive radio networks, Journal of
Experimental & Theoretical Artificial Intelligence, DOI: 10.1080/0952813X.2020.1818291

To link to this article: https://doi.org/10.1080/0952813X.2020.1818291

Published online: 14 Oct 2020.

Submit your article to this journal

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=teta20
JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE
https://doi.org/10.1080/0952813X.2020.1818291

A comprehensive survey on machine learning approaches for

dynamic spectrum access in cognitive radio networks
Amandeep Kaur and Krishan Kumar
Department of Electronics and Communication Engineering, National Institute of Technology, Hamirpur, India

ABSTRACT ARTICLE HISTORY

Due to exponential growth in demand for radio spectrum for wireless Received 2 July 2019
communication networking, the radio spectrum has become over- Accepted 29 August 2020
crowded. The fixed spectrum allocation policy of the radio spectrum KEYWORDS
leads to inefficient utilisation of the available spectrum, which diverted Cognitive Radio; dynamic
the attention of researchers towards different intelligent techniques to spectrum management;
access the spectrum dynamically and efficiently. The concept of Cognitive heterogeneous networks;
Radio (CR) has been considered as a promising technology to solve the intelligent techniques;
problem of spectrum scarcity through the utilisation of various unutilised machine learning
spectrum bands. In a future network deployment, multiple radio access
networks may coexist having different characteristics. Hence, it becomes
a challenge for CR networks to select the optimal network out of available
networks. For efficient realisation, CRs requires intelligent spectrum man­
agement techniques for Dynamic Spectrum Management (DSM). Till now,
there does not exist a literature survey that addresses the spectrum
management with machine learning techniques in an intelligent manner.
Hence, this paper presents the detailed classification and comprehensive
survey of various machine learning techniques for intelligent spectrum
management with their paradigms of optimisation for cognitive radio
networks. The paper also provides new directions and open issues for
the research community to work further in CR networks.

Introduction
The exponential growth in the demand for extra spectrum resources become more prominent to
support numerous wireless services leads to the advent of new technologies for high-speed data
networks (X. Zheng et al., 2008). According to Cisco, global mobile data traffic will increase sevenfold
between 2017 to 2022 (Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update,
2017–2022, 2018).
But radio spectrum is an inherently limited resource in which frequency bands are exclusively
assigned to the licenced users called Primary User’s (PUs) for a long term basis which leads to
spectrum scarcity in a particular spectrum band. In contrast, a survey of spectrum utilisation being
conducted by the Federal Communication Commission (FCC) has indicated that many portions of
the radio spectrum are not in use, called spectrum holes, for a significant amount of time which leads
to under-utilisation of the assigned spectrum. To overcome this problem, FCC reforms the spectrum
allocation policy and allows unlicensed users, also known as secondary users/Cognitive Radios (CRs)
to borrow unused radio spectrum from licenced users through Dynamic Spectrum Access (DSA)
(FCC, 2003).

CONTACT Amandeep Kaur adeep5524@yahoo.com Department of Electronics and Communication Engineering,

National Institute of Technology, Hamirpur, Himachal Pradesh 177005, India
© 2020 Informa UK Limited, trading as Taylor & Francis Group
2 A. KAUR AND K. KUMAR

The CR as an intelligent radio that can be reprogrammed and reconfigured dynamically. Such
intelligent radio automatically detects available channels in the wireless spectrum, changes accord­
ingly its transmission and reception parameters to switch between available vacant spectrum bands
dynamically. This process is known as Dynamic Spectrum Access which can be realised only with CR
networks (Akyildiz et al., 2008).
Figure 1 presents a system model showing the available spectrum holes of various networks such
as Television, GSM, Satellite, and Wi-Fi. CR requires a spectrum hole for its application which requires
intelligent techniques to access the spectrum dynamically. The intelligent spectrum management
includes spectrum sensing, spectrum decision, spectrum sharing, and spectrum mobility.
Figure 2 shows the currently fixed spectrum allocation policy decided by governmental agencies.
Due to this fixed spectrum allocation policy, some frequency bands are highly utilised and some are
lightly utilised. It is observed from Figure 2 that networks 1 and 2 are lightly utilised whereas
networks 3 and 4 are highly utilised. It is shown that there exist large numbers of spectrum holes
of heterogeneous networks that can be utilised with dynamic spectrum allocation policy. Figure 3
shows a dynamic spectrum allocation policy where CR access the unused spectrum. Thus, the
demand for extra spectrum bands for wireless applications can be solved with flexible usage of

Figure 1. System model showing spectrum holes of various networks available for CRs having multiple requirements.

Figure 2. The fixed spectrum allocation policy.

 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 3

Figure 3. The dynamic spectrum allocation policy.

available spectrum holes. As an intelligent wireless network, CR shares the spectrum with PUs in an
opportunistic manner to avoid interference among multiple spectrum users with its learning abilities
to adapt and act in a dynamic environment (Xu et al., 2014).
To address the problem of spectrum scarcity, each CR:(Akyildiz et al., 2008).

● Determine the presence of spectrum holes called spectrum sensing.

● Determine available channels including spectrum selection, routing protocol, and reconfigura­
tion called spectrum decision.
● Allocation of resources and serve its users without causing interference to PUs called spectrum
sharing.
● Vacate the channel on the arrival of PUs called spectrum mobility.

The most important issue is how to access the spectrum dynamically without interference with PUs.
This requires CR which is intelligent by nature and having learning capability to learn from past
experiences, which is an essential part of intelligence. Haykin envisioned CR as an intelligent wireless
communication system that is based on the methodology of understanding by building to achieve
two goals: highly reliable communication and efficient utilisation of radio resources (Khozeimeh &
Haykin, 2012). To perform these tasks, CRs must be equipped with intelligence like a human (Zafari et
al., 2019). The implementation of such capabilities requires functional architecture like the brain to
access the spectrum efficiently i.e. Cognitive Engine (CE) (Dong et al., 2012). The CE acts as a brain of
CR to perform various cognitive functions intelligently and provides learning capabilities to imple­
ment the cognitive cycle by making use of machine learning algorithms. The CR technology is based
on Software Defined Radio (SDR), which allows the radio to reconfigure through software, based on
the interaction with the surrounding environment in which it operates. However, in recent years
machine learning algorithms in CR networks gains a lot of attention from researchers (Kotsiantis,
2007) (Thilina et al., 2013). A look at a recent literature survey on CR networks reveals that various
leaning techniques are proposed that have been applied to numerous CR applications (Yau et al.,
2012)(A. He et al., 2010). Some authors presented machine learning techniques, particularly focused
on spectrum sensing and decision making in CR networks (Abbas et al., 2015).
Further, CR has to work under unknown environments where complete Channel State Information
(CSI) is not present or only partial CSI is available but it has to estimate the behaviour from other CRs
4 A. KAUR AND K. KUMAR

present in network to coordinate its actions. Another author surveyed machine learning techniques for
decision making and feature classification in different environmental conditions. In general, learning
becomes an indispensable part of CR if the input-output relation of the system is not known, or as in
the case of CR networks, due to channel uncertainty. Thus, learning becomes a necessary tool to
estimate that channel characteristics to reduce error probability (Zafari et al., 2019). In CR’s, several
parameters need to be adjusted simultaneously such as availability of spectrum (Tripathi et al., 2011),
transmit power (G. Yang et al., 2015), adaptive coding and modulation schemes (Zafari et al., 2019),
antenna selection, rate control (Hanif et al., 2011), spectrum handoff (Thakur et al., 2017), etc., and it is
not possible to identify and adjust all these parameters simultaneously. Thus, learning techniques can
be applied to perform specific CR tasks with efficiency and accuracy.
An overview of an existing literature survey is presented in Table 1. The survey reveals that
most of the authors have shown their interest in the specific issue of spectrum management.
Most of the authors provide different types of solutions to address the spectrum scarcity
problem. Some of the authors specifically discuss a problem related to spectrum management
whereas no work has been presented with machine learning techniques for dynamic spectrum
management which includes spectrum sensing, spectrum decision, spectrum sharing, and
spectrum mobility. In this survey, spectrum management with intelligent techniques, consider­
ing learning as an important parameter has been discussed. In particular, we provide an in-depth
discussion on different types of intelligent techniques such as Artificial Neural Network,
Metaheuristic Algorithms, Support Vector Machine, Bayesian Learning, Game Theory, and
Hidden Markov Models. The pros and cons of each technique in the context of spectrum
management have also been discussed. We firstly present a spectrum management framework.
Then we introduced various intelligent techniques used in CR networks as well as a survey of
state-of-the-art achievements of these techniques for dynamic spectrum management in CR
networks. The major contributions of this paper are summarised as follows:

Table 1. Comparison with an existing literature survey.

Reference Focused area Description
Our paper Spectrum This paper discussed complete spectrum management with intelligent
management techniques, its strengths, limitations, and evaluations based on the
requirement of CR networks.
(Akyildiz et al., 2006) Cross-layer design The author briefly investigates next-generation CR wireless networks for
issues DSM and addresses its cross-layer design issues.
(A. He et al., 2010) AI in CR networks This paper reviewed several artificial intelligence techniques that have
been applied to numerous CR applications but not specifically to spectrum
management.
(Bkassiny et al., 2013) Feature classification & Machine learning techniques in CR networks for feature classification
decision making (spectrum sensing) and decision-making under the non-Markovian
environment.
(Thilina et al., 2013) Spectrum sensing Unsupervised (K-means clustering, Gaussian mixture model) and
Supervised (Support vector machine, weighted K-nearest neighbour)
learning-based techniques implemented for cooperative spectrum sensing.
(Abbas et al., 2015) Spectrum sensing & The paper provides a survey on artificial intelligence and machine learning
decision making techniques to address spectrum sensing and decision-making issues
(Qadir, 2016) Routing The authors targeted cognitive routing as an issue.
(Azhar et al., 2011) AI in wireless networks Here, the artificial intelligence framework of neural networks in wireless
networks has been presented including CR networks.
(Ramzan et al., 2017) Multiple objectives This paper focused on the optimisation problem with multiple objectives.
optimisation
 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 5

● This paper presents a comprehensive survey of various intelligent techniques and presents
their applications in CR networks. The target of this paper is to provide a focused survey of
these techniques and evaluates its performance in spectrum management as major CR tasks
which include spectrum sensing, spectrum decision, spectrum sharing, and spectrum mobility.
● This paper presented state-of-the-art achievements in applying intelligent techniques to CR
networks along with their strength and limitations to provide an overview of active research in
the area of CR networks.
● The paper discusses research issues and challenges that are still an open issue and need the
attention of researchers.

The rest of the paper is organised as follows: Section 2 provides the spectrum management frame­
work for CR networks. The taxonomy of various intelligent techniques is presented in Section 3. The
evaluation of intelligent techniques along with their strengths and limitations are presented in
Section 4. Section 5 presents research issues, challenges, and future directions in the area of
spectrum management. Finally, the concluding remarks are given in Section 6.

Spectrum management framework

To address the critical challenges associated with the co-existence of PUs and CRs in CR networks,
CRs are required with the following functions in spectrum management:

● Interference Management: The important role of CR is to resolve the interference issues with
PUs which can be resolved in two manners:

(a) Proactive: In this, the CR switches its communication before the arrival of PUs.
(b) Reactive: CR switches its communication after the arrival of PUs.

● Quality of service (QoS) Awareness: In heterogeneous networks, the selection of an appro­

priate spectrum band is necessary to provide QoS aware communication

CR technologies provide intelligent spectrum management capabilities that could meet the ever-
increasing demand of spectrum, thus empowers radio with etiquette to avoid interference. The
complete spectrum management framework is presented in Figure 4. It shows that CR spectrum
management framework has four main steps:

Figure 4. The complete spectrum management framework.

6 A. KAUR AND K. KUMAR

(a) Spectrum Sensing: The first step of spectrum management starts with spectrum sensing on
the arrival of CR. Spectrum sensing provides the ability to detect the spectrum holes (in terms
of the frequency band, location, duration of availability) and PU activity by periodically
sensing the spectrum and use it without interference with PUs (Noorshams et al., 2010).
(b) Spectrum Decision: Spectrum decision includes spectrum analysis and decision making. It
deals with the selection of appropriate spectrum bands according to the required QoS. The
spectrum hole is characterised, with various parameters such as multipath environment,
operating frequency, interference, link delay (Pourpeighambar et al., 2017) and number of
PUs using the spectrum, being considered (X Liu et al., 2013). After spectrum analysis, the
decision has been made to access the spectrum hole. Various optimisation techniques can be
applied to obtain optimal decisions depending upon the radio environment whether it has to
optimise single objective or multiple objectives (Ramzan et al., 2017).
(c) Spectrum Sharing: The main function of spectrum sharing is to avoid collision among the CRs
as multiple users trying to access the spectrum while maintaining QoS. Spectrum sharing
deals with coordination among CRs to access the shared channels, resource allocation, and
spectrum access (Kour et al., 2018)(Akyildiz et al., 2008). The spectrum allocation includes the
assignment of spectrum band in a cooperative (Bayrakdar, 2020). or non-cooperative manner
using handshaking protocols of transmitter and receiver (X Liu et al., 2013). The spectrum
access includes collision avoidance among CRs in the access phase
(d) Spectrum Mobility: Spectrum mobility deals with spectrum handoff and connection manage­
ment. CRs are usually regarded as visitors which often need to switch from one spectrum hole
to another on the arrival of PUs in a dynamic environment (Christian et al., 2012). Spectrum
mobility utilises a reactive and proactive approach for the detection of PUs whereas connec­
tion management ensures that CR continuously transmits its data in a new spectrum hole
(Thakur et al., 2017).

Intelligent techniques in cognitive radio networks

A CR as an intelligent wireless device, which is aware of its environment, capable of learning and
adapt from its surrounding environment and learning considered to be an indispensable component
of an intelligent system which is considered to be a basic tool of CRs for dynamically access the
available spectrum without interfering with PUs. CR must have the capability to learn from current
observations and past experiences. However, CR has to work under different radio environments in
which CR might have full or partial CSI or sometimes under a completely unknown environment.
However, not being idealistic, due to fluctuations in the wireless channel, channel estimation errors,
and quantisation errors, it is not possible to obtain perfect CSI. Thus, CR might apply an intelligent
algorithm to estimate its actions concerning other CRs for spectrum management. Various techni­
ques in CR networks are shown in Figure 5.

Intelligence with artificial neural networks

Artificial Neural Networks (ANN) provides Artificial intelligence (AI) which aims to incorporate
intelligence in machines so that machines can perform like an expert. Also, machine learning,
a subclass of AI, gained a lot of attention from the researchers in CR networks. Learning can be
classified as supervised, unsupervised, and reinforcement learning (Jiang et al., 2017)(Duda et al.,
2001). Supervised learning learns from the training set and requires prior information about the
environment. On the other hand, unsupervised learning does not require any training set and it
performs self-adapting actions without any prior knowledge about the environment.
Reinforcement learning or learning with critic, the learning agents learn by observing actions of
other agents, and its performance is influenced by learning regime and operating environment
(Duda et al., 2001)(Dandurand & Shultz, 2009). However, it is a particular point of interest here that
 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 7

Figure 5. Intelligent techniques in CR networks.

how these intelligent algorithms have been applied to access the spectrum dynamically. In this
section, we present various ANN techniques that can be applied to CR networks.
Artificial Neural Network (ANN)
Artificial Neural Network (ANN or simply NN) consists of several interconnected neurons,
identical to the human brain, which is made up of real biological neurons. The first artificial
neuron was introduced in 1943 by the neurophysiologist W. McCulloch and logician Walter Pits.
ANN composed of a large number of artificial neurons which are interconnected to each other
that mimics the behaviour and natural processing of biological neuron .i.e., learning and
organisation (Tsagkaris et al., 2008). The human brain can perform fast due to its special ability
of parallel data processing and NN tries to mimic the behaviour of solving narrowly defined
problems. As stated earlier, NN consist of a pool of neurons, and these neurons are configured
in the form of layers and connected to other nodes with links defined by the weight wjk , which
determines the effect of the signal of a neuron j on a neuron k. In NN, three types of layers are
present, which are distinguished as the input layer, hidden layer, and output layer. The input
layer consists of neurons, which receive data whereas output layers consist of neurons, which
send data out from the NN layer and the hidden layer consists of neurons whose input, and
output remain in NN. Each neuron within NN uses some activation function to process the
input signals sk it receives from (a) neighbours belonging to different layers (b) external
sources. The type of activation function used depends on the problem to be addressed.

● Types of NN and Machine Learning

In this section, we describe different NN models.

Supervised NNs
In supervised NNs, the input and output are known and its objective is to discover the
relationship between two. The two main NN models are as Feed Forward NNs (FFNNs) and
Recurrent NNs (RNNs). FFNNs are further classified as: Single layer FFNNs (SL-FFNNs) and
Multilayer FFNNs (ML-FFNNs).
8 A. KAUR AND K. KUMAR

(a) Feed Forward NNs

In FFNNs, neurons feed their values in the forward direction. It is of two types describes as:

(i) Single-layer FFNN (SL-FFNNs): It consists of single-layer of neurons. Due to its single-layer
network, learning and training mechanism is easy but it has limited utility, as it is not able
to solve real-world complex problems.
(ii) Multilayer FFNN (ML-FFNNs): It consists multiple layers (hidden layers) between input and
output layer. These hidden layers solve complex problems using the activation function. As
stated earlier, some activation function is required to process the information, and the
selection of activation function depends on the problem to be solved. Table 2 shows various
types of activation functions to address different problems.

Various type of ML-FFNNs using different activation functions leads to slight variation in the
network. (Tumuluru et al., 2010), designed a multilayer spectrum predictor using a neural network,
which does not require prior knowledge of traffic characteristics of PUs. CR may predict channel
status based on sensing history which further saves sensing energy. (Liu et al., 2015) proposed ML-
FFNN based dynamic Chinese restaurant game for spectrum sensing, which considered CR as
customers to learn the table state (available spectrum hole) in the restaurant (network).
Some important examples of ML-FFNNs include Radial Basis Neural Networks and Convolutional
Neural Networks which are described below:

● Radial Basis Neural Networks (RBNN): RBNN composed of three layers: input layer, hidden layer,
and output layer. The hidden layer consists of neurons using a Gaussian transfer function gives
the output as
0 � �2 1
�r r�
�
B �x �
X B μ� CC
Yj ¼ wj expB 2 C (1)
j
@ σ j A

In Gaussian transfer function, ! μj and σj represents centre vector and width of jth neuron
x is input, ~
in hidden layer respectively, which need to be calculated for each neuron, ||.|| denotes Euclidean
distance. However, an initial value of the centre vector and width affects the prediction ability of the
network. An unsupervised learning technique i.e. K-means clustering (K-RBF) algorithm can be used
to obtain the values of centre vector and width of the hidden neurons associated to construct and
train RBNN in a more accurate way. The output of the transfer function weighted and summed as
shown in Equation (1).
Zhang (S. Zhang et al., 2013) proposed K-RBF for spectrum sensing based on previous information
of the PU spectrum which reduced prediction error to one-third as compared to RBF. Researchers
proposed a design of CE based on the RBF and Genetic Algorithm (GA) for multi-objective optimisa­
tion. RBF has strong learning capability while GA is good at multi-objective optimisation (Y. Yang
et al., 2012).

Table 2. Types of the activation function.

Sr. No. Activation Function Equation Application
1. Log Sigmoid Function Fk ðsk Þ ¼ 1þe1 sk Logistic regression problem
s s
2. Hyperbolic tangent Fk ðsk Þ ¼ eeskk þee skk Multi-layer NN
3. Linear Fk ðsk Þ ¼ sk Linear regression problem
 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 9

● Convolutional Neural Networks (Conv-Nets or CNN): CNN is another variant of ML-FFNN that has
been used effectively in the areas of image recognition and classification. CNN exploits spatially
local correlation by enforcing local connectivity with adjacent layer neurons, which does not
take entire data from preceding layers. LeNet was the very first CNN which become an
important tool for machine learning. Key operations of CNN include convolution, non-
linearity (ReLU), pooling (or sub-sampling), and classification (Fully connected layer). For
example, when an image is passed to CNN, features are extracted from the image using
convolution preserves the spatial relationship between pixels of an image by utilising a small
square of input data. Another operation i.e. ReLU has been performed after convolution, which
introduces non-linearity in CNN. ReLU stands for Rectifier Linear Unit, which is an element-wise
non-linear operation (applied per pixel) and replaces negative pixels in feature map by zero.
Pooling operation can be done after convolution+ReLU operation, which can be done in
different ways such as sum, average, max, etc. This operation reduces the dimensionality of
each feature map by only retaining important information. Finally, the classification process is
done. The output extracted from previous operations extract high-level features that are used
for feature classification of the input image. The term” Fully connected” means every neuron of
the preceding stage is connected to every neuron of the succeeding stage that classifies and
forms various classes based on the training dataset. Further, CNN can be trained using gradient
descent and backpropagation (Lecun et al., 1998). Backpropagation calculates the gradients of
error concerning all weights in the network and updates weights and parameter values using
gradient descent to minimise an output error.

Lee (Lee et al., (2019) cooperatively implemented the CNN spectrum sensing technique, in which
optimal strategy is used for combining sensing results of individual CRs obtained using CNN. Sensing
based on deep neural networks is used to learn from a large set of data through a backpropagation
algorithm. Deep sensing provides optimised combining strategy based on spectral and spatial
correlations of channel i.e. based on the location of CRs and PUs characteristics. Selim et al. (2018)
presented the CNN spectrum monitoring framework for radar bands in spectrum sharing scenarios.
The main idea behind this framework is to detect the presence of radio signals in the radio spectrum
even when this signal is overlapped by other signals due to simultaneous transmission.

(b) Recurrent Neural Network (RNN)

The idea behind RNN is to make use of sequential information, with the output being dependent on
previous computations. Here, the neurons are connected in a cyclic manner, which allows exhibiting
dynamic temporal behaviours shown in Figure 6. Unlike FFNNs, RNN does not propagate information
directly to the next connected layer but use their internal memory to process the input.
RNN are called recurrent because they perform the same operations for every element in the
input sequence.

Figure 6. Recurrent neural network.

10 A. KAUR AND K. KUMAR

Let X t ,Ot andHt be the input, output, and state of the hidden layer(s) on time t. In simple
mathematical form, RNN can be expressed as

Ot ¼ FðHt ; θÞ (2)

Ht ¼ FðHt 1 ; X t ; θÞ (3)

where the parameter θ defines weight and bias for the network. Equation (2) states that outputOt at
a particular time t depends upon the state of the hidden layer Ht and θ. Further, Ht depends upon its
previous statesHt 1 and inputX t at a time t as stated in Equation (3). To train RNN, the backpropaga­
tion algorithm through time approach is used which ‘unfold’ RNN in time and converted into FFNNs
consisting of identical copies of RNN. RNN consist of a large number of layers as compare to ML-
FFNNs and also training process is based on the passage of information through these layers and
propagation of unsuitable information through the network leads to the accumulation of large error
which leads to inaccurate results. Usually, the gradient descent method is used for backpropagation
training.
Deep Recurrent Neural Network (DRNN): DRNN (Tang & Li 2017) consists of multiple hidden layers
that enable the prediction of the spectrum of multiple time slots in CR networks since existing
methodology only predicts the spectrum of a single slot. Here, the channel state is divided into slots
and forming time series of channel state. Due to multiple hidden layers in DRNN, it is difficult to train
using a gradient descent method, which leads to slow convergence and gradient disappearance. To
avoid these problems, Extended Kalman Filter (EKF) for estimation of weights is used which over­
comes the problem of slow convergence and vanishes gradients in a gradient descent method.

Unsupervised NNs
As the name indicates unsupervised NNs only know the input and the main objective is to discover
patterns in features of input data. The system learns using certain test dataset and allows a lower-
dimensional representation of input data.

(a) Unsupervised ML-FFNNs (Auto-encoders)

Auto-encoders are the type of FFNNs specially designed for dimensionality reduction. In
other words, auto-encoders provide the same output as the input. In auto-encoders, encoder
function learns from input to representation and decoder function back from representation
to input i.e. reconstruction. Denoising auto-encoders (Bengio et al., 2013) are the extension
of basic auto-encoders, which attempts to address, identify function risk by introducing
noise. Random noise is added to the input in the form of percentage and then this noisy
data is fed in auto-encoder, which finds correlations within the input data to find correct
data instead of noise added data. In Sparse auto-encoders, a sparsity constraint on the
hidden layer is applied for extracting interesting features from unlabelled data even if
numbers of hidden layers are large. Authors proposed a novel modulation classification
method using denoising sparse auto-encoder as a classifier to improve spectrum sensing,
i.e. to avoid interference with PUs (Zhu & Fujii, 2016). Moreover, the denoising process
improves the performance of noise suppression by training the network with a corrupted
database. Further, Variational Auto-Encoders (VAE) emerged as a powerful unsupervised
learning framework for generating complicated data. They can learn a low dimensional
representation of high dimensional data (Doersch, 2016). Contractive Auto-encoders (CAE)
make use of explicit regulariser which learns a function even in slight variations of input
values. CAE adding a well-chosen penalty term to cost function which enables an auto-
encoder to perform better (Rifai & Muller, 2011).
 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 11

(b) Unsupervised Recurrent NNs (URNN)

URNN is a complex type of NNs, which are inspired by ideas from statistical physics which model
system as an energy function. RNNs are computationally powerful and learn from the temporal
behaviour of given training data. Compared to the FFNN, RNN is more powerful in representing
complex dynamics and having a compact size. Two main types of URNNs are Hopfield Network and
Boltzmann Machine, which are described next.

● Hopfield NNs: Hopfield NNs is a dynamic auto-associative model, which can store information in
a dynamically stable structure. Each Hopfield network has some energy function (Lyapunov
function) associated with it, which always tries to converges to a local minimum of the function.
Hopfield NNs is biologically plausible, as its working is similar to the human retina. Hasegawa
(Hasegawa et al., 2014) proposed an optimisation algorithm for decision making on radio
resources in centralised and decentralised CR networks. In a decentralised network, the energy
minimisation dynamics of Hopfield NNs are considered.
● Boltzmann Machine(BM): BM is a stochastic RNN, like the Hopfield network, in which the
network unit finds a global minimum of the energy function. BM is designed for discrete
variables having the disadvantage of being slow. Another problem with BM is it stops learning
correctly when the machine statics grows exponentially with the size of the machine and with
the magnitude of connection strength. Other variants of BM are Restricted Boltzmann Machine
(RBM) and Deep Boltzmann Machine (DBM).
(i) Restricted Boltzmann Machine (RBM): As the name indicates, a restriction is imposed with
neurons that they have symmetric connection between different groups of nodes but no
connections between the same group of nodes which allows more efficient training algo­
rithms than BM. MohanaPriya and Shalinie (2017) presented a secure routing protocol for CR
networks. The RBM algorithm self learns the routing procedure between source and destina­
tion node. Further, it provides energy-efficient routing to prevent flooding attacks.
(ii) Deep Boltzmann Machine (DBM): DBM (Salakhutdinov & Hinton, 2009) have the potential to
learn complex internal state representations, which is useful in speech recognition problem. It
also develops high-level representations to tune its model for a specific task from unlabelled
data as well as from limited labelled data.

Unsupervised competitive NNs

Unsupervised competitive learning is a form of unsupervised training where output neurons are said
to compete for input patterns. It employs a winner-take-all strategy since only winning neuron is
updated. Two major techniques of unsupervised competitive learning are

a. Self-Organising Maps/Kohenon Maps

Self-Organising Maps (SOM) are the special class of ANN, which is inspired by the cortex of the
human brain. SOM makes use of competitive learning in which neurons compete with each other to
win and then the winner neuron displaced to feature space to form clusters. The competition is
induced by inserting negative feedback between neurons, which forces neurons to organise them­
selves. For obvious reason, such a network is called self-organising maps. SOM was introduced by
Kohenon used to assist CRs to choose among various configurations to operate by considering bit
rate predictions (Tsagkaris et al., 2012). Khozeimeh and Haykin (2012) proposed a self-organising
dynamic spectrum management technique for CR networks in which CRs continuously senses the
environment, extract PUs and neighbouring CRs activity patterns, and store to obtain knowledge
that significantly reduces the probability of collision.
12 A. KAUR AND K. KUMAR

b. Adaptive Resonance Theory

Adaptive Resonance Theory (ART) is another class of unsupervised competitive NNs that is designed
for object identification and recognition based on matching between observer expectations and
sensory information. This ART matching criterion is set by vigilance parameter, ƿ (threshold of
recognition). High vigilance permits weight updation of winning recognition neuron while low
vigilance inhibits weight updation of recognition neuron and search procedure is carried out
(Carpenter et al. 2016). Another variant of ART is ART 1 and ART 2. ART 1 is the simplest type of
ART which accepts an only arbitrary sequence of binary input patterns whereas ART 2 supports
analog inputs patterns as well as binary input patterns (Carpenter & Grossberg, 1987).
In CR networks, a channel sensing algorithm based on ART-2 is proposed for cognitive wireless
mesh networks in which the spectrum is divided into disjoint sub-bands and sensing is performed in
each sub-band. This sensed information is further fused in a fusion centre, which is defined as
a pattern classification problem. In this algorithm, only pattern phase information is considered
which leads to wrong pattern classification (Zhu et al., 2008). Further modified ART-2 is described
which considers amplitude as well as phase information.

Reinforcement learning and NNs

Reinforcement Learning (RL) is another AI approach that enables an agent (decision maker) to
observe its state, choose an action in a particular state to obtain rewards. The transition from one
state to another depends on several factors such as previous state, action taken, and next state. The
main goal is to estimate the reward function for each state-action pair accumulates knowledge and
maximise reward function. To apply the RL model (single-agent approach), identified by state, action,
and reward. In the multi-agent approach, the set of agents is enabled to learn about each other
information to maximise its reward function. Unlike supervised NNs, it does not have any prior
knowledge about its input and output. It learns online from the environment and builds the knowl­
edge to achieve its objectives.
RL in the context of CR networks which provides intelligence in terms of dynamic channel
selection, spectrum sensing, routing, spectrum allocation (Y. Wang et al., 2019), power control
mechanism (K.L.A. Yau et al., 2014). Further, enhanced RL techniques in CR networks have also
been presented. The authors in (Saleem et al. 2015) addressed routing issues by clustering mechan­
isms using the RL approach. Further, routing based clustering improves network stability and
scalability. Moreover, spectrum aware cluster-based routing is presented which overcomes the
challenges of multihop routing.
In RL, an agent maps the situation (channel state) to maximise its reward function on a long term
basis. The agent learns continuously and the environment responds to the state through its action by
state transition after each epoch. The agent uses optimal policy to decide each time while taking the
next step which maximises its rewards. To obtain an optimal policy, the Dynamic Programming
approach can be used when perfect information about the system is available. The system is
modelled using Markov property which makes the RL model equivalent to Markov decision
Process (MDP). On the other hand, when perfect system information is not available, but the
sequence of past state, action, and rewards are available then the Monte-Carlo method can be
applied to obtain an optimal policy. Another approach named Temporal Difference (TD) learning
approach does not form the system model and obtain its optimal policy based on prediction.
Q-Learning (Q-L) and SARSA are the most widely used TD learning techniques (Sutton & Barto,
1998). These schemes are implemented for resource allocation in CR networks and it is observed that
SARSA converges slower than Q-L as shown in Figure 7. This is due to the reason that Q-L follows
a greedy approach for action selection according to off-policy whereas SARSA selects its action
corresponding to the current policy obtained.
Lo et al., (2010) proposed RL based cooperative spectrum sensing model in which CR learns to
find an optimal set of cooperative neighbours which reduces sensing delays. Further, RL based
 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 13

Figure 7. Comparison between the performance of Q-L and SARSA schemes.

optimum solution is provided which improves spectrum sensing under the shadowing effect. The
reward function depends on reporting time delay i.e., the time duration between CRs request of
cooperation to neighbouring node and arrival to sensing outcome. The authors in (Li, 2009)
proposed Q-Learning to address resource allocation problems in a multi-agent environment.
Koushik (Koushik et al. 2018) discussed RL based spectrum handoff and issues related to RL during
handoff. In this paper, the authors presented hardware implementation of RL and Transfer Learning
(TL) schemes to update its Q-value table greedily and shown that RL takes longer time to converge
than TL which has expert node to transfer its knowledge in form of Q-table.

Combinational NNs
(a) Neuro-Fuzzy NNs

Fuzzy logic is an attractive technique, particularly where it is difficult to express the mathematical
model of real-world problems. In fuzzy Sets, elements are assigned a certain degree of belonging­
ness. The numbers in the range [0,1] are used to represent the degree of belongings. Fuzzy logic
becomes a useful scheme to represent uncertain data. The fuzzy logic controller consists of three
main components as shown in Figure 8.

(i) Fuzzy Interface: Converts real-world values to suitable fuzzy sets.

(ii) Interference Engine: Interference engine uses a knowledge base to decide output for a specific
combination of input fuzzified variables and maps into output variables.
(iii) Defuzzy Interface: Converts output fuzzy sets to real-world values.

Figure 8. Illustration of fuzzy logic in CR networks.

14 A. KAUR AND K. KUMAR

The authors (Salgado et al., 2016) proposed a fuzzy algorithm is proposed for decision making and
particularly for backup channels for spectrum mobility. This proposed algorithm based on multiple
criteria decision-making techniques considers four parameters: availability probability of channel,
estimated channel time availability, the signal to interference plus noise ratio of the channel, and
bandwidth of the channel. Normalised weights (describes a relative degree of importance) are
assigned according to the importance of each criterion for the selection of a backup channel. The
pre-selection of back up channels decreases delay during spectrum mobility.
The authors (Matinmikko et al., 2009) proposed a fuzzy-based cooperative spectrum sensing
technique. In this technique, spectrum sensing decision from the individual node is considered as
input, fuses to form output based on combined sensing result. Spectrum sensing performance has
been characterised based on the probability of false alarm and probability of detection in a fading
environment.
However, fuzzy logic represents uncertain data, but it is difficult to design the degree of belong­
ingness set. So Fuzzy logic combined with neural networks and forms the neuro-fuzzy combinational
model. Neuro-fuzzy based spectrum mobility technique due to its capability to deal with uncertain
environmental conditions as well as in the heterogeneous environment. This technique considers
interference, bit error rate, and signal strength to find the quality of the channel in terms of fuzzy
patterns. Based on generated fuzzy patterns, neural .networks trained to estimate channel gain, not
only for spectrum mobility but also for spectrum assignment in a heterogeneous network
(Maheshwari & Singh, 2015).

b. Wavelets NNs

Wavelet NNs replaces standard activation function (like a sigmoid function) by wavelet activa­
tion function. A wavelet is a small function that grows and decays in a finite amount of time.
Unlike Fourier transform, wavelet NN can distinguish between stationary and non-stationary
signals.
However, wavelet NNs uses wavelet function (called mother wavelet) to derive daughter wavelet
function ψλ;t ðuÞthrough linear translation factor, t and scale/dilation factor, λ as shown in Figure 9.
The daughter wavelet is expressed as
� �
t u
ψλ;t ðuÞ ¼ ψ (4)
λ

An author in (Eltholth, 2016) proposed a spectrum predictor model, which is based on a discrete
wavelet transform nature that produces a time-frequency representation of the analysed signal. The

Figure 9. A daughter wavelet neuron.

 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 15

analysed time series is broken into sub-series, which represents the previous occupancy status of
a certain set of sub-channels. Thus, design an accurate wavelet NNs based spectrum predictor model
with less complexity.
Wavelet approach for wideband spectrum sensing is sub-divided into small bands and charac­
terised by frequency irregularities. Wavelet transform and multiscale wavelet approach have been
used to detect local irregularities that carry information about power spectral density and structure
irregularities of sub-bands that identify spectrum holes (Tian & Giannakis, 2006).

Other NNs
(a) Quantum NNs

Quantum NNs (QNNs) is based on quantum mechanics exploiting quantum information processing
to improve classical ANN. In classical ANN, much of the power is required due to its massively parallel,
distributed information processing and non-linear transformation operation performed by nodes.
On the other hand, QNN makes use of powerful quantum parallelism, which provides quantum
computing to process huge data sets. The author (Liu, 2016) presented QNN based spectrum sensing
algorithm in which quantum neuron with multiple energy levels is considered as the excitation
function of hidden layer neurons. Further, the improved version has been presented to increase
convergence speed and stability. Table 3 summarises DSM with ANN in CR networks.

Intelligence with support vector machine

Support Vector Machine (SVM) is supervised machine learning algorithms used for classification and
regression analysis. It is based on statistical learning theory with structural risk minimisation (Awe
et al., 2013). It is initially preferred for the classification of complex problems. For a given training set,
the input training vector belongs to one or another class of two groups. These two groups are
separated by an optimal hyperplane that has the largest distance to the nearest training set of any
class (maximal margin classifier) that achieves good separation. It is easier to train SVM when the
classes are linearly separable. In non-separable classification, non-linear SVM is obtained by introdu­
cing a kernel function. The function is said to be valid kernel if it satisfies Mercer’s Theorem
(Hofmann, 2006), which provides necessary and sufficient characterisation of a function as a kernel
function. A kernel represents a similarity measure in the form of a similarity matrix (Gram Matrix)
between its input objects. The gram matrix fuses all the necessary information for learning algo­
rithms merged in the form of the inner product. For more details, refer (Burges, 1998). SVM to
develop a real-time approach to sense PU as shown in Figure 10. The input composite signal includes
signal and noise, which is independent of each other in the time domain. The sampled data is
classified as PU or not based on testing and training of the SVM classification model.
For linear classification, a kernel function is proposed to map input low dimensional vector into
high dimensional features space (Zhang & Zhai, 2011). Y. Wang (Wang et al., 2014) proposed
a spectrum mobility prediction scheme that considers time-varying and space varying characteristics
of mobile CRs simultaneously. The joint feature extraction pattern is proposed which executes
spectrum mobility prediction through the classification of SVM with fast convergence speed. The
authors in Huang et al., (2009) presented an SVM based learning engine that is divided into three
parts as environmental awareness, knowledge actuation, and decision making. The learning engine
acquires knowledge about the environment through learning that puts intelligence in CRs. Table 4
summarises DSM with SVM in CR networks.
 16

Table 3. Summarises DSM with ANN in CR networks.

Class Type of NNs Authors Brief Description Application Area Other Related Applications
Supervised ML-FFNNs
NNs
Multilayer Layer Tumuluru et al. ● Multilayer perceptron-based spectrum prediction Spectrum Sensing ML-FFNNs based learning and adaption in CR networks is
NNs (2010) scheme discussed. (Baldo & Zorzi, 2008)(Katidiotis et al., 2010)
Liu (2016) (1) Spectrum Sensing does not require priori knowledge
A. KAUR AND K. KUMAR

of Traffic characteristics of PUs.

(2) Predict channel status based on sensing history
(3) Saves sensing energy and improves spectrum
utilisation
● ML-FFNNs learning based spectrum sensing technique
(1) Dynamic Chinese restaurant game to learn table state
based on ML-FFNN.
(2) CR considered as customers to learn the table state
(Available spectrum hole) in the restaurant (network).
(3) CR also consider table state (spectrum hole) not for
immediate as well as for future utility.
Radial Basis NNs Zhang, Hu et al. ● K-RBF for spectrum sensingK-Means clustering algo­ Spectrum Sensing Multi-objective Optimisation in Cognitive Engine with RBF
(2013) rithms proposed & GA(Y. Yang et al., 2012)
● based on previous information on the PU spectrum.
● More accurately trained as compared to RBF.
Convolutional Lee et al. (2019) ● CNN based cooperative spectrum sensing technique Spectrum Sensing CNN spectrum monitoring framework for radar bands in
NNs (1) Deep neural network (DNN) having multiple hidden spectrum sharing scenarios(Selim et al., 2018)
layers used for learning.
(2) combining individual CRs sensing and optimised com­
bining strategy through DNN.
Recurrent NNs
Deep Recurrent Tang and Li ● DRNN based Spectrum Prediction model Spectrum Sensing RNN and Matrix Methods for Cognitive Radio Spectrum
Neural (2017) (1) Predict the spectrum of multiple time slots using the Prediction and Security(Glandon, 2017).
Network MIMO strategy.
(2) Here, the continuous channel state is divided into
slots and forming time series of channel state.
Unsupervised ML-FFNNs (Auto-encoders)
NNs
(Continued)
Table 3. (Continued).
Class Type of NNs Authors Brief Description Application Area Other Related Applications
Auto-Encoders Zhu and Fujii ● Stacked Denoising Sparse Auto-encoder based modu­ Spectrum Sensing
(2016) lation classification
(1) Proposed modulation classification method to
improve spectrum sensing.
(2) Denoising process improves the performance of noise
suppression by training the network with a corrupted
database.
Recurrent NNs
Hopfield NNs Hasegawa et al. ● Hopfield NNs based optimisation algorithm for deci­ Spectrum Decision
(2014) sion makingTwo approaches are proposed to optimise
centralised and decentralised networks.
● In a centralised network, the selection of base stations
is considered as a minimum cost flow problem that
has to be optimised.
● In a distributed network, energy minimisation
dynamics of Hopfield NNs is considered as an objective
function
Boltzmann Machine
Restricted MohanaPriya ● RBM based secure routing protocol. Spectrum Decision
Boltzmann and Shalinie (1) Provides a secure self-learn routing algorithm.
Machine (2017) (2) Energy-efficient routing to prevent flooding attacks in
routing.
Competitive SOM Khozeimeh and ● SOM based dynamic spectrum management Spectrum Advanced learning using SOM is presented (Tsagkaris
NNs (CR Adhoc Haykin (2012) techniques Management et al., 2012).
C Networks) (1) CRs continuously sense the environment, extract PUs
and neighbouring CRs activity patterns, and stored to
obtain knowledge, which significantly reduces the
probability of collision.
(2) Using the feedback channel, RSA information has
been shared with neighbouring CRs.
(3) After sharing RSA information, weights have been
updated and sub-bands are allotted to CRs based on
its cost function.
Salem et al. ● SOM based spectrum sensing and channel selection Spectrum Sensing
(2016) scheme and Channel
(1) SOM improves network efficiency and spectrum utili­ selection
sation by decreasing the probability of false alarm and
JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE

miss detection
(2) To select the best channel out of available channels
segregates strategy has been applied.
17

(Continued)
 18

Table 3. (Continued).
Class Type of NNs Authors Brief Description Application Area Other Related Applications
Reinforcement – Saleem et al. ● RL based clustering and routing algorithm Spectrum Decision RL for dynamic channel selection in a single-agent and
Learning (2015) (1) RL provides artificial intelligence is applied as a tool to multi-agent environment is discussed (Yau et al., 2010).
form clustering and routing.
(2) A cluster-based routing algorithm is presented which
improves network performance by decreasing flood­
ing of routing overheads.
(3) Further, Spectrum aware Cluster-based routing is pre­
sented to overcome the challenges of multihop
routing.
A. KAUR AND K. KUMAR

- Lo and Akyildiz ● RL based cooperative spectrum sensing model Spectrum Sensing Q-Learning based Multi relay cooperative mechanism
(2010) (1) Cooperative spectrum sensing with Q-Learning is sensor network is presented (Peng et al., 2011).
considered.
(2) Scheme converges to Optimal solution under corre­
lated shadowing and adaptable to environmental
changes.
(3) The reward function depends upon reporting delay
time.
- Li (2009) ● Q-Learning based channel selection in a multi-agent Spectrum Sharing Q-Learning based power allocation schemes are discussed
environment. (Wu & Fei, 2010).
(1) Q-Learning to address the resource allocation
problem.
(2) Multi-agent RL with a Q-Learning framework is con­
sidered for channel selection.
(3) No coordination among CRs and each CR treated
other CR as part of the environment.
- Koushik et al. ● Hardware testbed learning based spectrum handoff Spectrum Handoff
(2018) (1) RL (Q-Learning) and Transfer Learning scheme-based
spectrum handoff is presented to update the Q-value
of the table.
(2) RL takes a longer time to provide an optimal solution
(update Q-value) than the TL scheme.
(3) RL each time restarts its learning process from scratch
even if a similar kind of situation prevails before.
Combinational Neuro-Fuzzy Maheshwari and ● Neuro-fuzzy based Spectrum Handoff technique Spectrum Mobility
NNs NNs Singh (2015) (1) This technique considers interference, Bit Error Rate
(BER), and signal strength to find the quality of chan­
nel in terms of fuzzy patterns.
(2) Based on generated fuzzy patterns, the neural net­
work is trained to estimate channel gain and select
the available spectrum in the heterogeneous network.
(Continued)
Table 3. (Continued).
Class Type of NNs Authors Brief Description Application Area Other Related Applications
Fuzzy Logic Matinmikko ● Fuzzy logic based cooperative spectrum sensing Spectrum Sensing Fuzzy logic based cross-layer optimisation is presented
et al. (2009) technique Spectrum (Baldo & Zorzi, 2008)
Salgado et al. (1) Spectrum sensing decision from an individual node is Mobility
(2016) considered as input cooperatively, fuses to form out­
put based on combined sensing result.
● Fuzzy based spectrum mobility algorithm
(1) proposed algorithm for multiple criteria decision mak­
ing and particularly for backup channels for spectrum
mobility.
(2) multiple criteria decision-making techniques consider
four parameters: availability probability of channel
(AP), estimated channel time availability (ETA), and
the Signal to interference plus noise ratio of the
channel (SINR) and bandwidth of the channel (BW).
(3) Normalised weights are assigned according to the
importance of each criterion for the selection of the
backup channel.
Wavelet NNs Eltholth (2016) ● Wavelet NN based Spectrum prediction Model Spectrum Sensing
Tian and (1) Analysed signal has been represented in the time- Spectrum
Giannakis frequency domain. Sensing
(2006) (2) Spectrum predictor model is based on the discrete
wavelet transform (DWT) nature
(3) The analysed time series is broken into sub-series,
which represents previous occupancy statutes of
a certain set of sub-channels.
(4) NN’s performance has been enhanced by coupling it
with Wavelet NNs.
● Wideband spectrum sensing using wavelet approach
(1) Wide frequency bands are decomposed into small
frequency bands that are characterised by local irre­
gularities in frequency.
(2) Wavelet transform modulus and multiscale wavelet
approaches are used for spectrum sensing.
Other NNs Quantum NNs Liu (2016) ● QNN based spectrum sensing algorithm Spectrum Sensing
(1) quantum neuron with multiple energy levels is con­
sidered as the excitation function of hidden layer
JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE

neurons.
(2) Energy is considered as input characteristics.
(3) improves convergence speed and convergence
stability.
19
20 A. KAUR AND K. KUMAR

Intelligence with metaheuristic algorithms

a. Genetic algorithm (GA)
GAs are stochastic search algorithms, which effectively solve machine learning and image processing
problems (Jedlicka & Ryba, 2016). The main advantage of using GA is parallelism, which speeds up its
simulation results. The computation in GA starts with the collection of chromosomes, which has
certain characteristics. The algorithm can be implemented using the following steps as shown in
Figure 11 (Siddique & Azam, 2010).

(1) Initialise the population of chromosomes.

(2) Evaluate the fitness level of each chromosome to rank them.
(3) Construction of the new population until the production of next-generation completes using
the following steps:
(i) Selection: The best chromosome is selected from the currently available population
based on its fitness level.
(ii) Crossover: With crossover probability, selected chromosome reproduces to generate
a new individual.
(iii) Mutation: Newly generated individuals will be mutated at a definite point in the
chromosome.
(4) The above steps are repeated until the desired results are obtained.

In CR networks, the GA uses biological behaviour representing each channel by a chromosome. Each
gene of the chromosome represents specific parameters. In the spectrum decision process, it is all about
the fair distribution and utilisation of available resources, and for fair resource distribution, optimal
distribution is required. According to L.Doyle, ‘An optimization process can be defined as the process
involved in selecting the best choice from the list of available choices to reach some kind of goal or at
least as near as possible to goal’ (Doyle, 2009). In CR networks, spectrum allocation also involves an
optimisation task to assign spectrum holes to CR found during the spectrum sensing process.
The spectrum allocation scheme for CR networks using GA in which each gene of chromosome
represents different parameters (data rate, frequency, bandwidth, error rate, and modulation/coding
scheme) which is associated with specific weight (Siddique & Azam, 2010). The weight for each gene
represented in binary form. Like the weight for each gene in the chromosome fitness point for each
gene is assigned. Once the fitness of each gene in the chromosome is calculated, the next step is the
construction of a new population, which involves selection, crossover, and mutation process. The
whole process is repeated until the optimum solution among the available solution set is achieved.
The authors in (Morabit et al., 2015) presented spectrum decisions based on GA to provide a new
spectrum band as requested by CR in the network. GA defines radio in terms of genes and
chromosomes and considers user quality of service (QoS) as an input to the GA procedure.
Further, chromosome population size is defined by available spectrum resource size and chromo­
some genes define the efficiency of spectrum allocation.

b. Ant colony optimisation (ACO)

ACO considered as computational technique proposed by Dorigo which is based on the searching
behaviour of Ants (Dorigo et al. 1997). Although real arts are blind, the search their food source via
the shortest path by play on the information through a substance called pheromones. The ants on its
transmission route release this liquid substance, which is accumulated on the shortest route and
soon ants, start to follow the smallest route. The behaviour of real ant has inspired the ant system,
which resolves many complex problems successfully. The characteristics of the ACO algorithm i.e.
parallel computation, self-organisation, and positive feedback that can help CR to achieve self-
adaption and learning capability to achieve global optimisation (Q. He et al., 2013) (Bayrakdar &
Çalhan, 2018). Different steps in the ACO algorithm are as follows:
 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 21

Figure 10. A diagram showing the basic idea of SVM for spectrum sensing.

(1) Ant probability distribution rule: In the ant system, artificial ants release the solution through
moving among vertex follow the principle as:

8
>
< ½τ ðtÞ�α ½η ðtÞ�β
ij ij
pkij ðtÞ ¼ P α β
if j 2 C (5)
>
: ½τij ðtÞ� ½ηij ðtÞ�
s2C

where pkij ðtÞ is state transition probability of ant moving from vertex i to j at the time t. C is a set of
vertex and s is unselected vertex in the traversing process. τij ðtÞ and ηij ðtÞ are pheromone intensity

Table 4. Summarises DSM with SVM in CR networks.

Type
of Application
SVM Authors Brief Description Area Other Related Applications
- Dandan (2011) ● SVM based spectrum sensing Spectrum Eigen vector and support vector
Awe and technique Sensing machine based learning approach for
Lambotharan (1) Presented SVM to develop real time spectrum sensing in multi antenna CR
(2015) approach to sense PU. networks is proposed (Awe et al.,
(2) The sampled data is classified as PU or 2013)
not based on testing and training of
SVM classification model.
(3) For linear classification, kernel func­
tion is proposed to map input low
dimensional vector into high dimen­
sional features space.
.
● Formulated multi-class SVM
Algorithms for solving multi-class
spectrum sensing problems.
(1) The performance of the detector has
been judged based on receiver oper­
ating characteristics curves and classi­
fication accuracy.
(2) Robust to joint spatiotemporal detec­
tion of spectrum hole.
- Wang et al. ● SVM based spectrum mobility predic­ Spectrum
(2014) tion scheme Mobility
(1) Considers time-varying and space
varying characteristics of mobile CRs
simultaneously.
(2) The joint feature vector extraction pat­
tern is proposed which executes
spectrum mobility prediction through
the classification of SVM with fast
convergence speed.
22 A. KAUR AND K. KUMAR

Figure 11. Illustration of genetic algorithm for CR networks.

and heuristic visibility (cost) of a direct route between the node i and j. The parameters α and β
controls the importance of pheromone and heuristic information respectively.

2. Local optimisation procedure: To avoid residual pheromone submerging heuristic information,

pheromone will keep updated after one ant finishes a path searching with follows the principle
as:

τij ðt þ 1Þ ¼ ð1 ρÞ:τij ðtÞ þ ρ:τ0 (6)

where τ0 is the initial pheromone value and ρis the evaporation coefficient of pheromone.

3. Global optimisation procedure: When all ants complete the cycle, the pheromones will be
globally updated as:

1
τij ð1 αÞ:τij þ α: (7)
Gmax

where α is constant and Gmax is the maximum objective function.

4. Stopping procedure

The procedure is stopped after completing a predetermined number of cycles to achieve the globally
best solution.
The author in (Jhajj, 2017) discussed the ACO technique to find the optimal sensing time of CR. As
CR has a fixed time frame for its sensing the environment and transmission of data, an optimal
sensing time is required that will maximise throughput with minimum interference with PUs. This
optimal sensing time can be found by ACO which implements the above steps to find an optimised
solution. The ACO has been proposed for reconfiguration decision making in CR networks (Q. He
et al., 2013). Reconfiguration means configure terminals and network devices to intelligently adapt
to environmental conditions, which is a challenging task. ACO deals with complex environmental
conditions as well as target multi-objective optimisation through parallel computation. Moreover,
ACO learns from the environment and reconfigure spectrum decision through self-organisation
process. Three types of reconfiguration schemes have been considered i.e. parameter variation
 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 23

reconfiguration scheme, radio resource management reconfiguration scheme and network access
reconfiguration scheme and showed the improved performance in the heterogeneous network.
A dynamic channel assignment as an optimisation problem to maximise the reward of CRs is
formulated in (He & Zhang, 2012). The algorithm also considers interference with PUs and SINR
constraints of CRs to maximise the spectrum utilisation. Table 5 summarises DSM with metaheuristic
algorithms in CR networks.

Intelligence with Bayesian learning

Bayesian learning signifies the importance of prior distribution which represents knowledge of
unknown parameters in uncertain environmental conditions. It is assumed that no prior information
is available about the spectrum but some prior information is available about the status of the
channel which should be explored through learning. Bayesian inference is an approach where Bayes’
rule is used for driving posterior distribution from the prior distribution which further considered as
prior for another posterior distribution and so on. In CR networks, CRs can compute the prior
distribution of a system parameterθ, denoted by PðθÞ, which represents the spectrum occupancy
status of PUs. Though spectrum sensing, data observed in n time slots X ¼ ½x1 ; x2 ; ::::::::xn �. The
likelihood functions of the parameter θ,PðX=θÞ, for observed data, conditional on parameter θ. After
acquiring prior probability distribution and the likelihood function, Bayesian inference can be used
to derive posterior density function, Pðθ=XÞwhich is conditioned on data X. Bayes’ Theorem can be
simply expressed as
Pðθ=XÞ ¼ PðX=θÞ:PðθÞ=PðXÞ (8)
In CR networks, Likelihood function denotes the probability of busy/idle previous states for a busy/
idle current state. In Baye’s theorem, the posterior distribution combines the information in both the
prior distribution function and the likelihood function, which represents stronger information and
narrower possible values for θ. The Bayesian network has 2 types of models, Bayesian Parametric (BP)
Model, and Bayesian Non-Parametric (BNP) Model. BP model has a finite set of parameters θ to
compute prior distribution and In the BNP model, it does not mean a lack of parameters but an
infinite-dimensional set of parameters is assumed to compute prior distribution. Recently BNP model
applications in CR networks get a lot of attention of researchers because it provides flexibility to
model to learn in an unknown environment.
A Bayesian approach based spectrum sensing in CR networks is presented in (Jacob et al., 2014).
The authors presented Bayesian inferences over single and multiple sensing frames to calculate the
posterior distributions of unknown parameters, which forms the basis of Bayesian learning (Manco-
vásquez et al., 2014). In Bayesian inference over multiple frames, learning from past samples has
been done using threshold-based approximation and Kullback Leibler based approximation. Further,
A Bayesian approach based decision-making engine in CR networks is proposed in (Huang et al.,
2010). CE learns from an environment and form rules to reconfigure transmission parameters
(modulation, code rate) which ensures QoS of CRs. The reconfiguration can be achieved with
parameter learning, structure learning, and inference algorithms that form rules to adapt to envir­
onmental conditions . Table 6 summarises DSM with a Bayesian approach in CR networks.

Intelligence with Hidden Markov Model

Hidden Markov Model (HMM) represented as a stochastic process that can be modelled as a Markov
chain, whose actual states are hidden, to analyse the temporal or dynamic behaviour of PU activity
pattern. In HMMs, several hidden states represent the probability of distribution over a sequence of
observations. The HMM gets its name from two properties in which it assumes:
1) Observations at the time tgenerated by some process whose state St is hidden from the
observer.
24 A. KAUR AND K. KUMAR

2) State of hidden process St satisfy Markov’s property which states that for a given St 1 , the
current state St is independent of all states before ðt 1Þi.e. For a given St , the output Yt is
independent of all other time indices states and observations.
HMM has found remarkable applications in CR networks since many environmental parameters
are partially observed or act as hidden states. Various authors proposed HMM-based spectrum
sensing schemes (Saad et al.,2016), (Ghosh et al. ,2009). HMM-based spectrum prediction for
industrial applications that accurately predict through multiple slots is proposed in (Saad et
al.,2016). Large numbers of hidden layers are considered which interpret as different PUs activity
levels and formulate the prediction problem as a maximum likelihood classification approach. Akbar
et al., 2007 presented a Markov based channel prediction algorithm and considers channel state
occupancy of PUs as Poisson distributed. Pham et al. (2014) presented the HMM based spectrum
handoff model in CR networks. The proposed approach infers the efficiency of HMM in correcting the
sensing sequence and prediction of channel status. Further, the proposed algorithm applied to the
spectrum mobility function in CR networks. An HMM based channel selection framework that
minimizes the delay incurred during channel selection is proposed in (Senthilkumar & Geetha
Priya, 2016). The proposed approach achieves a significant reduction in data loss and an increase
in transmission speed, by obtaining the best-matched channel for the users. Table 7 summarises
DSM with HMM in CR networks.

Intelligence with game theory

Game theory technique accounts for multi-agent decision making, in which the decision of each
player for action is based on the history of actions performed by other players. This involves the
learning process by each player, which may ultimately lead to a stable state. Every game involves
a set of players, actions, and payoff functions (or utility function). A player gets more rewards if it has
a higher payoff function. Mathematically, a game can be represented as:
G ¼ ðN; A; PÞ (9)
where N represents a set of players, A represents a set of actions, and P represents payoff
functions. In CR networks, the players in the games are CRs, which take actions based on observa­
tions of its environment in which it operates. As time progresses, CR learns from past actions and
from observing the action of other CRs and modify its actions accordingly. Several types of game
theory approach in CR networks are shown in Figure 12.

● Game-Theoretic Approaches:

Two types of game-theoretic approaches are cooperative and non-cooperative games. In coopera­
tive games, a group of players can make binding commitments and act jointly whereas, in non-
cooperative games, the individual player acts selfishly. In cooperative games, cooperation among
the players strengthens the position of the player in a game. Here the group of players forms
a coalition act as a single entity.
In non-cooperative, each user takes care of its benefit and selects optimal actions to maximise its
payoff function. The author in (Yu, 2013) presented cooperative games for spectrum sensing and
sharing and found that the applicability of various games depends on several factors. A non-
cooperative game can be classified based on information i.e. either complete information or an
incomplete information game. In the complete information games, each player observes other
players’ information i.e. payoff and their action. On the other hand, with incomplete information,
the game can be modelled as Bayesian game for decision making in which outcome can be
estimated based on Bayesian analysis (Wang et al., 2010). Several types of games have been adapted
to model various situations in CR networks. For example, in repeated games, each stage is usually
repeated. Let us considerN the set of CRs in the network and aðkÞ denotes the action taken by a player
Table 5. Summarises DSM with a metaheuristic algorithm in CR networks.
Type of Metaheuristic
Algorithms Authors Brief Summary Application Area Other Related Applications
Genetic Algorithm El Morabit et al. (2015) ● The genetic algorithm-based spectrum allocation scheme Spectrum Decision Importance of Genetic Algorithms in
(1) Defines the structure of chromosomes which is a set of genes. Machine learning (Shapiro, 2001)
(2) Each gene represents frequency, modulation, power, and BER
in particular.
(3) Each gene will be considered in the decision-making process.
(4) Provides a multi-objective optimisation technique to find
fitness function using a weighted sum approach (rank asso­
ciated with each objective) represents the importance of each
objective.
Siddique and Azam ● Genetic algorithm-based Spectrum optimisation techniques Spectrum Decision
(2010) (1) Chromosome population size is defined by available spectrum
resource size and genes define the efficiency of spectrum
allocation.
(2) Each gene of the chromosome represents a different para­
meter, which is associated with a specific weight.
(3) The weight for each gene represented in binary form i.e.,
fitness point for each gene is assigned.
(4) Once the fitness of each gene in the chromosome is calcu­
lated, the next step is the construction of a new population,
which involves selection, crossover, and mutation process.
Ant Colony Optimisation Jhajj (2017) ● ACO for Spectrum sensing Spectrum Sensing Self-Organisation paradigms and
(1) Optimal sensing time is calculated using ACO that will max­ optimisation approaches for
He et al. (2013) imise throughput with minimum interference with PUs. cognitive radio technologies:
● ACO based reconfiguration Decision-making process. Spectrum Decision A survey is provided (Liu et al.
(1) ACO deals with complex environmental conditions as well as 2013)
target multi-objective optimisation through parallel
computation.
(2) ACO learns from the environment and reconfigures spectrum
decision through a self-organisation process
(3) Three types of reconfiguration schemes have been considered
i.e. parameter variation, radio resource management, and
He and Zhang (2012) network access reconfiguration scheme.
● Dynamic channel assignment using ACO Spectrum Decision
(1) Formulating a dynamic channel assignment problem as an
JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE

optimisation problem and provide solution-based on ACO.

(2) Manage and assign channel resources dynamically while con­
P. Huang et al. (2013) sidering interference and SINR constraints.
● ACO based on demand Routing Algorithm Spectrum Decision
25

(1) M/G/1 queuing theory is introduced to characterise the spec­

trum usage behaviour of PUs and CRs.
(2) It jointly considers path and spectrum scheduling to improve
latency and packet loss rate.
 26

Table 6. Presents DSM with Bayesian approach in CR networks.

Type of Bayesian
Learning Authors Brief Summary Application Area Other Related Applications
- Jacob et al. (2014) ● A bayesian approach-based Spectrum Prediction Spectrum Sensing Service selection for CR based on the Bayesian
modelPredicts the spectrum occupancy status of approach has been presented (Homayounvala, 2015)
PUs from its previous spectrum occupancy pattern Non -Stationary Hidden Markov Model with Bayesian
using a Bayesian approach inference for channel quality prediction presented
A. KAUR AND K. KUMAR

● Status of N previous time slots are considered to (Xing et al., 2013).

predict the next stage. BNP model base Dirichlet process mixture model
- Manco-vásquez et al. (2014) ● Bayesian approach for multiantenna spectrum Spectrum Sensing (DPMM) for unsupervised classification technique
sensingBayesian detectors are proposed which has been applied (Ranjan & Ahmad, 2016).
learn from past decisions and to adapt to Dynamic Bayesian games for decision making in Ad-
a continuously changing environment. Hoc networks are presented (Nurmi, 2004).
● At each sensing period, Bayesian inference is applied
and posterior for channel occupancy is calculated
which act as priors for the next sensing frame
● Perform Bayesian inference over single and multiple
time frames to find the posterior distribution of
unknown parameters.
- Huang et al. (2010) ● A Bayesian learning-based Decision engineBayesian Spectrum Decision
network-based CR learning inference and decision-
making engine
● Form rule according to the variations in the environ­
ment (i.e. SNR) with learning which include structure
and parameter learning.
● Utilise these to reconfigure transmission parameters
(modulation, code rate) according to environmental
- S. Zheng et al. (2013) ● Bayesian approach-based spectrum sensing for digi­ Spectrum Sensing
tal signalsBayesian detector obtains prior distribu­
tion by considering PU activity and signalling
information of PU (symbol rate, modulation) to
improve CR throughput.
● An optimal detector for digital primary signals(MPSK,
M > 2) over the AWGN channel for low and high SNR
regime has been presented.
 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 27

at kth the stage of the game. In each stage k, players tend to maximise their payoff function, while
considering the history of action collected in hðkÞ ¼ ðað1Þ ; að2Þ ; ::::::::; aðkÞ Þ. In other words, players
ðkÞ
map their actions from history ai ¼ fi ðhðk 1Þ Þ. In CR networks, their action is the selection of channel
available and mapping of action depends on the history of PUs activity as well as activity pattern of
other CRs. In dynamic/repeated games, players came across a similar game number of times. The
cooperation among the players in repeated games can be introduced to get long term benefits. The
repeated games are applied for spectrum sharing scenarios where multiple CRs exists in (Li et al.,
2010). In this context, repeated games apply punishments to achieve a desirable outcome. When the
PUs activity is considered as stationary with an unknown environment then Stochastic games, also
known as Markov games, are introduced to model the network. The authors in (W. Wang et al., 2018)
addressed the routing problem using stochastic games. Stochastic routing game is decomposed into
several stages and at each stage stochastic learning is proposed to learn equilibrium strategy of
channel selection.
The Stackelberg game can be modelled for implementing spectrum sharing where PUs can
involve CRs as cooperative relays (H. Wang et al., 2010). In auction games, PUs act as auctioneer,
selling idle spectrum bands to CRs which are allowed to select the appropriate bidding strategy for
each channel to maximise their utility function. The concept of auction games has been applied to

Table 7. Presents DSM with HMM in CR networks.

Type of
HMM Authors Brief Summary Application Area Other Related Applications
- Saad et al. (2016) ● HMM-based Spectrum Prediction for Spectrum Dynamic spectrum
industrial applications.Addressing the Decision allocation using HMM is
prediction problem as a sequence of presented (Akbar &
the classification problem. Tranter, 2007).
● Large numbers of hidden layers which Channel status predictor
formulate the problem as maximum using HMM and/or
likelihood (ML) classification. Multilayer perceptron
- Ghosh et al. ● HMM in Spectrum SensingHMM pre­ Spectrum Sensing (MLP) (Tumuluru et al.,
(2009) dicts a busy or idle status of sub-band 2012).
by its PU
● Validate the existence of Markov chain
by collecting real-time measurements
in paging spectrum (928–948 MHz)
● Likelihood method to determine true
states, the complexity arises due to
this method is then reduced using
Viterbi coding
- Tran and Do ● The HMM-based Spectrum Mobility Spectrum
(2014) modelThe proposed approach infers Mobility
the efficiency of HMM in correcting
the sensing sequence and prediction
of channel status.
● Further, the proposed algorithm
applied to the spectrum mobility
function in CR Networks.
- Senthilkumar and ● Channel selection framework based Spectrum
Geetha Priya on HMM.Minimised channel selection Sensing,
(2016) delay channel
● Further enhanced its performance allocation
with an optimum routing algorithm
along with HMM.
● The proposed approach achieves
a significant reduction in data loss
and an increase in transmission speed,
by obtaining the best-matched chan­
nel for the users.
28 A. KAUR AND K. KUMAR

spectrum sharing in CR networks (Khaledi & Abouzeid, 2013), as well as in spectrum sensing (Sendrei
et al., 2015). Table 8 summarises DSM with game theory in CR networks.

Intelligent techniques evaluation: strength, limitations, and challenges

Intelligent techniques having many advantages but their implementation still facing some chal­
lenges. The intelligent techniques applications in CR networks, strengths, limitations, and imple­
mentation challenges are summarised in Table 9.

(a) Supervised NNs: Basically, supervised NNs accomplished its task under supervision which
requires complete knowledge about the environment in which it has to operate. However, in
the context of CR networks, CRs may not have complete knowledge about the radio environ­
ment in which it has to access the spectrum. NNs provides a high level of feature classification
but also required data labelling. It requires training under different environmental conditions
and outcome depends on the selection of initial parameters. Therefore, the selection of data
for training must be task-oriented and free from noise. Also, with the increase of network size,
the training process slows down which may lead to slower convergence. To improve its
efficiency, multiple hidden layers are introduced which requires large training data which
further slows down its training process.
(b) Unsupervised NNs: A major challenge CRs can face a lack of knowledge of the surrounding
radio environment. Even in this situation, CRs are expected to adopt changes in the environ­
ment so that they may not collide with PUs. CRs must be able to extract knowledge about PU
activities which makes unsupervised NNs an appealing approach in CR networks. In unsu­
pervised NNs, learning is based on a correlation among input data, and no information about
the correct output is available. So unsupervised NNs are used indirectly for spectrum

Game Components Game Type Players Strategy Action taken

o Group of players make bidding

commitment and take action jointly.
Cooperative o Players forms coalition act as single
Games entity.

o Individual player selects optimal action

Non- to maximize its payoff function.
Cooperative
Games

o Each stage is usually repeated and in

Users & Selection of
each stage players tends to maximize
Repeated History of PUs its payoff function.
and CRs activity channel
networks Games o Players maps its actions from history of
PUs and CRs activity.

PU Transmission
o PU select proper set of CRs which act
Stackelberg as cooperative relays.
Games o In return leases portion of access time
PU selected CRs to act as to CRs for its transmission.
cooperative relays
CR Transmission
o PU act as an auctioneer and sells idle
Auction spectrum band to CRs
Games o CRs has to select appropriate bidding
strategy for each spectrum band to
Bidding cost,
Bidder/Seller Tenderer/Buyer maximize its utility function.
payoff

Figure 12. Various game theory approaches in CR networks.

 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 29

management in CR networks like modulation classification, optimisation algorithm for spec­

trum decision, and routing algorithms.
(c) Unsupervised Competitive NNs: As the name indicates, it is a form of unsupervised learning
where output and input neuron compete and employs winner take all strategy. SOM is one of
the members of this class that make use of competitive learning in which neurons compete
with each other to win and winner neuron displaced to feature space to form clusters. The
authors in (Khozeimeh & Haykin, 2012) presented SOM based DSM for cognitive ad-hoc
networks. In this proposed scheme, PU and CR activities patterns are extracted and stored
in memory which significantly reduces the probability of collision. SOM seems to be more
realistic as it does not require prior knowledge about the radio environment. Further, the
application of SOM with a different type of input pattern does not require redesigning the
existing learning mechanism. SOM can be easily interpreted and understood. Moreover, it
requires sufficient data to form meaningful clusters. Another member of this category is ART
which designed for object identification, recognition, and pattern classification problem. It
has stable and self-regulating control structure but it does not always guarantee stability and
it has to empirically fix its vigilance parameter. This technique is not suitable directly for DSM
in CR networks.
(d) Reinforcement Learning: Reinforcement Learning is distinct from supervised and unsuper­
vised learning as it focuses on online learning rather than offline learning. RL learns from
positive and negative reinforcement (or rewards) to set its training examples. RL seems to be
one of the promising intelligent techniques to solve the spectrum management problem in
CR networks. RL is used in spectrum sensing, spectrum decision (in the multi-agent environ­
ment and with multiple objectives), spectrum sharing, and spectrum mobility. One of the
major drawbacks of the RL technique is slow convergence. The authors presented hardware
implementation of RL and Transfer learning for spectrum handoff in (Koushik et al., 2018) and
found that RL takes a long time to adjust its learning parameters as compare to Transfer
learning
(e) Combinational NNs: Combinational NNs have neuro-fuzzy NNs and wavelet NNs in this
category. Fuzzy logic is an attractive technique that uses human-understandable fuzzy logic
instead of using complicated mathematical modelling. The main advantage of fuzzy logic is
that it is faster to train and also it requires less computational resources. With fuzzy logic, the
solution can be obtained when the system has incomplete environmental knowledge. Some
disadvantages of fuzzy logic are difficult to create its model as with increasing dimensions
may lead to inefficient results. Further, it is difficult to estimate its membership function
according to user requirement and it requires more tuning.
(f) Support Vector Machine: SVM comes under the supervised learning category which is used
for object classification, pattern recognition, and regression analysis problem. SVM technique
provides superior performance in many applications due to its generalisation ability and
robustness against noise. SVM maps input vector from low dimensional feature to high
dimensional features which make them linearly separable. For non-linear mapping, a kernel
function is used. In CR networks, SVM is used as a classifier for signal classification. The
sampled data is classified as PU or not based on the training of SVM. Unlike ANN, it is not
suffered from overfitting problem and provides good performance in small problems. But in
complex problems, it provides poor performance and may require a high training set which
increases its computation complexity and storage requirements. Also, SVM requires large
labelled data for training and complete knowledge of the radio environment which may
divert the focus of researchers.
(g) Metaheuristic Algorithms: These algorithms solve multi-objective optimisation problems
through parallel computation. GA provides multi-objective optimisation based on a fitness
function but it may not always converge to the global optimum in case of large performance
metrics. The performance of GA highly depends on fitness function which may be based on
 30

Table 8. Presents DSM with game theory in CR networks.

A. KAUR AND K. KUMAR

Type of Game Authors Brief Summary Application Area Other Related Applications
Cooperative Yu (2013) ● Cooperative games for spectrum sensing and sharingDifferent game models in Spectrum Sensing
Games a cooperative framework are presented. & Spectrum
● Combinations of different approaches such as price-based, market-driven, agent Sharing
driven, incentive-driven are also explored.
Non-Cooperative Zhu et al. ● Non-cooperative game (Bayesian game) for spectrum decisionThe game can be Spectrum Decision Spectrum sharing based on the non-cooperative
games (2012) modelled as a Bayesian game of incomplete information. game (Bing. et al., 2010)
● Statistical learning techniques and a radio environment map can be used to map
uncertainties using Baye’s rule.
Repeated Games B. Wang et al. ● Self-learning Repeated game-based spectrum AccessSelfish behaviour due to non- Spectrum Sharing Stochastic game to address routing problem
(2007) cooperation among CRs are considered which are proved to be inefficient. (W. Wang et al., 2018).
● Modelling spectrum access as a self-learning process. Inspection based repeated spectrum sensing
and sharing schemes (Kim, 2017)
Stackelberg H. Wang et al. ● PU selects a proper set of CRs to serve as cooperative relays for its transmission and Spectrum Sharing Stackelberg games based relay selection for
Games (2010) in return leases portion of channel access time to CRs for its transmission physical layer security
(Fang et al., 2017)
Auction Games Khaledi and ● Spectrum sharing mechanism, which considers channels with different qualities Spectrum Sharing Spectrum allocation based on Auction
Abouzeid and CRs to express their preferences for each channel separately in the form of framework with no regret learning (Kasbekar &
(2013) vector bids. Sarkar, 2010).
Sendrei et al. ● Represents sealed bid multiple units sequential spectrum action to make a tradeoff Spectrum Sensing
(2015) between spectrum sensing time, risk, the revenue of PU, and payoffs of CRs.
Presents the evaluation of various intelligent techniques along with their strengths and limitations.
Intelligent Spectrum Spectrum Spectrum Spectrum
Class Technique Sensing Decision Sharing Mobility Strengths Limitations and Challenges
Supervised ML-FFNNs ● High-level feature classification. ● Multiple Hidden layers
NNs Radial Basis NNs ● Adaption ability to minor changes. ● Require different training Algorithms
Convolutional NNs ● Slow Convergence
Recurrent NNs
Deep Recurrent
NNs
Unsupervised ML-FFNNs ● No prior knowledge required ● Slow training
NNs (Auto- Encoders) xxxx ● Good performance in image classi­ ● Lack of theoretical justification.
fication problems
Recurrent NNs
Hopfield NNs ● Used in optimisation problems. ● Dimensionality reduction
● Simple implementation. ● Identify similarities in input data, which causes error.
Boltzmann Machine ● Computationally efficient ● Crude approximation
● Better in ignoring random data
● Fast Algorithms
Competitive SOM/Kohenon ● Easily interpreted and understood ● Requires sufficient data to form meaningful clusters.
NNs Maps ● Capable to handle several types of
classification problems
Adaptive ● Stable and fast learning ● Does not guarantee stability
Resonance ● Self-regulating control structure ● Need to empirically fix vigilance parameter
Theory (ART) ● Requires no supervisory control
ART-1 ● Suitable for online learning
ART-2 problems.
Reinforcement ● Guaranteed to be optimal for data ● Slow convergence
Learning & NNs optimisation framework
● Overall behaviour sensitive to
changes in reward
Combinational Neuro-Fuzzy NNs ● Faster to train ● Difficult to create a model
NNs Fuzzy Logic ● Robust to disturbances ● Membership function estimation is hard.
Wavelet NNs ● Less computational resources ● Need more fine-tuning
required
Other NNs Quantum NNs ● Easy to train due to quantum ● Time and space complexity
parallelism ● Quantum associative memory has exponential gain in
● Faster processing as compare to storage capacity as compared to classical ANN
JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE

classical ANN
(Continued)
31
 32

(Continued).
Intelligent Spectrum Spectrum Spectrum Spectrum
Class Technique Sensing Decision Sharing Mobility Strengths Limitations and Challenges
Support Vector ● Overfitting is not common. ● Long Learning period
A. KAUR AND K. KUMAR

Machine ● Not trapped in local minima due to ● Difficult to understand learned weights
convex optimisation
● Fewer parameters and training
samples need to be considered
Metaheuristic Genetic Algorithm ● Intrinsically parallel ● Slow process
Algorithms ● Chances to get an optimal solution ● Not always provide an optimum solution
is more ● Require fitness function
● Easy to understand
● Better convergence with increasing
genes
Ant Colony ● Inherent parallelism ● Slower convergence
Optimisation ● Adaptability
Bayesian Approach ● More accurate decision ● Requires prior information of the system
● Probabilistic model ● High computational complexity
● An incorrect assumption about prior skew to skew
inferences
Hidden Markov ● Efficient learning algorithm ● Cannot express dependencies between hidden states
Model ● Strong statistical model ● Difficult to decode.
● The genetic Algorithm improves ● Training computationally complex.
model training efficiency.
Game Theory ● Provides a solution for multi-agent ● An infinite number of strategies.
system ● A finite number of players
● Reduces complexity of adaption ● Risk and uncertainty
● Not always converges
 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 33

prior knowledge. Wrong prior knowledge may lead to the generation of bad chromosomes.
Thus, the selection of better genes (each gene represents bandwidth, modulation scheme,
data rate, etc.) to generate the next genes are very critical. Further, GA is considered a very
slow process since fitness function is calculated through selection, crossover, and mutation
process. GA improves its convergence speed either by combining with RBNNs or by increas­
ing the number of genes. In ACO, parallel computation, self-organisation, and positive feed­
back can help CR to achieve self-adaption and learning capability to achieve global
optimisation. ACO algorithm involves local optimum value based on which global optimum
value is obtained. ACO can easily adapt environmental changes but its performance is poor
for local searches.
(h) Bayesian Approach: Bayesian approach relies on a probabilistic model which signifies the
importance of prior distribution to derive posterior distribution using Baye’s theorem. The
Bayesian approach requires prior knowledge about the radio environment. Incorrect informa­
tion may lead to skew inferences. It has high computational complexity due to high dimen­
sional integrals. Bayesian learning can combine with other techniques such as HMM for
channel quality prediction in CR networks. Dynamic Bayesian games are used for decision
making. Further, the Non-Parametric Bayesian model-based Dirichlet process mixture model
for unsupervised classification techniques can be used.
(i) Hidden Markov Model: HMM is a stochastic model based on the Markov model which is highly
relevant to the CR network applications as environmental parameters that are partially
observed and act as hidden states. The selection of appropriate models for training is
a very important task in HMM. Due to the presence of multiple hidden layers, it is difficult
to decode the sequence. HMMs have been extensively used in CR networks for spectrum
sensing, channel selection, and spectrum mobility. One of the major drawbacks while using
HMM is that it requires a training sequence. But its training process is quite complex. It can be
combined with other techniques such as GA to improve its training efficiency.
(j) Game Theory: Game Theory is a mathematical model which provides a solution for self-
centred multi-agent systems where the decision of individual agent affects other players
decision. Particularly in CR networks, each CR act as a player and their action may include the
selection of parameters according to user requirements. The goal of game theory is to provide
the best outcome (optimal solution) while considering the interest of all players. Game-theory
has been applied in several applications in CR networks such as spectrum sensing, decision
making, spectrum access, etc. One of the major drawback while using game theory is to make
a model which requires statistical information about the radio environment. As the environ­
ment is dynamic, which leads to the shift optimal solution before the convergence. It is
difficult to structure a game that always provides an optimal solution. Another drawback is
a limited number of players. As players increases, it may decrease its convergence speed
which is another important factor need to be considered. Further, the game theory requires
complete knowledge of the environment and also need labelled data for training. Incomplete
or imperfect information can lead to uncertainty. Table 9 presents the evaluation of various
intelligent techniques along with their strengths and limitations.

Research issues, challenges, and future directions

The intelligent techniques provide a promising solution towards the realisation of the DSM but there
are certain research issues and challenges in DSM that need to be addressed carefully. New radio
capabilities breed new demands for spectrum access, which presents an open challenge in CR
networks. Some of these research issues, challenges, and future directions are summarised below:

● Wideband and higher frequencies: With the increasing demand for wireless traffic and applica­
tions, not only the higher spectrum efficiency but also more bandwidth resources are required
34 A. KAUR AND K. KUMAR

which presents a major challenge in CR networks. As the number of users increases in current
wireless systems, scalability becomes an extremely important issue. For such purpose, novel
brain empowered/intelligent techniques need to be designed together with reconfiguration
capability to achieve the best performance.
● Interference management: Due to coexistence among heterogeneous networks provide a new
challenge in DSM. First and foremost, consideration is how well the devices are coexisting
together. There is also a need to develop adaption in modulation schemes as well as other
parameters, which enables devices to avoid interference. Moreover, cooperation between
different networks and among CRs further mitigate the effect of interference. The amount of
cooperation between heterogeneous networks and among CRs is also an additional challenge
that needs to be addressed.
● Privacy and security: The deployment of new spectrum access technologies and their realisation
raises new security challenges that have not been studied previously. Furthermore, regulators
and policymakers have to consider what data from spectrum usage can be collected to access
spectrum utilisation without trespassing on the user’s privacy. There is a need to ensure the
correct implementation of the deployed system and when they are not, enforcement proce­
dures will be needed to solve the DSM problem.
● Intelligent techniques based on green CR networks: The recent exponential growth of wireless
technologies used in daily life needs to consider issues related to health and the environment.
Hence, the designing of the future CRs need to be energy efficient to cut carbon emission.
There exist limited work for DSM with consideration of green communication. Thus, there is
a need to design DSM techniques considering green communication.
● Massive MIMO with intelligence: Massive MIMO uses a large number of antennas to provide an
extra degree of freedom and diversity to improve its performance. To use this extra degree of
freedom efficiently, intelligence needs to be incorporated to improve the perception capability
and reconfigurability of CRs.
● Regulatory and policy reform challenges: Beyond technical issues, there are also policy reform
challenges in DSM. Future deployed systems may employ spectrum dynamically in which PUs
can sell their spectrum to CRs temporarily. Thus, the service level agreements for spectrum
sharing need to be reconsidered. Furthermore, strategies need to be designed for dynamic
spectrum auctions and markets.
● Multi-objective optimisation: In CR networks, multi objectives are conflicting with each other
such as minimisation of power consumption, maximisation of throughput, minimisation of BER,
etc. The optimisation of multiple objectives is a challenging task that needs to be addressed
carefully.

Other challenges include lack of incentives for spectrum sharing, authorisations constraints, and
hardware, software, and protocols related issues need to be considered which require interdisciplin­
ary collaboration among researchers of diverse backgrounds to address these challenges.

Conclusion
The paper has provided a comprehensive review and the classification of intelligent techniques for
DSM in CR networks. For efficient realisation, CRs are combined with intelligent techniques so that
dynamic and intelligent spectrum management can be done. The intelligent spectrum management
schemes are surveyed in the context of spectrum sensing, spectrum decision, spectrum sharing, and
spectrum mobility. The main emphasis of the work is to elaborate on the role of intelligent
techniques in CR to enhance its learning capability. We have presented state-of-art achievements
in applying intelligent techniques for DSM along with their strength and limitations. It provides an
overview of active research in the area of dynamic spectrum management in CR networks.
 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 35

Unluckily, available techniques for spectrum management still lack the ample reflection of various
network parameters in real-world scenarios which are quite complex to model, so some model-free
schemes would become increasingly important. In future work, eliciting and encouraging coopera­
tive behaviour through rewards and mechanism design will become important and looks promising
to be an important area of further research.

Disclosure statement
No potential conflict of interest was reported by the authors.

Funding
This work is financially supported by Department of Science and Technology, Government of India vide project
reference no. [SR/WOS-A/ET-52/2017] under Women Scientist Scheme-A.

ORCID
Amandeep Kaur http://orcid.org/0000-0002-8325-668X
Krishan Kumar http://orcid.org/0000-0003-4020-4051

References
Abbas, N., Nasser, Y., & El Ahmad, K. (2015). Recent advances on artificial intelligence and learning techniques in
cognitive radio networks. EURASIP Journal on Wireless Communications and Networking, 2015(1), 174. https://doi.org/
10.1186/s13638-015-0381-7
Akbar, I. A., & Tranter, W. H. (2007). Dynamic spectrum allocation in cognitive radio using hidden Markov models:
Poisson distributed case. In Proceeding of IEEE SoutheastCon (pp. 196–201). Richmond, VA. https://doi.org/10.1109/
SECON.2007.342884
Akyildiz, I. F., Lee, W. Y., Vuran, M. C., & Mohanty, S. (2006). NeXt generation/dynamic spectrum access/cognitive radio
wireless networks : A survey. Computer Networks, 50(13), 2127–2159. https://doi.org/10.1016/j.comnet.2006.05.001
Akyildiz, I. F., Lee, W. Y., Vuran, M. C., & Mohanty, S. (2008). A survey on spectrum management in cognitive radio
networks. IEEE Communication Magazine, 46(4), 40–48. https://doi.org/10.1109/MCOM.2008.4481339
Awe, O. P., & Lambotharan, S. (2015). Cooperative spectrum sensing in cognitive radio networks using multi-class
support vector machine algorithms. In Proceedings of 9th International Conference on Signal Processing and
Communication Systems (ICSPCS) (pp. 1–7). Cairns, QLD. https://doi.org/10.1109/ICSPCS.2015.7391780
Awe, O. P., Zhu, Z., & Lambotharan, S. (2013). Eigenvalue and support vector machine techniques for spectrum sensing
in cognitive radio networks. In Proceedings of 2013 Conference on Technologies and Applications of Artificial
Intelligence (pp. 223–227). Taipei. https://doi.org/10.1109/TAAI.2013.52
Baldo, N., & Zorzi, M. (2008). Fuzzy logic for cross-layer optimization in cognitive radio networks. In Proceedings of 4th
IEEE Consumer Communications and Networking Conference(Vol. 2007, pp. 1128–1133). Las Vegas, NV. https://doi.org/
10.1109/MCOM.2008.4481342
Bayrakdar, M. E. (2020). Cooperative communication based access technique for sensor networks. International Journal
of Electronics, 107(2), 212–225. https://doi.org/10.1080/00207217.2019.1636313
Bayrakdar, M. E., & Çalhan, A. (2018). Artificial bee colony–based spectrum handoff algorithm in wireless cognitive radio
networks. International Journal of Communication Systems, 31(5), e3495. https://doi.org/10.1002/dac.3495
Bengio, Y., Yao, L., Alain, G., & Vincent, P. (2013). Generalized denoising auto-encoders as generative models. In
Proceedings of the 26th International Conference on Neural Information Processing Systems (pp. 899–907). Neural
Information Processing Systems (NIPS): Papers published at the Neural Information Processing Systems Conference
Bing., Y., Rui, Y., & Ye, F. (2010). Non-cooperative spectrum allocation based on game theory in cognitive radio networks.
In Proceedings of 5th IEEE International Conference on Bio-Inspired Computing: Theories and Applications (BIC-TA) (pp.
1134–1137). Changsha. https://doi.org/10.1109/BICTA.2010.5645094
Bkassiny, M., Li, Y., & Jayaweera, S. K. (2013). A survey on machine-learning techniques in cognitive radios. IEEE
Communications Surveys & Tutorials, 15(3), 1136–1159. https://doi.org/10.1109/SURV.2012.100412.00017
Burges, C. J. C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge
Discovery, 2(2), 121–167. https://doi.org/10.1023/A:1009715923555
Carpenter, G. A., & Grossberg, S. (1987). ART 2: Self-organization of stable category recognition codes for analog input
patterns. Applied Optics, 26(23), 4919–4930. https://doi.org/10.1364/AO.26.004919
36 A. KAUR AND K. KUMAR

Carpenter, G. A., Grossberg, S., Systems, A., & Systems, N. (2016). Encyclopedia of machine learning and data mining.
Springer.
Christian, I., Moh, S., Chung, I., & Lee, J. (2012). Spectrum mobility in cognitive radio networks. IEEE Communications
Magazine, 50(6), 114–121. https://doi.org/10.1109/MCOM.2012.6211495
Dandurand, F., & Shultz, T. R. (2009). Connectionist models of reinforcement, imitation, and instruction in learning to
solve complex problems. IEEE Transactions on Autonomous Mental Development, 1(2), 110–121. https://doi.org/10.
1109/TAMD.2009.2031234
Doersch, C. (2016). Tutorial on variational autoencoders. arXiv preprint. arXiv:1606.05908.
Dong, X., Li, Y., & Wei, S. Q. (2012). Design and implementation of a cognitive engine functional architecture. Chinese
Science Bulletin, 57(28–29), 3698–3704. https://doi.org/10.1007/s11434-012-5102-6
Dorigo, M., & Gambardella, L. M. (1997). Ant colony system: A cooperative learning approach to the traveling salesman
problem. IEEE Transactions on Evolutionary Computation, 1(1), 53–66. https://doi.org/10.1109/4235.585892
Doyle, L. (2009). Essentials of cognitive radio. Cambridge University Press. https://doi.org/10.1017/CBO9780511576577
Duda, R. O., Hart, P. E., & Stork, D. G. (2001). Pattern classification. John Wiley.
El Morabit, Y., Mrabti, F., & Abarkan, E. H. (2015). Spectrum allocation using genetic algorithm in cognitive radio
networks. In Proceedins of 3rd International Workshop on RFID and Adaptive Wireless Sensor Networks (RAWSN) (pp.
90–93), Agadir. https://doi.org/10.1109/RAWSN.2015.7173287
Eltholth, A. A. (2016). Spectrum prediction in cognitive radio systems using a wavelet neural network. In Proceedings of
24th International Conference on Software, Telecommunications and Computer Networks (SoftCOM)(pp. 1–6). Split.
https://doi.org/10.1109/SOFTCOM.2016.7772181
Fang, H., Xu, L., & Choo, K. K. R. (2017). Stackelberg game based relay selection for physical layer security and energy
efficiency enhancement in cognitive radio networks. Applied Mathematics and Computation, 296, 153–167. https://
doi.org/10.1016/j.amc.2016.10.022
FCC. Docket no.03-322. 2003. Notice of proposed rule making and order.
Ghosh, C., Cordeiro, C., Agrawal, D. P., & Rao, M. B. (2009). Markov chain existence and Hidden Markov models in
spectrum sensing. In Proceedings of IEEE International Conference on Pervasive Computing and Communications (pp.
1–6). https://doi.org/10.1109/PERCOM.2009.4912868
Glandon, A. M. (2017). Recurrent neural networks and matrix methods for cognitive radio spectrum prediction and security
[Master of Science (MS), Thesis]. Electrical/Computer Engineering, Old Dominion University. https://doi.org/10.
25777/7xga-r535
Hanif, M. F., Smith, P. J., Taylor, D. P., & Martin, P. A. (2011). MIMO cognitive radios with antenna selection. IEEE
Transactions on Wireless Communications, 10(11), 3688–3699. https://doi.org/10.1109/TWC.2011.092011.101095
Hasegawa, M., Hirai, H., Nagano, K., Harada, H., & Aihara, K. (2014). Optimization for centralized and decentralized
cognitive radio networks. Proceedings of the IEEE, 102(4), 574–584. https://doi.org/10.1109/JPROC.2014.2306255
He, A., Kyoon Bae, K., Newman, T. R., Gaeddert, J., Kim, K., Menon, R., Morales-Tirado, L., Neel, J. J., Zhao, Y., Reed, J. H., &
Tranter, W. H. (2010). A survey of artificial intelligence for cognitive radios. IEEE Transactions on Vehicular Technology,
59(4), 1578–1592. https://doi.org/10.1109/TVT.2010.2043968
He, Q., Feng, Z., & Zhang, P. (2013). Reconfiguration decision making based on Ant Colony optimization in cognitive
radio network. Wireless Personal Communications, 71(2), 1247–1269. https://doi.org/10.1007/s11277-012-0872-3
He, Q., & Zhang, P. (2012). Dynamic channel assignment using Ant Colony optimization for cognitive radio networks. In
Proceedings of IEEE Vehicular Technology Conference (VTC Fall) (pp. 1–5). Quebec City, QC. https://doi.org/10.1109/
VTCFall.2012.6398951
Hofmann, M. (2006). Support vector machines - Kernels and the kernel trick an elaboration for the hauptseminar.
Homayounvala, E. (2015). A Bayesian approach to service selection for secondary users in cognitive radio networks.
International Journal of Advanced Computer Science and Applications, 6(10), 201–204. https://doi.org/10.14569/
IJACSA.2015.061029
Huang, P., Lin, F., & Zhou, J. (2013). ACO-based routing algorithm for cognitive radio networks. Mathematical Problems in
Engineering, 2013, 1–8. https://doi.org/10.1155/2013/264504
Huang, Y., Jiang, H., Hu, H., & Yao, Y. (2009). Design of learning engine based on Support Vector Machine in cognitive
radio. In Proceedings of International Conference on Computational Intelligence and Software Engineering, Wuhan (pp.
1–4). https://doi.org/10.1109/CISE.2009.5363127
Huang, Y., Wang, J., & Jiang, H. (2010). Modeling of learning inference and decision-making engine in cognitive radio. In
Proceedings of 2nd International Conference on Networks Security, Wireless Communications and Trusted Computing
(pp. 258–261). Wuhan, Hubei. https://doi.org/10.1109/NSWCTC.2010.195
Jacob, J., Jose, B. R., & Mathew, J. (2014). Spectrum prediction in cognitive radio networks: A bayesian approach. In
Proceedings of 8th International Conference on Next Generation Mobile Apps, Services and Technologies (pp. 203–208).
Oxford. https://doi.org/10.1109/NGMAST.2014.40
Jedlicka, P., & Ryba, T. (2016). Genetic algorithm application in image segmentation. Pattern Recognition and Image
Analysis, 26(3), 497–501. https://doi.org/10.1134/S105466181603007X
Jhajj, H. K. (2017). Ant colony optimization for spectrum sensing in cognitive radio networks.
 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 37

Jiang, C., Zhang, H., Ren, Y., Han, Z., Chen, K.-C., & Hanzo, L. (2017). Machine learning paradigms for next-generation
wireless networks. IEEE Wireless Communications, 24(2), 98–105. https://doi.org/10.1109/MWC.2016.1500356WC
Kasbekar, G. S., & Sarkar, S. (2010). Spectrum auction framework for access allocation in cognitive radio networks. IEEE/
ACM Transactions on Networking, 18(6), 1841–1854. https://doi.org/10.1109/TNET.2010.2051453
Katidiotis, A., Tsagkaris, K., & Demestichas, P. (2010). Performance evaluation of artificial neural network-based learning
schemes for cognitive radio systems. Computers & Electrical Engineering, 36(3), 518–535. https://doi.org/10.1016/j.
compeleceng.2009.12.005
Khaledi, M., & Abouzeid, A. A. (2013). Auction-based spectrum sharing in cognitive radio networks with heterogeneous
channels. Information Theory and Applications Workshop (ITA) (pp. 1–8). San Diego, CA. https://doi.org/10.1109/ITA.
2013.6502923
Khozeimeh, F., & Haykin, S. (2012). Brain-inspired dynamic spectrum management for cognitive radio ad hoc networks.
IEEE Transactions on Wireless Communications, 11(10), 3509–3517. https://doi.org/10.1109/TWC.2012.081312.111538
Kim, S. (2017). Inspection game based cooperative spectrum sensing and sharing scheme for cognitive radio IoT system.
Computer Communications, 105, 116–123. https://doi.org/10.1016/j.comcom.2017.01.015
Kotsiantis, S. B. (2007). Supervised machine learning: A review of classification techniques. Informatica, 31(3), 249–268.
http://www.informatica.si/index.php/informatica/article/view/148
Kour, H., Jha, R. K., & Jain, S. (2018). A comprehensive survey on spectrum sharing: Architecture, energy efficiency and
security issues. Journal of Network and Computer Applications, 103, 29–57. https://doi.org/10.1016/j.jnca.2017.11.010
Koushik, A. M., Bentley, E., Hu, F., & Kumar, S. (2018). A hardware testbed for learning-based spectrum handoff in
cognitive radio networks. Journal of Network and Computer Applications, 106, 68–77. https://doi.org/10.1016/j.jnca.
2017.11.003
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition.
Proceedings of the IEEE, 86(11), 2278–2324. https://doi.org/10.1109/5.726791
Lee, W., Kim, M., & Cho, D. (2019). Deep cooperative sensing: Cooperative spectrum sensing based on convolutional
neural networks. IEEE Transactions on Vehicular Technology, 68(3), 3005–3009. https://doi.org/10.1109/TVT.2019.
2891291
Li, H. (2009). Multi-agent Q-learning of channel selection in multi-user cognitive radio systems: A two by two case. In
Proceedings of IEEE International Conference on Systems, Man and Cybernetics (pp. 1893–1898). San Antonio, TX.
https://doi.org/10.1109/ICSMC.2009.5346172
Li, H., Liu, Y., & Zhang, D. (2010). Dynamic spectrum access for cognitive radio systems with repeated games. In
Proceedings of IEEE International Conference on Wireless Communications, Networking and Information Security (pp.
59–62). Beijing. https://doi.org/10.1109/WCINS.2010.5541766
Liu, J., Jing, X. J., Sun, S., Wang, X., Cheng, D., & Huang, H. (2015). Multilayer feed-forward neural network learning based
dynamic chinese restaurant model for dynamic spectrum access in cognitive radio networks. In Proceedings of 15th
International Symposium on Communications and Information Technologies (ISCIT) (pp. 173–176). Nara. https://doi.
org/10.1109/ISCIT.2015.7458335
Liu, K. (2016). Optimization algorithm of cognitive radio spectrum sensing based on quantum neural network.
Automatic Control and Computer Sciences, 50(5), 324–331. https://doi.org/10.3103/S0146411616050072
Liu, X., Zhang, Y., Li, Y., Zhang, Z., & Long, K. (2013). A survey of cognitive radio technologies and their optimization
approaches. In Proceedings of 8th International Conference on Communications and Networking in China (CHINACOM)
(pp. 973–978). Guilin. https://doi.org/10.1109/ChinaCom.2013.6694736
Lo, B. F., & Akyildiz, I. F. (2010). Reinforcement learning-based cooperative sensing in cognitive radio ad hoc networks. In
Proceedings of 21st Annual IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (pp.
2244–2249). Instanbul. https://doi.org/10.1109/PIMRC.2010.5671686
Maheshwari, P., & Singh, A. K. (2015). A fuzzy logic based approach to spectrum assignment in cognitive radio networks.
In Proceedings of IEEE International Advance Computing Conference (IACC) (pp. 278–281). Banglore. https://doi.org/10.
1109/IADCC.2015.7154713
Manco-vásquez, J., Lázaro-Gredilla, M., Ramírez, D., Vía, J., & Santamaría, I. (2014). A Bayesian approach for adaptive
multiantenna sensing in cognitive radio networks. Signal Proceesing, 96(B), 228–240. https://doi.org/10.1016/j.sigpro.
2013.10.005
Matinmikko, M., Rauma, T., Mustonen, M., Harjula, I., Sarvanko, H., & Mammela, A. et al. (2009). Application of fuzzy logic
to cognitive radio systems. IEICE Transactions on Communications, E92B(12), 3572–3580. https://doi.org/10.1587/
transcom.E92.B.3572
MohanaPriya, P., & Shalinie, S. M. (2017). Restricted Boltzmann machine-based cognitive protocol for secure routing in
software defined wireless networks. IET Networks, 6(6), 162–168. https://doi.org/10.1049/iet-net.2017.0054
Noorshams, N., Malboubi, M., & Bahai, A. (2010). Centralized and decentralized cooperative spectrum sensing in
cognitive radio networks: A novel approach. In Proceedings of 11th IEEE International Workshop on Signal
Processing Advances in Wireless Communications (SPAWC) (pp. 1–5), Marrakech. https://doi.org/10.1109/SPAWC.
2010.5670998
38 A. KAUR AND K. KUMAR

Nurmi, P. (2004). Modelling routing in wireless ad hoc networks with dynamic Bayesian games. In Proceedings of 1st
Annual IEEE Communications Society Conference on Sensor and Ad Hoc Communications and Networks (pp. 63–70).
Santa Clara, CA, USA. https://doi.org/10.1109/SAHCN.2004.1381903
Peng, J., Li, J., Li, S., & Li, J. (2011). Multi-relay cooperative mechanism with Q-learning in cognitive radio multimedia
sensor networks. In Proceedings of 10th International Conference on Trust, Security and Privacy in Computing and
Communications (pp. 1624–1629). Changsha. https://doi.org/10.1109/TrustCom.2011.225
Pham, C., Tran, N. H., Do, C. T., Moon, S. I., & Hong, C. S. (2014). Spectrum handoff model based on Hidden Markov model
in Cognitive Radio Networks. In Proceedings of International Conference on Information Networking 2014 (ICOIN2014)
(pp. 406–411). Phuket. https://doi.org/10.1109/ICOIN.2014.6799714
Pourpeighambar, B., Dehghan, M., & Sabaei, M. (2017). Multi-agent learning based routing for delay minimization in
cognitive radio networks. Journal of Network and Computer Applications, 84, 82–92. https://doi.org/10.1016/j.jnca.
2017.01.026
Qadir, J. (2016). Artificial intelligence based cognitive routing for cognitive radio networks. Artificial Intelligence Review,
45, 25–96. https://doi.org/10.1007/s10462-015-9438-6
Ramzan, M. R., Nawaz, N., Ahmed, A., Naeem, M., Iqbal, M., & Anpalagan, A. (2017). Multi-objective optimization for
spectrum sharing in cognitive radio networks: A review. Pervasive and Mobile Computing, 41, 106–131. https://doi.
org/10.1016/j.pmcj.2017.07.010
Ranjan, N., & Ahmad, S. (2016). A survey on techniques in NLP. International Journal of Computer Applications, 134(8), 6–9.
https://doi.org/10.5120/ijca2016907355
Rifai, S., & Muller, X. (2011). Contractive Auto-encoders: Explicit invariance during feature extraction. In Proceedings of
28th International Conference on Machine Learning (pp. 2011). Bellevue, WA, USA.
Saad, A., Staehle, B., & Knorr, R. (2016). Spectrum prediction using hidden Markov models for industrial cognitive radio.
In Proceedings of 12th IEEE International Conference on Wireless and Mobile Computing, Networking and
Communications (WiMob) (pp. 1–7), New York, NY, . https://doi.org/10.1109/WiMOB.2016.7763231
Salakhutdinov, R., & Hinton, G. (2009). Deep Boltzmann Machines. In Proceedings of the 12th International Conference on
Artificial Intelligence and Statistics (AISTATS), Clearwater Beach, Florida.
Saleem, Y., Yau, K. L. A., Mohamad, H., Ramli, N., & Rehmani, M. H. (2015). SMART: A SpectruM-aware clusteR-based
rouTing scheme for distributed cognitive radio networks. Computer Networks, 91, 196–224. https://doi.org/10.1016/j.
comnet.2015.08.019
Salem, T. M., Abdel-mageid, S., Abdel-kader, S. M., & Zaki, M. (2016). ICSSSS : An intelligent channel selection scheme for
cognitive radio ad hoc networks using a self organized map followed by simple segregation. Pervasive and Mobile
Computing, 39, 195–213. https://doi.org/10.1016/j.pmcj.2016.06.008
Salgado, C., Hernandez, C., Molina, V., & Beltran-Molina, F. A. (2016). Intelligent algorithm for spectrum mobility in
cognitive wireless networks. Procedia Computer Science, 83, 278–283. https://doi.org/10.1016/j.procs.2016.04.126
Selim, A., Paisana, F., Arokkiam, J. A., Zhang, Y., Doyle, L., & DaSilva, L. A. (2018). Spectrum monitoring for radar bands
using deep convolutional neural networks. arXiv:1705.00462.
Sendrei, L., Pastirčák, J., Marchevský, S., & Gazda, J. (2015). Cooperative spectrum sensing schemes for cognitive radios
using dynamic spectrum auctions. In Proceedings of 38th International Conference on Telecommunications and Signal
Processing (TSP) (pp. 159–162). Prague. https://doi.org/10.1109/TSP.2015.7296243
Senthilkumar, S., & Geetha Priya, C. (2016). Hidden Markov model based channel selection framework for cognitive radio
network. Computer & Electrical Engineering, 65, 516–526. https://doi.org/10.1016/j.compeleceng.2016.06.016
Shapiro, J. (2001). Genetic algorithm in machine learning and its applications, 2049. https://doi.org/10.1007/3-540-
44673-7_7
Siddique, T., & Azam, A. (2010). Spectrum optimization in cognitive radio networks using genetic algorithms. EURASIP
Journal on Wireless Communications and Networking, 133.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. MIT Press Cambridge.
Tang, Z., & Li, S. (2017). Deep recurrent neural network for multiple time slot frequency spectrum predictions of
cognitive radio. KSII Transactions on Internet and Information Systems, 11(6), 3029–3045. https://doi.org/10.3837/
tiis.2017.06.013
Thakur, P., Kumar, A., Pandit, S., Singh, G., & Satashia, S. N. (2017). Spectrum mobility in cognitive radio network using
spectrum prediction and monitoring techniques. Physical Communication, 24, 1–8. https://doi.org/10.1016/j.phycom.
2017.04.005
Thilina, K. M., Saquib, N., & Hossain, E. (2013). Machine learning techniques for cooperative spectrum sensing in
cognitive radio networks. IEEE Journal on Selected Areas in Communications, 31(11), 2209–2221. https://doi.org/10.
1109/JSAC.2013.131120
Tian, Z., & Giannakis, G. B. (2006). A wavelet approach to wideband spectrum sensing for cognitive radios. In Proceedings
of 1st International Conference on Cognitive Radio Oriented Wireless Networks and Communications (CROWNCOM) (pp.
1–5). Mykonos Island. https://doi.org/10.1109/CROWNCOM.2006.363459
Tripathi, P. S. M., Chandra, A., Kumar, A., & Sridhara, K. (2011). Dynamic spectrum access and cognitive radio. In
Proceedings of 2nd International Conference on Wireless Communication, Vehicular Technology, Information Theory
 JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE 39

and Aerospace & Electronic Systems Technology (Wireless VITAE) (pp. 1–5). Chennai. https://doi.org/10.1109/
WIRELESSVITAE.2011.5940911
Tsagkaris, K., Bantouna, A., & Demestichas, P. (2012). Self-organizing maps for advanced learning in cognitive radio
systems. Computers & Electrical Engineering, 38(4), 862–881. https://doi.org/10.1016/j.compeleceng.2012.03.008
Tsagkaris, K., Katidiotis, A., & Demestichas, P. (2008). Neural network-based learning schemes for cognitive radio
systems. Computer Communications, 31(14), 3394–3404. https://doi.org/10.1016/j.comcom.2008.05.040
Tumuluru, V. K., Wang, P., & Niyato, D. (2010). A neural network based spectrum prediction scheme for cognitive radio. In
Proceedings of IEEE International Conference on Communications (pp. 1–5). Cape Town. https://doi.org/10.1109/ICC.
2010.5502348
Tumuluru, V. K., Wang, P., & Niyato, D. (2012). Channel status prediction for cognitive radio networks. Wireless
Communication Mobile Computing, 12(10), 862–874. https://doi.org/10.1002/wcm.1017
Wang, B., Ji, Z., & Liu, K. J. R. (2007). Self-learning repeated game framework for distributed primary-prioritized dynamic
spectrum access. In Proceedings of 4th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc
Communications and Networks (pp. 631–638). San Diego, CA. https://doi.org/10.1109/SAHCN.2007.4292875
Wang, H., Gao, L., Gan, X., Wang, X., & Hossain, E. (2010). Cooperative spectrum sharing in cognitive radio networks: A
game-theoretic approach. In Proceedings of IEEE International Conference on Communications (pp. 1–5). Cape Town.
https://doi.org/10.1109/ICC.2010.5502052
Wang, W., Kwasinski, A., Niyato, D., & Han, Z. (2018). Learning for robust routing based on stochastic game in cognitive
radio networks. IEEE Transactions on Communications, 66(6), 2588–2602. https://doi.org/10.1109/TCOMM.2018.
2799616
Wang, Y., Ye, Z., Wan, P., & Zhao, J. (2019). A survey of dynamic spectrum allocation based on reinforcement learning
algorithms in cognitive radio networks. Artificial Intelligence Review, 51(3), 493–506. https://doi.org/10.1007/s10462-
018-9639-x
Wang, Y., Zhang, Z., Ma, L., & Chen, J. (2014). SVM-based spectrum mobility prediction scheme in mobile cognitive radio
networks. The Science World Journal, 2014, 1–11. https://doi.org/10.1155/2014/395212
White paper: Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2017–2022, 2018.
Wu, Z., & Fei, Y. (2010). Channel and power allocation in cognitive radio networks. In Proceedings of 2010 7th
International Symposium on Wireless Communication Systems (pp. 1061–1065). York. https://doi.org/10.1109/ISWCS.
2010.5624510
Xing, X., Jing, T., Huo, Y., Li, H., & Cheng, X. (2013). Channel quality prediction based on Bayesian inference in cognitive
radio networks. In Proceedings of IEEE INFOCOM (pp. 1465–1473). Turin. https://doi.org/10.1109/INFCOM.2013.
6566941
Xu, X., Chai, X., & Zhang, Z. (2014). Self-organization approaches for optimization in cognitive radio networks. China
Communications, 11(4), 121–129. https://doi.org/10.1109/CC.2014.6827574
Yang, G., Li, B., Tan, X., & Wang, X. (2015). Adaptive power control algorithm in cognitive radio based on game theory. IET
Communications, 9(15), 1807–1811. https://doi.org/10.1049/iet-com.2014.1109
Yang, Y., Jiang, H., Liu, C., & Lan, Z. (2012). Research on cognitive radio engine based on genetic algorithm and radial
basis function neural network. In Procceedings of Spring Congress on Engineering and Technology (pp. 1–5). Xian.
https://doi.org/10.1109/SCET.2012.6342027
Yau, K. A., Komisarczuk, P., & Teal, P. D. (2010). Applications of reinforcement learning to cognitive radio networks. In
Proceedings of IEEE International Conference on Communications Workshops (pp. 1–6). Capetown. https://doi.org/10.
1109/ICCW.2010.5503970
Yau, K. A., Komisarczuk, P., & Teal, P. D. (2012). Reinforcement learning for context awareness and intelligence in wireless
networks: Review, new features and open issues. Journal of Network and Computer Applications, 35(1), 253–267.
https://doi.org/10.1016/j.jnca.2011.08.007
Yau, K. L. A., Poh, G. S., Chien, S., & Al-Rawi, H. A. A. (2014). Application of reinforcement learning in cognitive radio
networks: Models and algorithms. The Science World Journal, 2014, 1–23. https://doi.org/10.1155/2014/209810
Yu, Q. (2013). A survey of cooperative games for cognitive radio networks. Wireless Personal Communication, 73(3),
949–966. https://doi.org/10.1007/s11277-013-1225-6
Zafari, F., Gkelias, A., & Leung, K. K. (2019). A survey of indoor localization systems and technologies. IEEE
Communications Surveys & Tutorials, 21(3), 2568–2599. http://doi.org/10.1109/COMST.2019.2911558 doi:10.1109/
COMST.9739
Zhang, D., & Zhai, X. (2011). SVM-based spectrum sensing in cognitive radio. In Proceedings of 7th International
Conference on Wireless Communications, Networking and Mobile Computing (pp. 1–4). Wuhan. https://doi.org/10.
1109/wicom.2011.6040028
Zhang, S., Hu, J., Bao, Z., & Wu, J. (2013). Prediction of spectrum based on improved RBF neural network in cognitive
radio. International Conference on Wireless Information Networks and Systems (WINSYS) (pp. 1–5). Reykjavik.
Zhang, Z., Long, K., & Wang, J. (2013). Self-organization paradigms and optimization approaches for cognitive radio
technologies: A survey. IEEE Wireless Communications, 20(2), 36–42. https://doi.org/10.1109/MWC.2013.6507392
40 A. KAUR AND K. KUMAR

Zheng, S., Kam, P. Y., Liang, Y. C., & Zeng, Y. (2013). Spectrum sensing for digital primary signals in cognitive radio:
A bayesian approach for maximizing spectrum utilization. IEEE Transactions on Wireless Communications, 12(4),
1774–1782. https://doi.org/10.1109/TWC.2013.030413.120735
Zheng, X., Cui, L., Chen, J., Wu, Q., & Wang, J. (2008). Cooperative spectrum sensing in cognitive radio systems. Congress
on Image and Signal Processing (pp. 262–266). Sanya, Hainan. https://doi.org/10.1109/CISP.2008.234
Zhu, X., & Fujii, T. (2016). A novel modulation classification method in cognitive radios using higher-order cumulants and
denoising stacked sparse autoencoder. In Proceedings of Asia-Pacific Signal and Information Processing Association
Annual Summit and Conference (APSIPA) (pp. 1–5), Jeju. https://doi.org/10.1109/APSIPA.2016.7820860
Zhu, X., Liu, Y., Weng, W., & Yuan, D. (2008). Channel sensing algorithm based on neural networks for cognitive wireless
mesh networks. In Proceedings of 4th International Conference on Wireless Communications, Networking and Mobile
Computing (pp. 1–4), Dalian. https://doi.org/10.1109/WiCom.2008.312