Deep Learning Algorithms Zoran Gacovski download

https://ebookbell.com/product/deep-learning-algorithms-zoran-
gacovski-50883686

Explore and download more ebooks at ebookbell.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

Deep Learning Algorithms And Applications Witold Pedrycz Shyiming Chen

https://ebookbell.com/product/deep-learning-algorithms-and-
applications-witold-pedrycz-shyiming-chen-10548736

Various Deep Learning Algorithms In Computational Intelligence Oscar

Humberto Montiel Ross

https://ebookbell.com/product/various-deep-learning-algorithms-in-
computational-intelligence-oscar-humberto-montiel-ross-51333770

Embedded Deep Learning Algorithms Architectures And Circuits For

Alwayson Neural Network Processing Bert Moons Daniel Bankman Marian
Verhelst

https://ebookbell.com/product/embedded-deep-learning-algorithms-
architectures-and-circuits-for-alwayson-neural-network-processing-
bert-moons-daniel-bankman-marian-verhelst-52246890

Integrating Deep Learning Algorithms To Overcome Challenges In Big

Data Analytics R Sujatha

https://ebookbell.com/product/integrating-deep-learning-algorithms-to-
overcome-challenges-in-big-data-analytics-r-sujatha-33727212
Deep Learning By Example A Handson Guide To Implementing Advanced
Machine Learning Algorithms And Neural Networks Menshawy

https://ebookbell.com/product/deep-learning-by-example-a-handson-
guide-to-implementing-advanced-machine-learning-algorithms-and-neural-
networks-menshawy-7240872

Machine And Deep Learning Algorithms And Applications Uday Shankar

Shanthamallu

https://ebookbell.com/product/machine-and-deep-learning-algorithms-
and-applications-uday-shankar-shanthamallu-37583854

Handson Deep Learning Algorithms With Python Master Deep Learning

Algorithms With Extensive Math By Implementing Them Using Tensorflow
Sudharsan Ravichandiran

https://ebookbell.com/product/handson-deep-learning-algorithms-with-
python-master-deep-learning-algorithms-with-extensive-math-by-
implementing-them-using-tensorflow-sudharsan-ravichandiran-10746278

Handson Deep Learning Algorithms With Python Sudharsan Ravichandiran

https://ebookbell.com/product/handson-deep-learning-algorithms-with-
python-sudharsan-ravichandiran-170506082

Novel Financial Applications Of Machine Learning And Deep Learning

Algorithms Product Modelling And Applications Mohammad Zoynul Abedin

https://ebookbell.com/product/novel-financial-applications-of-machine-
learning-and-deep-learning-algorithms-product-modelling-and-
applications-mohammad-zoynul-abedin-48060318
本书版权归Arcler所有
 Deep Learning Algorithms

本书版权归Arcler所有
本书版权归Arcler所有
 Deep Learning Algorithms

Edited by:
Zoran Gacovski

ARCLER
P r e s s

www.arclerpress.com

本书版权归Arcler所有
 Deep Learning Algorithms
Zoran Gacovski

Arcler Press
224 Shoreacres Road
Burlington, ON L7L 2H2
Canada
www.arclerpress.com
Email: orders@arclereducation.com

HERRN(GLWLRQ2

ISBN: (HERRN)

This book contains information obtained from highly regarded resources. Reprinted
material sources are indicated. Copyright for individual articles remains with the au-
thors as indicated and published under Creative Commons License. A Wide variety of
references are listed. Reasonable efforts have been made to publish reliable data and
views articulated in the chapters are those of the individual contributors, and not neces-
sarily those of the editors or publishers. Editors or publishers are not responsible for
the accuracy of the information in the published chapters or consequences of their use.
The publisher assumes no responsibility for any damage or grievance to the persons or
property arising out of the use of any materials, instructions, methods or thoughts in the
book. The editors and the publisher have attempted to trace the copyright holders of all
material reproduced in this publication and apologize to copyright holders if permission
has not been obtained. If any copyright holder has not been acknowledged, please write
to us so we may rectify.

Notice: Registered trademark of products or corporate names are used only for explana-
       

© 2022 Arcler Press

ISBN: 978-1-77469-183-0 (Hardcover)

Arcler Press publishes wide variety of books and eBooks. For more information about
Arcler Press and its products, visit our website at www.arclerpress.com

本书版权归Arcler所有
 DECLARATION

Some content or chapters in this book are open access copyright free
published research work, which is published under Creative Commons
License and are indicated with the citation. We are thankful to the
publishers and authors of the content and chapters as without them this
book wouldn’t have been possible.

本书版权归Arcler所有
本书版权归Arcler所有
 ABOUT THE EDITOR

Dr. Zoran Gacovski has earned his PhD degree at Faculty of Electrical
engineering, Skopje. His research interests include Intelligent systems and
Software engineering, fuzzy systems, graphical models (Petri, Neural and
Bayesian networks), and IT security. He has published over 50 journal and
conference papers, and he has been reviewer of renowned Journals. Currently,
he is a professor in Computer Engineering at European University, Skopje,
Macedonia.

本书版权归Arcler所有
TABLE OF CONTENTS

List of Contributors .......................................................................................xv

List of Abbreviations .................................................................................... xxi
Preface.................................................................................................. ....xxiii

Section 1: Methods and Approaches for Deep Learning

Chapter 1 Advancements in Deep Learning Theory and Applications:

Perspective in 2020 and Beyond ............................................................... 3
Abstract ..................................................................................................... 3
Introduction ............................................................................................... 4
Deep Network Topologies.......................................................................... 8
Application of Deep Learning .................................................................. 11
Modern Deep Learning Platforms ............................................................ 14
Training Algorithms.................................................................................. 17
Routine Challenges of Deep Learning ...................................................... 19
Available Open-Source Datasets .............................................................. 21
References ............................................................................................... 24

Chapter 2 Deep Ensemble Reinforcement Learning With Multiple

Deep Deterministic Policy Gradient Algorithm....................................... 29
Abstract ................................................................................................... 29
Introduction ............................................................................................. 30
Background ............................................................................................. 32
Methods .................................................................................................. 34
Results and Discussion ............................................................................ 39
Conclusions ............................................................................................. 50
References ............................................................................................... 51
Chapter 3 Dynamic Decision-Making For Stabilized Deep Learning
Software Platforms .................................................................................. 55
Abstract ................................................................................................... 55
Introduction ............................................................................................. 56
Stabilized Control for Reliable Deep Learning Platforms .......................... 57
The Use of Lyapunov Optimization for Deep Learning Platforms ............. 63
Emerging Applications ............................................................................. 68
Conclusions ............................................................................................. 69
Acknowledgements ................................................................................. 70
References ............................................................................................... 71

Chapter 4 Deep Learning For Hyperspectral Data Classification Through

Exponential Momentum Deep Convolution Neural Networks ................ 73
Abstract ................................................................................................... 73
Introduction ............................................................................................. 74
Feature Learning ...................................................................................... 75
Structure Design of Hyperspectral Data Classification Framework ........... 76
Exponential Momentum Gradient Descent Algorithm .............................. 77
Experiment and Analysis .......................................................................... 80
Conclusion .............................................................................................. 86
Acknowledgments ................................................................................... 87
References ............................................................................................... 88

Chapter 5 Ensemble Network Architecture for Deep Reinforcement

Learning .................................................................................................. 93
Abstract ................................................................................................... 93
Introduction ............................................................................................. 94
Related Work ........................................................................................... 95
Ensemble Methods for Deep Reinforcement Learning .............................. 97
Experiments ........................................................................................... 100
Conclusion ............................................................................................ 102
References ............................................................................................. 104

x
 Section 2: Deep Learning Techniques Applied in Biology

Chapter 6 Fish Detection Using Deep Learning ..................................................... 109

Abstract ................................................................................................. 109
Introduction ........................................................................................... 110
Literature Review ................................................................................... 111
Materials and Methods .......................................................................... 113
Data Augmentation ................................................................................ 118
Results and Discussion .......................................................................... 126
Conclusion ............................................................................................ 129
Acknowledgments ................................................................................. 130
References ............................................................................................. 131

Chapter 7 Can Deep Learning Identify Tomato Leaf Disease? ............................... 135
Abstract ................................................................................................. 135
Introduction ........................................................................................... 136
Related Work ......................................................................................... 137
Materials and Methods .......................................................................... 138
Experiments and Results ........................................................................ 143
Conclusion ............................................................................................ 149
Acknowledgments ................................................................................. 150
References ............................................................................................. 151

Chapter 8 Deep Learning For Plant Identification In Natural Environment ........... 157
Abstract ................................................................................................. 157
Introduction ........................................................................................... 158
Proposed Bjfu100 Dataset and Deep Learning Model ............................ 159
Experiments and Results ........................................................................ 162
Resnet26 on Flavia Dataset .................................................................... 165
Conclusion ............................................................................................ 166
Acknowledgments ................................................................................. 167
References ............................................................................................. 168

Chapter 9 Applying Deep Learning Models to Mouse Behavior Recognition ......... 171
Abstract ................................................................................................. 171
Introduction ........................................................................................... 172

xi
 The Mouse Behavior Dataset ................................................................. 174
Experiments and Results ........................................................................ 175
Conclusions ........................................................................................... 186
Acknowledgements ............................................................................... 186
References ............................................................................................. 187

Section 3: Deep learning Applications in Medicine

Chapter 10 Application of Deep Learning in Neuroradiology: Brain

Hemorrhage Classification Using Transfer Learning ............................. 191
Abstract ................................................................................................. 191
Introduction ........................................................................................... 192
Related Work ......................................................................................... 194
Convolutional Neural Network .............................................................. 195
Transfer Learning ................................................................................... 196
Materials and Methods .......................................................................... 197
Results and Discussion .......................................................................... 204
Limitations ............................................................................................. 210
Conclusion ............................................................................................ 211
References ............................................................................................. 212

Chapter 11 A Review of the Application of Deep Learning in Brachytherapy.......... 217

Abstract ................................................................................................. 217
Introduction ........................................................................................... 218
Organ Delineation and Segmentation .................................................... 219
Segmentation and Reconstruction of the Applicator
(Interstitial Needles) ..................................................................... 220
Dose Calculation ................................................................................... 222
Application of Treatment Planning System ............................................. 222
Others ................................................................................................... 223
Conclusions ........................................................................................... 224
References ............................................................................................. 225

xii
Chapter 12 Exploring Deep Learning and Transfer Learning for Colonic
Polyp Classification ............................................................................... 229
Abstract ................................................................................................. 229
Introduction ........................................................................................... 230
Materials and Methods .......................................................................... 232
Results and Discussion .......................................................................... 242
Conclusion ............................................................................................ 250
Acknowledgments ................................................................................. 251
References ............................................................................................. 252

Chapter 13 Deep Learning Algorithm For Brain-Computer Interface ...................... 259

Abstract ................................................................................................. 259
Introduction ........................................................................................... 260
Critical Review of the Related Literature ................................................ 273
Comparison of Classification Algorithms ................................................ 276
Discussion ............................................................................................. 277
Methodology ......................................................................................... 280
Conclusion ............................................................................................ 281
References ............................................................................................. 282

Section 4: Deep Learning in Pattern Recognition Tasks

Chapter 14 The Application of Deep Learning In Airport Visibility Forecast ........... 287
Abstract ................................................................................................. 287
Introduction ........................................................................................... 288
Deep Learning ....................................................................................... 288
The Establishment of Prediction Model .................................................. 289
Predictive Effect Test............................................................................... 291
Conclusions ........................................................................................... 295
References ............................................................................................. 297

Chapter 15 Hierarchical Representations Feature Deep Learning

For Face Recognition............................................................................. 299
Abstract ................................................................................................. 299
Introduction ........................................................................................... 300
Images Preprocessing............................................................................. 302
Feature Extraction .................................................................................. 304

xiii
 Designing the Classifiers of Supervised Learning .................................... 307
Designing the Classifier Combining Unsupervised
and Supervised Learning .............................................................. 315
Experiments ........................................................................................... 322
Conclusion ............................................................................................ 332
Acknowledgements ............................................................................... 332
References ............................................................................................. 334

Chapter 16 Review of Research on Text Sentiment Analysis Based

on Deep Learning .................................................................................. 341
Abstract ................................................................................................. 341
Introduction ........................................................................................... 342
Brief Review on the Research Progress of Text Sentiment Analysis .......... 343
Introduction to Text Sentiment Analysis Based on Deep Learning ........... 344
Summary and Prospect .......................................................................... 348
References ............................................................................................. 350

Chapter 17 Classifying Hand Written Digits With Deep Learning ........................... 353
Abstract ................................................................................................. 353
Introduction ........................................................................................... 354
Digit Classification with Deep Networks................................................ 354
Experiment ............................................................................................ 360
Conclusions ........................................................................................... 361
References ............................................................................................. 364

Chapter 18 Bitcoin Price Prediction Based on Deep Learning

Methods ................................................................................................ 367
Abstract ................................................................................................. 367
Introduction ........................................................................................... 368
Dataset Exploration................................................................................ 368
Pre-Processing ....................................................................................... 369
Models .................................................................................................. 369
Results ................................................................................................... 371
Conclusion and Discussion.................................................................... 375
References ............................................................................................. 376

Index ..................................................................................................... 377

xiv
LIST OF CONTRIBUTORS

Md Nazmus Saadat
University of Kuala Lumpur, Malaysia

Muhammad Shuaib
University of Kuala Lumpur, Malaysia

Junta Wu
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen
518071, China

Huiyun Li
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen
518071, China

Soohyun Park
Korea University, Seoul, Republic of Korea

Dohyun Kim
Naver Webtoon Corporation, Seongnam, Republic of Korea

Joongheon Kim
Korea University, Seoul, Republic of Korea

Qi Yue
Xi’an Institute of Optics and Precision Mechanics, CAS, Xi’an 710119, China
University of Chinese Academy of Sciences, Beijing 100039, China
Xi’an University of Posts and Telecommunications, Xi’an 710121, China

Caiwen Ma
Xi’an Institute of Optics and Precision Mechanics, CAS, Xi’an 710119, China

Xi-liang Chen
Institute of Command Information System, PLA University of Science and Technology,
No. 1, Hai Fu Road, Guang Hua Road, Qin Huai District, Nanjing City, Jiangsu Province
210007, China
Lei Cao
Institute of Command Information System, PLA University of Science and Technology,
No. 1, Hai Fu Road, Guang Hua Road, Qin Huai District, Nanjing City, Jiangsu Province
210007, China

Chen-xi Li
Institute of Command Information System, PLA University of Science and Technology,
No. 1, Hai Fu Road, Guang Hua Road, Qin Huai District, Nanjing City, Jiangsu Province
210007, China

Zhi-xiong Xu
Institute of Command Information System, PLA University of Science and Technology,
No. 1, Hai Fu Road, Guang Hua Road, Qin Huai District, Nanjing City, Jiangsu Province
210007, China

Jun Lai
Institute of Command Information System, PLA University of Science and Technology,
No. 1, Hai Fu Road, Guang Hua Road, Qin Huai District, Nanjing City, Jiangsu Province
210007, China

Suxia Cui
Department of Electrical and Computer Engineering, Prairie View A&M University,
Prairie View, TX 77446, USA

Yu Zhou
Department of Electrical and Computer Engineering, Prairie View A&M University,
Prairie View, TX 77446, USA

Yonghui Wang
Department of Computer Science, Prairie View A&M University, Prairie View, TX
77446, USA

Lujun Zhai
Department of Electrical and Computer Engineering, Prairie View A&M University,
Prairie View, TX 77446, USA

Keke Zhang
College of Engineering, Northeast Agricultural University, Harbin 150030, China

Qiufeng Wu
College of Science, Northeast Agricultural University, Harbin 150030, China

Anwang Liu
College of Engineering, Northeast Agricultural University, Harbin 150030, China

xvi
Xiangyan Meng
College of Science, Northeast Agricultural University, Harbin 150030, China

Yu Sun
School of Information Science and Technology, Beijing Forestry University, Beijing
100083, China

Yuan Liu
School of Information Science and Technology, Beijing Forestry University, Beijing
100083, China

Guan Wang
School of Information Science and Technology, Beijing Forestry University, Beijing
100083, China

Haiyan Zhang
School of Information Science and Technology, Beijing Forestry University, Beijing
100083, China

Ngoc Giang Nguyen

Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa,
Japan;

Dau Phan
Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa,
Japan;

Favorisen Rosyking Lumbanraja

Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa,
Japan;

Mohammad Reza Faisal

Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa,
Japan;

Bahriddin Abapihi
Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa,
Japan;

Bedy Purnama
Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa,
Japan;

xvii
Mera Kartika Delimayanti
Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa,
Japan;

Kunti Robiatul Mahmudah

Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa,
Japan;

Mamoru Kubo
Institute of Science and Engineering, Kanazawa University, Kanazawa, Japan

Kenji Satou
Institute of Science and Engineering, Kanazawa University, Kanazawa, Japan

Awwal Muhammad Dawud

Department of Computer Engineering, Cyprus International University, Nicosia, Cyprus

Kamil Yurtkan
Department of Computer Engineering, Cyprus International University, Nicosia, Cyprus

Huseyin Oztoprak
Department of Computer Engineering, Cyprus International University, Nicosia, Cyprus

Hai Hu
Applied Nuclear Technology in Geosciences Key Laboratory of Sichuan Province,
Chengdu University of Technology, Chengdu, China

Yang Shao
Applied Nuclear Technology in Geosciences Key Laboratory of Sichuan Province,
Chengdu University of Technology, Chengdu, China

Shijie Hu
Applied Nuclear Technology in Geosciences Key Laboratory of Sichuan Province,
Chengdu University of Technology, Chengdu, China

Eduardo Ribeiro
Department of Computer Sciences, University of Salzburg, Salzburg, Austria
Department of Computer Sciences, Federal University of Tocantins, Palmas, TO, Brazil

Andreas Uhl
Department of Computer Sciences, University of Salzburg, Salzburg, Austria

Georg Wimmer
Department of Computer Sciences, University of Salzburg, Salzburg, Austria

xviii
Michael Häfner
St. Elisabeth Hospital, Vienna, Austria

Asif Mansoor
National University of Sciences and Technology, Islamabad, Pakistan

Muhammad Waleed Usman

National University of Computer and Emerging Sciences, Islamabad, Pakistan

Noreen Jamil
National University of Computer and Emerging Sciences, Islamabad, Pakistan

M. Asif Naeem
National University of Computer and Emerging Sciences, Islamabad, Pakistan

Lei Zhu
Training Center of Xinjiang Air Traffic Management Bureau, Urumqi, China

Guodong Zhu
College of Atmospheric Science, Nanjing University, Nanjing, China
Meteorological Center of Xinjiang Air Traffic Management Bureau, Urumqi, China

Lei Han
Meteorological Center of Xinjiang Air Traffic Management Bureau, Urumqi, China

Nan Wang
Meteorological Center of Xinjiang Air Traffic Management Bureau, Urumqi, China

Haijun Zhang
Guangdong Provincial Key Laboratory of Conservation and Precision Utilization of
Characteristic Agricultural Resources in Mountainous Areas, Meizhou, China
School of Computing, Jiaying University, Meizhou, China

Yinghui Chen
Guangdong Provincial Key Laboratory of Conservation and Precision Utilization of
Characteristic Agricultural Resources in Mountainous Areas, Meizhou, China
School of Mathematics, Jiaying University, Meizhou, China

Wenling Li
College of Science, Yanbian University, Yanji, China

Bo Jin
College of Science, Yanbian University, Yanji, China

xix
Yu Quan
Department of Economics and Management of Yanbian University, Yanji, China

Ruzhang Yang
Shanghai Foreign Language School, Shanghai, China

Xiangxi Jiang
Barstow School of Ningbo, Ningbo, China

xx
LIST OF ABBREVIATIONS

      

AUV Autonomous underwater vehicle
BA Boltzmann addition
BM Boltzmann multiplication
CPU Central processing unit
CAD Computer-aided diagnosis
CT Computer tomography
CV Computer vision
CNN Convolution neural network
DBN Deep belief network
DCNN Deep convolution neural network
DNNs Deep neural network
DRL Deep reinforcement learning
DRBM Deep restricted Boltzmann machine
DPG Deterministic policy gradient
GRU Gated recurrent units
GPU Graphical processing unit
HBPNNs Hybrid BP neural networks
ICH Intracranial haemorrhage
JELSR Joint embedding learning and sparse regression
LR Logistic regression
LSTM Long short-term memory network
MV Majority voting
MDP Markov decision processes
MRSF Minimum redundancy spectral feature selection
MVEP Motion-onset visual evoked potential
MLP Multi-layer perceptron
NIRS Near-infrared spectroscopy
PMI Pointwise mutual information
PFCL Prior fully connected layers
QDA Quadratic discriminant analysis
RBF Radial Basis Function
RH Relative humidity
RBM Restricted Boltzmann machine
RMSE Root mean square error
SVD Singular value decomposition
SCP Slow cortical potentials
TORCS The open racing car simulator

xxii
PREFACE

The Deep learning is a branch of machine learning based on data presentation via
complex representations with high degree of abstraction - that are obtained by applying
learned nonlinear transformations. Deep learning methods find their application in
important areas of artificial intelligence, such as: computer vision, natural language
processing, speech and sound comprehension, as well as in bioinformatics. Deep
learning is a class of machine learning algorithms that:
 uses multilayer nonlinear processor units to extract and transform features.
Each subsequent layer takes as input the output elements of the previous
layer.
 learns in a supervised and / or unsupervised manner.
 learns a number of levels of representation - corresponding to different
degrees of abstraction.
 uses some form of descending gradient algorithm to train through error
backpropagation.
The layers used in deep programming include the hidden layers of the artificial neural
network and a multitude of statement formulas.
This book covers the most important discriminant and generative deep models with
special emphasis on practical implementations. We cover the key elements of classical
neural networks and provides an overview of the building blocks, regularization
techniques, and learning methods that are specific to deep models. Also we consider the
deep convolutional models and illustrates their application in image classification and
natural language processing.
The generative deep models are often used in computer vision applications and natural
language processing. Sequence modeling by deep feedback neural networks can be
applied in the field of natural language processing. Practical implementations of deep
learning are made in modern dynamic languages (Python, Lua or Julia), and also with
application frameworks for deep learning (e.g. Theano, TensorFlow, Torch).
This edition covers different topics from deep learning algorithms, including: methods
and approaches for deep learning, deep learning applications in biology, deep learning
applications in medicine, and deep learning applications in pattern recognition systems.
Section 1 focuses on methods and approaches for deep learning, describing
advancements in deep learning theory and applications - perspective in 2020 and
beyond; deep ensemble reinforcement learning with multiple deep deterministic policy
gradient algorithm; dynamic decision-making for stabilized deep learning software
platforms; deep learning for hyperspectral data classification through exponential
momentum deep convolution neural networks; and ensemble network architecture for
deep reinforcement learning.
Section 2 focuses on deep learning applications in biology, describing fish detection
using deep learning; deep learning identification of tomato leaf disease; deep learning
for plant identification in natural environment; and applying deep learning models to
mouse behavior recognition.
Section 3 focuses on deep learning applications in medicine, describing application
of deep learning in neuroradiology: brain hemorrhage classification using transfer
learning; a review of the application of deep learning in brachytherapy; exploring
deep learning and transfer learning for colonic polyp classification; and deep learning
algorithm for brain-computer interface.
Section 4 focuses on deep learning applications in pattern recognition systems, describing
application of deep learning in airport visibility forecast; hierarchical representations
feature deep learning for face recognition; review of research on text sentiment analysis
based on deep learning; classifying hand written digits with deep learning; and bitcoin
price prediction based on deep learning methods.
 SECTION 1:

Methods and Approaches for

Deep Learning
 CHAPTER 1

ADVANCEMENTS IN DEEP
LEARNING THEORY AND
APPLICATIONS:
PERSPECTIVE IN 2020
AND BEYOND

Md Nazmus Saadat and Muhammad Shuaib

University of Kuala Lumpur, Malaysia

ABSTRACT
The aim of this chapter is to introduce newcomers to deep learning, deep
learning platforms, algorithms, applications, and open-source datasets.
This chapter will give you a broad overview of the term deep learning,
in context to deep learning machine learning, and Artificial Intelligence
(AI) is also introduced. In Introduction, there is a brief overview of the
research achievements of deep learning. After Introduction, a brief history
of deep learning has been also discussed. The history started from a famous
scientist called Allen Turing (1951) to 2020. In the start of a chapter after

Citation: Md Nazmus Saadat and Muhammad Shuaib (December 9th 2020). Advance-
ments in Deep Learning Theory and Applications: Perspective in 2020 and beyond,
Advances and Applications in Deep Learning, Marco Antonio Aceves-Fernandez, Inte-
chOpen, DOI: 10.5772/intechopen.92271.
Copyright: © 2020 by authors and IntechOpen. This paper is an open access article
distributed under a Creative Commons Attribution 3.0 License .
4 Deep Learning Algorithms

Introduction, there are some commonly used terminologies, which are used
in deep learning. The main focus is on the most recent applications, the
most commonly used algorithms, modern platforms, and relevant open-
source databases or datasets available online. While discussing the most
recent applications and platforms of deep learning, their scope in future is
also discussed. Future research directions are discussed in applications and
platforms. The natural language processing and auto-pilot vehicles were
considered the state-of-the-art application, and these applications still need
a good portion of further research. Any reader from undergraduate and
postgraduate students, data scientist, and researchers would be benefitted
from this.
Keywords:- Deep learning, machine learning      ,
neural networks

INTRODUCTION
Deep learning is focusing comprehensively on video, image, text and audio
recognition, autonomous driving, robotics, healthcare, etc. [1]. Deep learning
is a result orientated field of study that why getting very much attention
from researcher and academicians. The Rina Dechter introduced the word of
deep learning in 1986, the main motivation behind the advent of field deep
learning was making an intelligent machine that mimic the human brain. In
humans, the brain is the most important and decision-making organ; brain
takes decision based on sight, smell, touch, and sounds. The brain also can
store memory and solve complex problems based on their experience.
For the last few decades, the researchers dreamed of making a machine
that is as intelligent as, like our brains, they started studying the biological
structure and working of the human brain. Making a robot that performs
certain duties and self-driving cars is to reduce roadside incidents.
Because according to the World Health Organization (WHO), 1.35 million
people die every year in road incidents [2] and approximately 90% of the
incidents are due to human errors [3]. To develop state-of-the-art devices
for the applications listed above, ones need to think in a different way of
         
of the most innovative paradigms that make it possible up to some extent. In
deep learning, the word deep indicates the number of layers through which
         
     ! "    
intelligence machine learning or deep learning because all these overlap
 Advancements In Deep Learning Theory And Applications: ... 5

each other some way or the other. Machine learning is any sort of computer
program that can learn by their own without having specially programmed
by the programmer. There are two types of machine learning: supervised
learning and unsupervised learning. In supervised learning, you teach or
train the machine with a fully labeled data, the machine learns from the
labeled data and then anticipate the unforeseen data. In supervised learning,
the machine can only give you correct output when the input is already
experienced in training phase; it is based on experience; the more is the
training dataset or experience of your machine the higher is the chances of
getting the actual output. It is a time-consuming process and also required a
lot of expertise in data science. On the other hand, in unsupervised learning,
supervision of a model is not needed, rather the model work on its own
catches new data and discovers the information inside the data. It usually
deals with label-less data; compared to supervised learning, unsupervised
          
patterns.
Deep learning models are agile and result oriented in terms of
complicated abstractions. Deep learning models are mostly based on ANN,
categorically CNNs, although there are deep belief networks, generative
models, propositional formulas and Boltzmann machine also play their part
(Figure 1).

Figure 1. Deep learning a subset of machine learning and AI.

Deep learning has been evaluated as a game-changer in AI and computer
vision. Today, state-of-the-art object detection is possible only due to deep
learning [4]; traditional methods of object detection are not enough to cater
with detection so smartly. To understand the whole image of object detection,
              #    
6 Deep Learning Algorithms

calculate the concept and locations of the objects in every image, that is,
object detection which is based on face detection, pedestrian detection,
and skeleton detection [5]. Deep learning has cutting-edge technology
              
healthcare. It has a very deep impact on the life of the people or societies
because its application is always the need of the day. The deep learning
    $    #
data analytics. Big data analytics is the number of complicated processes
'      * 
methods used to identify the hidden patterns, unknown correlations market
trends, and customer preference from huge dataset. Big data analytics can
 # #          
#              
Deep learning is an emerging area of research and modern application.
The deep learning       +
covers industry, business, and healthcare; it combines all the hot research-
       < +  +   #   # 
optimization, and cyber-physical systems; these all are seen interdependent.
Gartner has proposed top ten technology trends for 2020, some of them
are, hyper-automation, human augmentation, AI Security, IoT, Autonomous
things; etc.; all are related to AI, machine learning, and deep learning some
way or the other. Surely, deep learning will bring a bunch of innovations
to everywhere whether it is industry, health-care or business intelligence.
According to Ref. [6], machine learning and AI will be used more in 2020
experts says in the survey conducted by the computer-world.
In 2019, many researchers, academicians, and teachers claimed that
deep learning is over because it cannot do common-sense reasoning;
Rodney Brooks a professor in MIT says that some popular press started
stories that the deep learning will be over by 2020. In 2020, hybrid,
interdisciplinary, collaborative, and open-minded research is expected to
add more contribution. The topics that are expected to be more prevalent in
2020 are common-sense reasoning, active learning and life-long learning,
multi-modal and multi-task learning, open-domain dialogue conversation,
medical applications and autonomous vehicles, ethics that includes privacy,
  #  # 
There are two most common deep learning platforms: TensorFlow and
PyTorch; these two platforms compete; and this competition is very fruitful
for the community; TensorFlow is easy to use, integrated with Keras; while
 Advancements In Deep Learning Theory And Applications: ... 7

on the other hand, Pytorch has TPU support, etc. In 2020, it is expected to
have a platform which can easily transform a TensorFlow model to Pytorch
and vice versa. There is a need to develop an actively developed stable
reinforcement learning framework. The higher layers of abstractions are
expected in 2020 like Keras, so that machine learning is used outside the
    

History
Deep learning is a sub branch of machine learning, and machine learning is a
sub branch of artificial intelligence. Deep learning is a set of algorithms that
processes large set of data and imitates the thinking process. The history of
deep leaning is started from 1943, when Warren McCulloch and Walter Pitts
created a neural network-based computer model. There basic aim was to
mimic thought process of human brain; they used algorithms and mathematics
to make the threshold logic to mimic human thought process. Alan Turing
called the father of AI concluded in 1951 that the machines would not take
much time in started thinking of their own; at some point of time, they would
be able to talk to each other; and it is also expected that they would take the
control of the universe. In context to this, the frank Rosenblatt introduced
single and multi-layer artificial neural network (1957–1962). The history
amazed us when the world champion of chess player called Kasparov was
defeated by deep blue computer in 1997. In 1957–62, the single layer and
multi-layer perceptron’s was introduced. The first deep feedforward general
purpose learning algorithm multilayer perceptron’s by Alexey Icakhnenko
and Lapa was published in 1967. In 1971, a deep network with eight layers
trained by the group method of data handling algorithm was described
already. The idea of backpropagation, Recurrent Neural Network (RNN),
and restricted Boltzmann machine (RBM) was introduced in 1970–1986. In
1979-1998, the Convolution Neural Network (CNN), Bidirectional RNN,
and long short-term memory (LSTM) were the state of the art. The deep
belief network (DBN) was introduced by Geoff Hinton in 2006. The data
sets called ImageNet and AlexNet that was created in 2009. Generative
Adversarial Network (GAN) is a class of machine learning system invented
by Ian Goodfellow and his colleagues in 2014. Coming up in history in 2016
Google DeepMind challenge match between Alpha Go versus Lee Sedol,
the AlphaGo win all the matches from a world champion Lee Sedol. AlfaGo
and AlfaZero are computer programs developed by artificial intelligence
8 Deep Learning Algorithms

research company called DeepMind in (2016–2017); it plays the board

game Go. The transformer introduced in 2017–19 a deep learning model
used specially used for Natural language Processing (NLP). Although there
is a lot of community contributed to the deep learning but Yann LeCun,
Geoffrey Hinton, and Yoshua Bengio have received Turing awards in 2018.

DEEP NETWORK TOPOLOGIES

Deep neural network (DNN)

In DNN, there is multilayer perceptron or hidden layer between the input
and output. All the layers are connected to previous layers; by going through
each layer, the network estimates the exact output based on the weights and
activation function. Through DNN, we can model any complex non-linear
relation. The backbone of the DNN is the characteristic of learning about
the feature that is most relevant to the targets [7]. The DNN has research
gap in model selection, training dynamics, by using graph convolution
neural network combination optimization, and Bayesian neural network
for estimation of uncertainty. There are a lot of applications for DNN, that
is, computer vision, machine translation, social network filtering, playing
board, video games, and medical diagnosis (Figure 2).

Figure 2. Deep neural network.

 Advancements In Deep Learning Theory And Applications: ... 9

Recurrent neural network (RNN)

RNN is a type of deep learning network that is used specifically when there
is sequential data or time-series, that is, video, speech, etc. The RNN usually
maintained the data from the previous state to the next state. It is called
recurrent because it performs the same function for each input, while the
output is different because it also depends on past calculations. The state-
of-the-art topic of deep learning with RNN is Long Short-Term Memory
Network (LSTM). RNN provides the solution to many problems, that is,
intelligent transportation system [8], solving time-varying matrix inversion
[9], and many more. The RNN is famous for sentence evaluation and
linguistic data processing (Figure 3).

Figure 3. Recurrent neural network.

Deep belief network (DBN)

DBN is a probabilistic unsupervised deep learning algorithm. It has many
layers of hidden variables. To solve the more complex problems, it needs more
hidden layers; each layer is a special statistical relation with the other layer.
DBN can learn probabilistically; after learning, BDN needs training under
supervisor to perform classification. The DBN is used to recognize clusters
and generates images, video sequences, and motion-capture data (Figure 4).
10 Deep Learning Algorithms

Figure 4. Deep belief network.

Boltzmann machine (BM)

The BM is a network that is a uniformly attached, neuron-like unit, which is
responsible for taking decisions stochastically about whether to be off or on.
Computational problems are solved through BM like search, optimization,
and learning problem. Many features are uncovered in learning algorithm
that shows very complex behavior in training dataset. Boltzmann machine
is used for classification and dimensionality reduction.

Restricted Boltzmann machine (RBM)

RBM introduced in 1986 by Smolensky: two layers visible and hidden
units, while there is no connection between visible-visible and hidden-
hidden. It can learn a probability distribution over a collection of datasets.
The applications of RBM are features learning, collaborative filtering,
dimensionality reduction, and classification.

Convolutional neural network (CNN)

In CNN, the layers are delicately connected to input layer as well as each
other. There is a specific function for each neuron of the subsequent layer
like it is only responsible for only a part of the input. CNN is now widely
used for remote sensing, computer vision, audio, and text processing [10].

Deep auto-encoder
Just like others, deep auto-encoder has also many hidden layers. The difference
between a simple auto-encoder and deep-auto-encoder is the simple auto-
encoder that has one hidden layer, while the deep-auto-encoder has many
 Advancements In Deep Learning Theory And Applications: ... 11

hidden layers. In deep-auto-encoder, the training is complex normally, you

need to train one hidden layer first to reconstruct the structure of the input
data, and this input data are further used to train other hidden layers and so
on. Some applications of deep auto-encoder are image extraction, image
generation recommendation system, and sequence to sequence prediction.

Gradient descent (GD)

GD is used to reduce the overall cost function; it is considered as an
optimization algorithm and is widely used for determination of coefficient
function in machine learning. When there is not possible to estimate the
parameters analytically, then GD is used to calculate the desired parameters.
Using the GD weight of the model is updated for every epoch. It is used for
supervised machine learning.

Stochastic gradient descent (SGD)

Just like GD, SGD is also an optimization algorithm but GD is used when
the datasets are small, while SGD is usually used when the datasets are
large, and SD becomes very costly if used for a large number of datasets.

APPLICATION OF DEEP LEARNING

Deep learning is new and state-of-the-art technology used for large scale
applications now-days. Deep learning (also called differential programming
or structure learning) is member of a large family of machine learning class.
It is edge-cutting technology used for many different new research fields
which are stated below.

Deep learning in automatic speech recognition

The automatic speech recognition is the convincing application of deep
learning. Speech recognition means making speech as in input to a machine
that can make the input process very easy and has a hundred of other
advantages as well, that is, illiterate people can also use technology, speech
coding, text to speech synthesis, speech recognition, speaker recognition,
speech enhancement, speech segmentation, language identification, and
many more [11]. The speech is the natural form of communication, hence it
is considered a very convincing application.
12 Deep Learning Algorithms

Image recognition
Image recognition based on deep learning becomes very famous and
accurate result-oriented technology based on the training and experience
of machine. Deep learning plays a very important part in image recognition
and image classification in underwater target recognition [12] although the
images from underwater are always noisy and deteriorated. MNIST is one
of the most renowned examples used for image classification, below is the
simple of dataset of MNIST dataset (Figure 5).

Figure 5. Image example of handwritten digits from the MNIST dataset.

Natural language processing

LSTM helps a lot in language modeling and machine translation [13];
language modeling task is to understand the language. To implement the
language, models’ neural networks are used. Google translate is the most
famous and widely used application in this regard; Google translate is used
for more than 100 languages all over the world. It also used LSTM; and it
learns from millions of examples and translates the whole sentence rather
than word by word translation. BERT (Google) is one of the most common
technologies in this field achieved a lot of benchmarks, that is, sentence
classification, sentence pair classification, sentence pair similarity, sentence
tagging, create contextualized words embedding, question answering, and
multiple-choice questions. There are some other transformer-based language
models developed in 2019, which are XLNet (Google/CMU), RoBERTa
(Facebook), Distil BERT (hugging Face), CTRL (Salesforce), GPT-2 (Open-
AI), ALBERT (Google), and Magatron (NVIDIA). Magatron is the largest
transformer model ever trained. It has 8.3 million parameters transformer
language model. XLNet is the best transformer in terms of performance;
XLNet outperforms BERT on 20 tasks often by a large margin. ALBERT
developed by Google is used to reduce the parameters via cross-layer
 Advancements In Deep Learning Theory And Applications: ... 13

parameters sharing. The state of the artwork in this domain is about multi-
domain task-oriented dialogue system [14]. In 2020, it expected to combine
common sense reasoning with language models, extending language model
context to thousands of words and to have more focus on open-domain
dialogue (Figure 6).

Figure 6. NLP and deep learning.

Games and robotics

Robots are the agents who are artificially intelligent and working in the real-
world replacing humans. OpenAI and Dota 2 are popular games; in 2017,
1v1 bot beats top professional Dota 2 players; in 2018, OpenAI five lost two
games against top Dota 2 player, while in 2019, OpenAI five beat OG team
(the world champion in 2018). The OpenAI five win in 2019 is only because
of the more training compute; the current version of OpenAI has consumed
800 petaflops/day and experiences about 45,000 years of dota self-play over
10 real-time months.
The current version has 99.9%-win rate versus the 2018 version. It is one
of the best experiences in deep learning that systems that learn to play with
each other and incrementally improving. OpenAI Rubiks Cube Manipulation
is another example from Robotics. The researchers are expecting in 2020
to implement reinforcement-learning methods in the manipulation of real-
world interaction tasks.
In games, experts are loss from different machines, using these machines
to assist human experts in discovering new strategies. Waymo a company
that is focusing on developing auto-pilot like Tesla in October 2018; they
have 10 million miles on road and now in 2020 they have 20 million miles
on road 20,000 of classes for structure test, also initiated testing without
having a safety driver.
14 Deep Learning Algorithms

Financial fraud detection

Deep learning is playing a very important role in financial fraud detection.
With the advent of technology and a significant amount of e-commerce
platforms, the number of e-payments is increasing day by day chances of
financial fraud, which is also a source of headache for banks and other
financial institutions. Thus, focusing on fraud detection is a hot area of
research. The author of [15] used auto-encoder for financial fraud detection
[16]. This research uses deep learning model for fraud detection, while [17]
proposed a solution to fraud detection using machine learning approach.

Deep learning in health-care

In this modern era of computing, deep learning also produced best results
medical and health care, that is, deep learning is used for cancer cell
coordination, organ segmentation, protein folding, lesion detection, and
image enhancement in the field of medicine. There are several other issues
like [18, 19, 20, 21] and much more where deep learning is directly involved
in the suggestion of the ultimate solution to the problem in healthcare.

Military
Deep learning is used for making many different military devices used in wars
or other spy services. The military is also working on robots to train the robots
to handle the critical situation through these robots. The militaries of some
countries are making their weapons more intelligent using AI. In a war zone,
AI can be embedded in the robots for remote surgical support in healthcare.

Cybersecurity
Cybersecurity is also one of the hot research areas; deep learning models
are used for the cybersecurity of the Internet of Things (IoT) [22]. The IoT
devices are usually low power devices having power-constrained that’s why
always vulnerable to external threats. Deep learning models can detect threats
more accurately than any other technology. The author of [23] used deep
learning and machine learning for intrusion, spam, and malware detection.

MODERN DEEP LEARNING PLATFORMS

Open-sources deep learning platforms discussed in this section. It will
provide a quick review of the open-source platforms for beginners and
mediocre because every platform has its pros and cons.
 Advancements In Deep Learning Theory And Applications: ... 15

TensorFlow
The TensorFlow is new and open-source platform for differential
programming; it was developed by Google team called Google brain and
was first released in 2015 [24]. In February 2017, they released version 1.0.0;
TensorFlow can work on CPU and GPU; it is available for Mac, Linux, and
windows and also for mobile computing platform android and iOS. It is the
most famous machine learning library in the world today. Its best-supported
client language is python but there is also interface available in C++, Java,
and GO. It is easy to use and have Keras integration. TensorFlow has many
of its versions available like for mobiles TensorFlow lite, for industry
TensorFlow Serving, etc.

Pytorch
Pytorch is also machine learning and deep learning library, based on torch
library. It was initially released by Facebook’s AI Research lab (FAIR) in 2016.
Pytorch has two high-level features, Tensor computing with graphics processing
units (GPU), and auto-diff based deep neural network. It is too easy in Pytorch
to move tensors to and from GPU. Pytorch Mobile is the version of Pytorch
used for mobiles. There are some key features of Pytorch; the first feature is
called imperative programming; most of the python code is imperative; this
type of programming is more flexible. The other feature of Pytorch is dynamic
computation graphs, it run time the system generates the graph structure,
dynamic graph work well for dynamic networks like RNN, dynamic graph also
makes debugging very easy. The Pytorch provides maximum flexibility and
speed during implementing and building deep neural network.

Theano
Theano is designed by Montreal Institute for Learning Algorithms (MILA),
which is very famous after their deployment, but unfortunately, there is no
support after version 1.0.0 (November 2017). It is a python library designed
for code compilation optimization [25]; it is primarily used for mathematical
operations like multi-dimensional arrays. Theano was far better than other
python libraries like Numpy in terms of speed, computing symbolic graphs,
and stability optimizations. Tensor operations, GPU computation, and
parallelism are also supported by Theano.
16 Deep Learning Algorithms

Microsoft cognitive toolkit (CNTK)

CNTK is used for commercial-grade distributed deep learning. It can be used
as a standalone tool for machine learning or also can be included as a library in
C++ programs, python, and C#; its model evaluation functionality can be also
used from Java programs. It supports ONNX that allows sharing model with
frameworks Caffe2, MXNet, and PyTorch [26]. CNTK can be used only on Linux
and Windows. The CNTK is considered as a powerful machine learning platform
similar surge of performance as compared to other widely used platforms [27].

Keras
Keras is a powerful library written in python; it uses TensorFlow, Theano,
and CNTK as a framework because it does not have their framework. Keras
can work on GPUs and CPUs and can also support RNNs and CNNs. The
beauty of Keras is it has the ability of fast and easy prototyping; Keras is
user-friendly. It has been ranged one of the most cited API in 2018 and has
enough number of users on board.

Deep learning 4J
It is distributed open-source, robust deep learning framework for Java
designed by Skymind [28] which is added a lot to Java ecosystem and
eclipse foundation. It has compatibility with Clojure and Scala APIs just
like Keras; it is also able to work with both CPUs and GPUs. It is widely
used for academics and industrial applications.

Torch
It is a scientific computing open-source machine learning framework released
in October 2002; it is not able to work on CPUs; it is only made to focus on
GPUs accelerated computing. It is developed in programming language C
and based on Lua, a contribute in a LuaJIT, a scripting language. Max OSX
and Ubuntu 12+ can use this framework, although they have Platform for
Windows, but their implementations are not supported officially [29].

Caffe and Caffe2

CAFFE (Convolutional Architecture for Fast Feature Embedding) created
by Berkeley AI Research (BAIR) is a framework for deep learning. It is
developed in C++ with a python interface. Caffe2 was introduced by the
research group of Facebook in 2017, but Caffe2 was merged in PyTorch in
 Advancements In Deep Learning Theory And Applications: ... 17

March 2018. It supports multiple platforms, that is, Mac OS X, Windows,

Linux, iOS, and Android [30].

Apache MXNet
An MXNet is a fast-scalable deep learning platform that supports many
programming languages, i.e., Scala, Julia, C++, R, Python, Gluon API,
and Perl APIs. Like Torch, it is also made only for GPUs, and it is very
competent in multi GPUs implementations. The Apache MXNet is scalable
flexible and portable, and due to these qualities, it attracts many users.

TRAINING ALGORITHMS
One of the most important parts of deep learning is learning algorithms.
The deep neural network can be differentiated only through the number of
layers; if the number of layers increases, the network becomes deeper and
more complex. Each layer has its specific function or can detect or help in
the detection of the special feature.
According to the author [31], if the problem is face recognition, the
    # !        
higher features such as the nose, eye, ears, etc., the next layer can further
dig out the features, and so on. Thus, each layer is developed earlier to
the development of training algorithm like gradient descent; that’s why
     #       
variation. This was discussed by Yann et al. [32]; they further concluded that
a system with less manual and more automatic design can give better results
in pattern recognition.
Backpropagation is the solution; it takes information from the data
                   
recognition. List of few famous training algorithms is listed below.

Gradient descent
In statistics, data science, and machine learning, we optimize a lot of stuffs;
when we fit a line with linear regression, we optimize the intercept and
slope; when we use logistic regression, we optimize a squiggle; when we
use t-SNE, we optimize clusters. The gradient descent is used to optimize all
these and tons of others as well.
Gradient descent algorithm is similar to Newton’s roots finding algorithm
of 2D function. The methodology is very simple; just pick a point randomly
18 Deep Learning Algorithms

on a curve and move toward the right or left along x-axis depending on the
positive and negative value of the slope of the function at the given point
up-till the value of y-axis, that is, function or f(x) becomes zero. There is
the same concept behind the gradient descent; we move or traverse along
a specific path in many-dimensional space weight when the error rate is
reduced to your limits than we stop. It is one of the underlying concepts for
most of deep learning and machine learning algorithms.

(1)

Stochastic gradient descent

A method used for optimizing an objective function with the iterative
method is called stochastic gradient descent. It can also be called gradient
descent optimization. Stochastic gradient descent would randomly pick one
sample for each step and from that, just use this one sample to calculate
the derivatives, thus in super sample example, stochastic gradient descent
reduced the number of terms by computed by 3.If we had one million
samples than the stochastic gradient descent would reduce the number of
terms by computed by factor of one million. In stochastic gradient descent,
#  #       
applied, in here update of weights is more frequent, so we reach a global
minimum in less time (Figure 7).

Figure 7. Comparison of GD and SGD.

Momentum
In stochastic gradient descent to update the weight or to calculate step
size, a fixed multiplier is used as a learning rate; this can cause the update
to overshoot a potential-minima; if the gradient is too steep or delay, the
convergence of the gradient is noisy. The concept of momentum used in
 Advancements In Deep Learning Theory And Applications: ... 19

Physics is velocity exponentially decreasing an average of gradient [33].

This prevents the descent going in the wrong direction.

Levenberg-Marquardt algorithm
This type of algorithm is used for curve fitting or non-linear least-squares
problems. This algorithm is also called as deep least-square; these kinds of
issues arise usually in the least-squares curve fitting. It was first introduced
by Kenneth Levenberg in 1944, although it was rediscovered by statistician
called Donald Marquardt in 1963.

Backpropagation through time

It is one of the famous and standard methods used to train the recurrent neural
network. It was developed independently by several researchers. Unlike
general-purpose optimization techniques, it is faster in training RNN. The
backpropagation through time also has issues with local optima [34].

ROUTINE CHALLENGES OF DEEP LEARNING

According to Google trends graph more and more expert and professionals
have attracted toward deep learning in last five year; the percentage of
professionals increased from 12 to 100% [35, 36]. Deep learning is used
everywhere, that is, bio-informatics, computer vision, IoT security, health-
care, e-commerce, digital marketing, natural language processing, and many
more [37, 38]. Because of the very hot research area, there must have some
challenges which are enlisted below.

Non-contributing columns or inputs

When dealing with data or making a model, several inputs are not necessary
for finding any feature, so it is advised to drop un-necessary attributes.
There is also necessary to find one best column and make it separate from
the dataset; it can be done using numpy array in Keras; but it is difficult and
challenging to find best match attribute.

Number of hidden layers

The number of hidden layers is directly propositional to computational
complexity and deepness of the network. To deal with a large number of
layers require a high computational cost, difficult to manage a large number
of neurons.
20 Deep Learning Algorithms

Optimization algorithms
In model optimizations, gradient descent optimizer helps to make the model
cost minimum by adjusting the value; choosing an optimizer is also a
challenging task to do, because sometimes it makes your cost of model high
rather than decreasing the model cost.

Loss function
Is from the name indicate loss function, it estimates the loss or the difference
between the expected outcome and the actual outcome the formula for loss
function is listed below.
?\^'    _      `{|
There are many different ways to calculate the loss function; choosing
a loss function is also one of the essential and challenging tasks of deep
learning

Activation function
There are many different activation functions; every activation function
does not produce the same results; sigmoid activation function shows good
results with binary classification problem. One needs to be careful about
Tanh activation function because of the vanishing gradient problem. In
multi-labeled classification, softmax is the best option; Relu should be used
when there is much zeros in the input side because Relu is good in dead
neuron generation. It is also a point to use the required activation function.

Epoch
When the dataset is passed backwards and forward through the whole neural
network, it is called one epoch, as after every epoch value of weights as-
signed is analyzed to make model. The weights are changed, checked, and
tested in every cycle for the same dataset simulation. The main memory is
keeping the record of all the training data; sometimes it is not possible to
keep all the record in main memory, like for larger datasets, so the epoch is
#   #     -
resented as an epoch output. Dealing with epoch is also a challenging task
in deep learning.
 Advancements In Deep Learning Theory And Applications: ... 21

AVAILABLE OPEN-SOURCE DATASETS

Research in machine learning and deep learning is started since last many
decades hence significant improvement it brings to the society in terms of
various application-based on deep learning and machine learning. There are
many freely available datasets on the web which can be used by researchers
for various purposes.
Image datasets (Table 1):

Table 1. Open source image datasets

Pascal VOC MS COCO

MNIST handwritten digits NORB

CIFAR10/CIFAR100 color images data set COIL100

with

Caltech101 Google’s Open Images

Caltech 256 COIL 20

The dataset of street view LabelMe

STL-10 ImageNet

Geospatial datasets available online:

 NEXRAD
 OpenstreetMAP
 Landsat8
Dataset available for text (Table 2):
22 Deep Learning Algorithms

Table 2. Text open-source datasets

Google books Ngrams Yelp open 20 newsgroups

dataset

UCI’s Spambase Prediction UCI machine learning Text clas-

(Older) repository  
datasets

SQuAD Google books Broadcast news WikiText

Ngrams

Penn Treebank Reuters news Billion words dataset: Common

dataset crawl

Artificial datasets:
 Arcade Universe
 Dataset inspired from baby-AIschool
 All images and question datasets
 Deep vs. shallow comparison ICML
 Background correlation
 Rectangles data
 Mnist variations
Facial datasets (Table 3):

Table 3. Databases for face recognitions.

Labeled faces in the UMD faces annotated CASIA WebFace facial

wild dataset

MS-Celeb-1M Olivetti Multi-Pie

JACFEE FERET mmifacedb

Indian face database The Yale face database Mut1nyFace/head segmentation

dataset

Recent additions of datasets (Table 4):

 Advancements In Deep Learning Theory And Applications: ... 23

Table 4. Free databases developed recently

The UZH-FPV drone rac- North Korean missile Flickr-Faces-HQ Da-

ing dataset test database taset (FFHQ)

Hotels-50K MIMIC-CXR Google Audioset

Two new evaluation data- Open-source biometric Uber 2B trip data

sets data recognition

Yelp Open Dataset Core50 Data portals

Open data monitor Quandl data portal Mutiny face/head seg-

mentation dataset

Awesome public dataset Head CT scan dataset Open datasets

WAPo Chess dataset NLP datasets

24 Deep Learning Algorithms

REFERENCES
1. Aliper A, Plis S, Artemov A, Ulloa A, Mamoshina P, Zhavoronkov
A. Deep learning applications for predicting pharmacological
properties of drugs and drug repurposing using transcriptomic data.
Molecular Pharmaceutics. 2016;13(7):2524-2530
2. World Health Organization. Global status report on road safety.
Available from: https://www.who. int/violence_injury_prevention/
road_safety_status/2018/en/ [Accessed: 31 Janaury 2018]
3. U. D. O. Transportation. Critical Reasons for Crashes Investigated in
the National Motor Vehicle Crash
4. Causation Survey. Washington, DC: National Center for Statistics and
Analysis; 2015
5. Zhao Z-Q , Zheng P, Xu S-T, Wu X. Object detection with deep
learning: A review. IEEE Transactions on Neural Networks and
Learning Systems. 2019;30(11):3212-3232
6. Kobatake H, Yoshinaga Y. Detection of spicules on mammogram
based on skeleton analysis. IEEE Transactions on Medical Imaging.
1996;15(3):235-245
7. Top 3 enterprise tech trends to watch in 2020. 2020. Available from:
https://www.computerworld.com/ article/3512109/top-3-enterprise-
techtrends-to-watch-in-2020.html
8. Zhong S, Hu J, Fan X, Yu X, Zhang H. A deep neural network combined
    `+~?|      
for hydroxyl radical rate constants of water contaminants. Journal of
Hazardous
9. Materials. 2020;383(5)
10. Kong J, Huang J, Yu H, Deng H, Gong J, Chen H. RNN-based default
logic for route planning in urban environments. Neurocomputing.
2019;338:307-320
11. Zhang Z, Zheng L, Wang M. An exponential-enhancedtype
varying-parameter RNN for solving time-varying matrix inversion.
Neurocomputing. 2019;338:126-138
12. Konstantinidis D, Argyriou V, Stathaki T. A modular CNN-based
building detector for remote sensing
13. images. Computer Networks. 2020;138:107034
Random documents with unrelated
content Scribd suggests to you:
an eminent degree also all the qualities of a successful political
leader,—ability, tact, diplomacy, decision of character, foresight, the
statesmanlike breadth of view which soars beyond the triumphs of
the hour to grasp the necessities of the morrow, and that loyalty
which inspires confidence and renders alliances durable. As an
orator, it may be fairly said that he has few equals. Few public
speakers of his day excel him in the art of swaying an audience,
whether cultured or illiterate. He touches their feelings or appeals to
their reason with a force and a logic that always tell. A brilliant
lawyer and a perfect master of parliamentary fence, he has also
been described as belonging to that class of men who are always
ready for duty, always equipped for a fight, and his blows invariably
tell with sledge-hammer force. At the same time it must be
conceded that he is a manly fighter, never taking an unfair
advantage of an adversary, and always showing the courteous and
polished Frenchman’s aversion to unnecessarily wound the feelings
of others. His astonishing industry also constitutes one of his chief
claims to the admiration of his friends, coupled with the courage and
pluck which has carried him to victory against what at one time
appeared the most desperate odds. He has lived a busy life, divided
between journalism, law and politics; but it is mainly in his public
capacity that his assiduity and powers of application have come to
be most known and appreciated. Whether as leader of the
Opposition or of the Government, he has been and is an
indefatigable worker, always at his post and accomplishing more in a
day than other public men usually do in weeks. Another secret of his
great prestige among his fellow countrymen is to be found in his
acute and rapid perception of the drift of popular opinion in his
province, and the people’s growing confidence in the earnestness of
his patriotism. As already stated, Mr. Premier Mercier is still in the
full prime and vigour of life, his age being only forty-seven. He first
saw the light in Iberville county, in the year 1840. He comes of a
family of simple farmers, or habitants, as they are styled in Lower
Canada, originally from Old France, but settled for several
generations in the county of Montmagny, below the city of Quebec.
His father was not wealthy, and had to provide for the wants of a
large household; but he was a man of energy and foresight, and
thought no sacrifice too great to arm his children for the battle of life
by means of a liberal education. At the age of fourteen years, young
Mercier was sent to the Jesuits’ College in Montreal to complete his
education, which he finally did after a brilliant course of study; and,
even to the present day, the premier of Quebec reverts with
pleasurable recollection to his early struggles after knowledge, and
loses no occasion to testify his affectionate and grateful regard for
the masters who first taught his “young idea how to shoot.” The
ardour with which he took up the cause of the Jesuits during last
session of the Quebec legislature, and championed it to victory in
the passing of their charter bill, is largely explained by this feeling,
strengthened by the conviction that the legislature had no warrant to
refuse to one religious order the ordinary privilege of civil rights
which it had so freely granted to others. Like the vast majority of his
French Canadian fellow countrymen, the premier of Quebec is, of
course, a Roman Catholic, and imbibed a lively faith in the doctrines
of that church from his parents and the teachers of his youth. That
faith has not diminished, but increased with his maturer years. Still
there was a time, and not yet very remote either, when, on account
of his political liberalism and alliances, his orthodoxy was more than
once seriously questioned by his political foes to his personal and
party detriment. However, this has all passed away. It is now
conceded by Papal authority that a man may be a Liberal in politics
and yet a good Catholic; and the Lower Canadian clergy have come
to understand that Mr. Mercier is not only a sincere Catholic in theory
and practice, but that the interests of their church are as safe in his
hands as in those of the self-constituted champions who proclaim
their zeal for the faith from the housetops. At the same time, he is
no narrow-minded bigot. There is probably no public man in the
dominion free from religious or sectional bias. He never asks “the
brave soldier who fights by his side in the cause of mankind, if their
creeds agree.” A French Canadian in heart and soul, and a thorough
son of the soil, still strict and impartial justice to all classes, races
and creeds; undue favour to none, seems to be the motto upon
which he has always acted in the past and desires to act in the
future. Now, to return to the career of our subject. Some time after
leaving college, young Mercier decided to make the law his
profession. He accordingly entered the office of Laframboise &
Papineau, at St. Hyacinthe, and was admitted to practice in 1865.
But, three years before this event, he may be said to have entered
public life, towards which the ardent young man felt himself
irresistibly attracted. In 1862, at the age of twenty-two years, he
became editor-in-chief of the Courrier de St. Hyacinthe, and made
his mark as a vigorous and trenchant political writer. This was before
confederation, during the Sandfield Macdonald-Sicotte
administration. To that government, with its liberal and moderate
policy, and its programme of conciliation between Upper and Lower
Canada, the young journalist gave a warm support. But in the
excited state of public opinion in the two provinces at the time, the
task of pacification which it had undertaken was beyond its strength,
and after a short and stormy existence, it succumbed. At this stage
in Canadian history the political situation was exceedingly strained.
Not only were parties in the legislature about evenly balanced, but
Canadian politics were complicated by such burning and difficult
questions as the Separate Schools, Representation by Population,
and the construction of the Intercolonial Railway. Finally, despairing
of reducing this apparent chaos to order, Mr. Sicotte retired, and
Sandfield Macdonald reconstructed the cabinet by taking in from
Lower Canada Mr. Dorion, now Sir A. A. Dorion, chief-justice of the
Court of Queen’s Bench of the province of Quebec, and by openly
repudiating the principle until then recognized of the double
majority. Mr. Mercier who, in the Courrier de St. Hyacinthe, had
sustained the Sicotte administration, went over to the opposition
with his leader. He continued, with Cartier and a group of moderate
liberals, to form part of the opposition, which he then regarded as a
national opposition, and his powerful pen in the Courrier de St.
Hyacinthe contributed immeasurably to the defeat of the ministerial
candidate when the seat for St. Hyacinthe became vacant by Mr.
Sicotte’s elevation to the bench. When the confederation scheme
was broached in 1864 as the only means of cutting the Gordian knot
of the political deadlock between the united provinces of Upper and
Lower Canada, Mr. Mercier, who had supported Cartier in his
opposition to the Macdonald-Dorion ministry, felt himself unable to
approve his alliance with George Brown for the establishment of
confederation, believing that the realization of the latter would be
the death-warrant of the French Canadian influence, that the project
was only another expedient to retain power in Tory hands, and that
behind it, in the mind of Sir John A. Macdonald, lurked a long-
meditated design to force a legislative union upon the provinces. His
views, however, in this respect, were shared only by a small
minority, and he resigned in consequence the editorial chair of the
Courrier de St. Hyacinthe. But, later on, in 1865, when the project
was regularly discussed in parliament, Mr. Mercier’s objections to it
found expression through an opposition on the floor of the house;
weak in numbers, it is true, but resolute and untiring in their efforts
to render it less obnoxious to the French Canadians, and more
favourable to the rights of the provinces. All or nearly all of the
causes of friction which have since developed between the central
and the local governments in the working of the new constitution,
were then exhaustively ventilated by the liberals. They demanded,
with Mr. Holton, that the Federal Act should expressly recognise the
sovereignty of the provinces, and that only restricted and delegated
powers should be conferred on the central government. They
protested against the mode of constituting the Senate, the principle
of the nomination of the lieutenant-governors by the federal
ministry, and the right of veto upon the acts of the Provincial
legislatures. To every assault upon the integrity of the scheme,
Cartier invariably opposed the stereotyped reply that the Federal Act
was a “sacred compact,” and that not one line of it could be altered
without provoking a breach with the other provinces. This non
possumus style of argument was successful in procuring the
rejection of all the amendments proposed in the parliament of united
Canada. But it found no echo in New Brunswick and Nova Scotia,
whose legislatures, while approving the confederation principle,
refused to ratify the scheme in all its details. The whole question,
with the right of amendment, seemed to be thus thrown open anew,
and the hopes of the Lower Canadians, who looked to extract the
most protection for their province from the project, once more
revived. Meanwhile, while these events had been transpiring, Mr.
Mercier had resumed the editorial direction of the Courrier de St.
Hyacinthe in the month of January, 1866, having formed with Mr. de
la Bruère, now speaker of the Quebec Legislative Council, Mr.
Bernier, now superintendent of Education in Manitoba, and Paul de
Cazes, his brother-in-law, a syndicate whose programme, in view of
the adoption of the new constitution, was to give it fair play and to
endeavour to make the most of it, after Lafontaine’s example in
1840. The opposition of the Maritime provinces having re-opened,
de jure, the right of amendment, Mr. Mercier and his colleagues
hailed the event with gratification; but, to their surprise, in February,
1866, La Minerve, of Montreal, and other newspapers, began to
spread the rumour that the question would not be again submitted
to the Canadian legislature, and that Cartier had consented to refer
the settlement of the difficulties to Imperial arbitration. Thereupon,
the managers of the Courrier de St. Hyacinthe published an article in
which they distinctly declared that, if the principle of arbitration was
accepted, they would go into opposition. A fortnight later, Cartier
proposed to refer the pending difficulties to Imperial arbitration, and
there was nothing left to Mr. Mercier and his colleagues of the
Courrier but to execute their threat and transfer their talents and
influence to the opposition. They were unanimous on the subject,
and the article announcing their determination was prepared by Mr.
de la Bruère. But, before it could be published next morning, Messrs.
de la Bruère and Bernier, who have ever since remained
Conservatives and attached to the fortunes of Sir John A.
Macdonald, suddenly changed their views and refused to allow it to
appear. A rupture ensued between the partners, and Mr. Mercier and
Mr. de Cazes withdrew from the Courrier de St. Hyacinthe, this time
for good. There is reason to believe that the turn of events at this
stage so disgusted Mr. Mercier with politics that he resolved to
abandon them altogether. At all events he retired from public life,
and during the next five years devoted himself exclusively to the
practice of his profession as a lawyer, only reappearing on the scene
in 1871, after confederation, on the formation of the Parti National.
As the occasion and objects of this movement in the province of
Quebec may be either forgotten or not well understood at the
present day, it may be useful to recall that the attitude of the
Conservative government of Sir John A. Macdonald on the New
Brunswick Separate School question in 1871, as later on the Riel
question in 1886, provoked a split among his Conservative following
from Lower Canada. A number of bold and ardent French Canadian
spirits conceived that the opportunity was a favourable one to make
another effort for the triumph of the principles for which they had so
long and unsuccessfully battled, to set aside all party divisions and
to rally under one standard all patriotic souls, Liberal and
Conservative, in order to secure the predominance of the provincial
influence over the hybrid alliances by which a majority was
constituted and maintained in the Federal parliament. In other
words, the promoters of the national movement held that in a
confederation honestly and properly worked, the representatives of
the people should above all regard themselves as plenipotentiaries
of the provinces, and that instead of dividing into conservatives and
liberals, it was their first duty to group themselves by provinces for
the common defence of their provincial or national interests. At the
head of the new party were such men as Messrs. Holton, Dorion,
Loranger, Laframboise, Jetté, Mercier, F. Cassidy, L. O. David, and
Béique, in the Montreal district, and Messrs. Letellier de St. Just,
Joly, Thibaudeau, Langelier, Pelletier, and Shehyn, in the district of
Quebec. Their platform included protection, complete provincial
autonomy, and decentralization, vote by ballot, the trial of election
contestations by the law courts, the abolition of dual representation,
suppression of the Legislative Council, economy in the public
expenditure, and the suspension of the construction of the Canadian
Pacific Railway until the resources of the country warranted the
completion of that great work without saddling the people with the
burthens of a ruinous debt. Mr. Mercier threw himself heart and soul
into this movement, which promised to realize his dearest
aspirations. He lent powerful assistance to the election of his friend,
Hon. F. Langelier, for Bagot county, and in the following year, at the
general elections of 1872, he was himself returned as the federal
member for Rouville. On the meeting of the Dominion parliament in
1873, he took an active and leading part in the exciting debate on
the New Brunswick Separate Schools question, and, with Hon. John
Costigan from that province, then plain Mr. Costigan, he also
eloquently defended Rev. Father Michot, a Catholic priest, whose
goods had been destrained, and person imprisoned for debt by the
authorities of New Brunswick, because of his refusal to pay tax
towards the support of the Protestant schools. The result was that
the government was beaten by a majority of thirty-five through the
French Canadian vote, supported by the Liberals of Ontario; but Sir
John A. Macdonald refused to recognize this adverse decision as a
ministerial defeat, and announced his intention of referring the
question of the New Brunswick schools to the Imperial government.
A cabinet crisis was thus averted for the moment, but it was
destined to be not long delayed. The last echoes of the fierce debate
on the school question had hardly died away, when suddenly and
almost without a note of warning, the astounding revelations which
have since passed into history under the title of “The Pacific
Scandal,” were sprung upon the parliament and country. In the
midst of the most intense excitement all over the dominion,
parliament adjourned in May, 1873, and between that date and the
following August, when it was to meet again, Mr. Mercier was one of
the most active in stumping the province of Quebec against the
government, and in promoting the petition to the governor-general
against the alleged intention to prorogue the house. To the prayer of
this petition, however, Lord Dufferin did not deem it advisable to
assent, and parliament was prorogued on the very day of its
reassembling in August. But it was called again towards the end of
October, and, after a seven days’ debate, which will remain forever
memorable in Canadian annals, Sir John A. Macdonald announced
that he had placed his resignation in the hands of his excellency.
Two days later, the Liberal government of Mr. Mackenzie was formed,
followed two months later, in January, 1874, by a dissolution of the
Dominion parliament. At the general elections which ensued, Mr.
Mercier had intended to again offer as a candidate for the county of
Rouville in the interest of the new Liberal ministry; but, as another
Liberal candidate of much local influence, Mr. Cheval, also proposed
to run, he withdrew from the field rather than create a division,
which might throw the constituency into Tory hands. In 1875 he
once more reappeared on the scene in Bagot, which he stumped in
favour of Mr. Bourgeois, now a judge of the Superior Court, with
whom he had formed in 1873 one of the strongest law partnerships
in the country. In 1878, when Mr. Delorme, the Liberal member for
St. Hyacinthe, and now clerk of the Quebec Legislative Assembly,
retired from the representation of that county, Mr. Mercier manned
the breach in the Liberal interest; but was defeated by Mr. Tellier, the
Conservative candidate, who carried the seat by the narrow majority
of six votes. But for that disappointment he was consoled in the very
following year by the brilliant victory on the same ground, which
ratified his entry into the provincial government, and was the
prelude to a new and more important phase of his public life. In
March, 1879, when Hon. Mr. Joly, the then Liberal premier of
Quebec, invited Mr. Mercier to fill the cabinet vacancy created by the
death of Mr. Bachand, his ministry was virtually in a moribund
condition. It did not command a large enough majority, and above
all one sufficiently solid to survive the restoration of Sir John A.
Macdonald to power at Ottawa, after the fall of the Mackenzie
government. Coming events were already casting their shadows
before; the Letellier question, as it was called, had waxed in
bitterness; and there is little doubt that Mr. Joly and his colleagues
foresaw clearly the near approach of their own official death. But
they had resolved, for the honour of the cause and its future
interests, to fight it out bravely and worthily to the end. They
needed the help of a sturdy and experienced spirit for the purpose,
and Mr. Mercier, who did not hesitate a moment about undertaking
the task, was a few days afterwards elected to the Quebec
legislature for St. Hyacinthe by the large majority of 307 votes. As
solicitor-general in Mr. Joly’s cabinet, Mr. Mercier’s official career was
too brief to permit of his displaying more than the qualities of an
admirable law officer of the Crown; but, on the floor of the Quebec
Assembly, he at once took a foremost place as an orator, debater
and legislator. After the fall of the Joly cabinet, Mr. Mercier
momentarily entertained the idea of retiring from public life for good
and all, not that he despaired of the righteousness in his own mind
of the cause which he supported, but more probably because this
last attempt of the Liberals to capture and hold Quebec province, in
which he had been called to take a too tardy part, had strengthened
his long rooted conviction, that that party as then constituted in
Lower Canada, were acting on too narrow and defective a basis to
make successful headway against the existing combination of Tory
interests and prejudices. Accordingly, having in the meantime
removed in March, 1881, from St. Hyacinthe to Montreal, where he
had formed a new law partnership with Messrs. Beausoleil &
Martineau, he announced his intention to not come forward at the
general elections of that year. This announcement produced a most
powerful sensation throughout the province, but especially among
his constituents of St. Hyacinthe, who, regardless of their party
divisions, rose as one man to beg of him to reconsider his decision,
which he finally did after long and earnest reflection, when he was
returned once more to the legislature by acclamation. About this
period of his career, or shortly afterwards, occurred the incident of
the coalition, which came very nearly splitting up the Liberal party.
Enlightened men in the ranks of both parties in the province felt that
the existing state of things could not continue much longer; that
their public men were wasting their energies in fruitless contention;
and that ruin, political and financial, stared Quebec in the face
unless the politicians on both sides clasped hands to forget old feuds
and to form a strong coalition government on the broad national
ground which might fearlessly apply the heroic remedies demanded
by the critical nature of the situation. Mr. Mercier was all the more
open to the advances made him from the other side, both during the
administrations of Mr. Chapleau and his successor, the late Mr.
Mousseau, in favour of this new departure, that he had strenuously
advocated a policy of conciliation and union for the national good
throughout his whole public life. He probably made a mistake in
supposing that the hour was ripe for the fruition of such a policy,
and that nothing more was needed to a general conviction of its
necessity. But even so, the error was a generous one, prompted by
patriotism. The proposals for a coalition, however, did not emanate
from Mr. Mercier, but from his adversaries, that he only consented to
entertain them upon certain well defined and strictly honourable
conditions, and that in the entire business he was true to the
controlling idea of his career as to the absolute necessity of union
for the salvation of his native province. In the beginning of the
session of 1883, Hon. Mr. Joly resigned the direction of the provincial
Liberal party, and Mr. Mercier was unanimously chosen to succeed
him, on Mr. Joly’s own motion, as the leader of the opposition. In
this new and important role he at once found fitting opportunity and
scope to display the great qualities which in so brief a period have
placed him in the foremost ranks of French Canadian statesmen.
Within the short space of three years he successively showed what
an able and intrepid leader can do with the support of a small but
disciplined and trusty band of parliamentary followers, to retrieve the
fallen fortunes of his party, and to defend and lead to victory a
popular cause the moment circumstances placed it in his hands.
During the first portion of his task, Mr. Mercier maintained a struggle
which cannot be otherwise characterized than as heroic. With a
following in the House of Assembly reduced to fifteen members
against fifty, he kept in check three successive governments of his
adversaries, and if he did not succeed in defeating the two first by a
vote, he at least forced them to take flight. One after the other,
Messrs. Chapleau and Mousseau were compelled to retire from the
field, admitting themselves to be too grievously stricken to continue
the fight any longer against so sturdy an opponent, whose scathing
denunciations of their policy and administrative methods were
gradually arousing public opinion from its apathy with regard to the
financial and political dangers that seemed to threaten the safety of
the province. During this period, too, as well as during the rule of
the succeeding Ross administration, Mr. Mercier not only exerted a
mighty influence on current legislation, but proved himself the
fearless and ardent defender of provincial rights, and lost no
occasion to condemn in forcible terms what he had characterized as
the grovelling and ruinous subserviency of the provincial
conservatives to the overshadowing influence of Ottawa. His
sympathy with the cause of constitutional liberty also found strong
expression on more than one occasion in support of the Irish Home
Rule movement and against coercion, and the various resolutions of
the Quebec legislature on the subject either owed their paternity to
him or in a large measure their adoption. From the session of 1886,
the last of that parliament, the Ross ministry emerged woefully
crippled by the sustained vigour of Mr. Mercier’s assaults, and with
the outlook for the general elections complicated and darkened for
the success of the Tory cause by the Riel affair. Still, even under the
circumstances, it is doubtful whether, with the influence and active
assistance of the Ottawa government, and in the usual way, Mr. Ross
would not have carried a majority of the constituencies but for the
split in the conservative ranks and the astounding energy and ability
thrown by Mr. Mercier into the campaign, which preceded the
general elections, and which was probably the most anxious and
exciting ever fought in Lower Canada. As the accepted leader of the
new National party formed in that province out of a combination of
the liberals and conservative bolters, he not only directed the whole
movement, but personally traversed the province almost from end to
end, addressing as many as one hundred and sixty public meetings,
and everywhere making his influence felt for the promotion of the
cause. The elections came on in October, 1886, and resulted in a
victory for the Nationals. But for several months afterwards the
country was kept in a painful state of ferment by the refusal of the
Ross government to recognize their defeat or to call the legislature.
It has been charged that they spent the interval in endeavouring to
seduce the few National Conservatives elected from their allegiance
to Mr. Mercier; but, if so, they failed, and the circumstance only
tends to further attest his tact and skill as a political manager and
strategist. Finally they were compelled by the force of public opinion
to meet the representatives of the people in January, 1887, when Mr.
Mercier and his supporters met with a triumphal reception at the
provincial capital, and the popular verdict rendered against the
Tories at the polls in October was ratified by a majority of nine in the
House of Assembly on the first vote for the election of the speaker.
Still the Ross ministry would not resign until Mr. Mercier rendered
their humiliation more complete by taking the control of the house
out of their hands, and carrying the adjournment against their will,
amid one of the most exciting scenes ever witnessed in legislative
halls. In a few more hours the Ross administration had ceased to
exist. Mr. Mercier was called upon by the lieutenant-governor to form
a new cabinet, and in less than twenty-four hours more, with his
usual decision and promptitude, he had made his choice of his
colleagues, and announced it to the legislature and the country, both
of which received it with marked satisfaction. He also demanded and
obtained an adjournment of both houses until the following March,
in order to allow of his own re-election and that of his colleagues
(which took place in each case by acclamation), and to get time to
prepare his programme for the regular work of the session, when
the speech from the throne was delivered, and he publicly appeared
for the first time as leader of the Government and the Assembly.
Considering the shortness of the time at their disposal for
preparation, the policy formulated by the new government
constituted a very satisfactory instalment of the reforms which Mr.
Mercier and his friends had advocated while in opposition. Its
principal planks were the restoration of the finances to a sound
basis, the readjustment of the representation, and the better
protection of provincial rights and autonomy. The measures
proposed for the purpose by ministers, with the exception of that
relating to the readjustment of the representation which was held
over for more exhaustive study until another session, were all
sanctioned by the house, and by the end of the session the
government’s majority had materially increased in the Assembly,
while in the Crown-nominated branch, the Legislative Council, much
less partisan obstruction was encountered than had been
anticipated. Its close left him more firmly seated in the saddle than
ever, and with an addition to his prestige and popularity, which has
been since largely increased by the marvellous success of his
administration as evidenced in the settlement of the long pending
dispute with Ontario, respecting the division of the Common School
Fund, and the unusually advantageous negotiation of the new
provincial loan of three and a half millions. These and a number of
 We have endeavoured during the last session to remove the regrettable
prejudices which our enemies have succeeded in creating in the hearts of the
Protestant minority against us, and especially against myself. We did not
concern ourselves with the injustice of which we have been the victims, and
we have always been just and sometimes very liberal towards Protestants.
We were determined to revenge acts of injustice by acts of justice, and to
answer injuries by acts of kindness and words of courtesy. All the English
Protestant members of the legislature, with the exception of one, have
systematically and invariably voted against us, and have refused to grant us
that “British fair play” of which Englishmen so much boast. This conduct on
the part of the minority has not made us deviate from the right path—the
path of justice; we have been just towards the minority as if it had been
likewise just towards us, and we will continue to give it that “British fair play”
which its representatives in the legislature have so constantly refused to
accord to us. But let the Protestant minority permit me to say now, before
this immense audience, composed for three-fourths of French Canadians and
Catholics, that the National Party will respect and cause to be respected the
rights of that minority; that the National Party desires to live in peace and
harmony with all races and creeds; and that it intends to render justice to all,
even to those who refuse to render it in return.

In private life the premier of Quebec is a charming conversationalist,

and one of the most genial of companions. He has been twice
married, firstly, to Léopoldine Boivin, of St. Hyacinthe, who died
leaving one daughter; and lastly, to Virginie St. Denis, also of St.
Hyacinthe. Madame Mercier is one of the most distinguished
members of French Canadian society, and fittingly adorns the
prominent position to which she has been called by the side of her
eminent husband.

Chamberlain, David Cleveland, Insurance and

General Agent, Pembroke, Ontario, was born at Point Fortune,
province of Quebec, on the 22nd July, 1838. His father was Hiram
Chamberlain, and his mother, Elizabeth Minerva Hayes. The family
removed from Point Fortune in 1842, to a place on the Ottawa river,
a new settlement in the township of Westmeath, in Renfrew county,
then known as the Head of Paquett’s Rapids. Though at the time the
place was little better than a wilderness, Mr. Chamberlain, sen.,
began to manufacture lumber, and successfully carried on this
business until his death, which occurred in Quebec city in 1854, from
cholera. He left a family consisting of a widow and six children, the
subject of our sketch being the eldest. After securing some
education at the public school, David engaged himself as clerk with
Alexander Fraser, a lumber merchant, who, by the way, subsequently
married his sister, and with this gentleman he remained until 1868,
when he removed to Pembroke, and began business on his own
account as a merchant. He continued to trade until 1876, and then
gave up mercantile pursuits, adopting in lieu thereof a general
insurance agency. Since then he has worked hard, and has
succeeded in building up a profitable business in that line. He now
represents in that district of country twelve of the principal English
and Canadian fire insurance companies, and the Standard Life
Insurance Company of Scotland, doing business in Canada. Outside
of business, Mr. Chamberlain has taken a part in the world’s work.
He is a member of the Oddfellows’ organization; has been a school
trustee; was for twelve years a member of the High School board;
treasurer of the township of Westmeath; and at present is treasurer
of the school moneys of the town of Pembroke. He belongs to the
Methodist denomination; and in politics is a Liberal-Conservative. On
January 10, 1860, he married Martha Maria Huntington, daughter of
Erastus Huntington, and has a family of five children living.

Angers, Hon. Auguste Réal, Judge of the Superior

Court, Quebec, was born in the city of Quebec on the 4th of October,
1838. His father, F. R. Angers, was a lawyer who occupied a
distinguished position at the Quebec bar. Justice Angers studied at
Nicolet College, in the province of Quebec, and entered his father’s
office to study law. He was admitted to the bar in 1860, and
practised his profession with marked success in the law firm of
Casault, Langlois and Angers. In 1874, he was made a Queen’s
counsel. When the Hon. J. E. Cauchon resigned his seat in 1874, the
electors of the county of Montmorency elected him to represent
them in the provincial parliament. In the same year the Hon. M. de
Boucherville was called upon to form a new cabinet, and he offered
the portfolio of solicitor general to Mr. Angers, whose brilliant
reputation had marked him as a future minister. He accepted, taking
the oath on the 22nd of September, and therefore becoming a
minister without ever having occupied a seat in parliament. In 1875
Mr. de Boucherville taking a seat in the Legislative Council, the
leadership of the Assembly fell into the hands of Mr. Angers, who
became attorney-general on the 26th January, 1876. Messrs. Angers
and de Boucherville worked harmoniously together, both being
scrupulously honest and equally devoted to the public interests. At
that time the North Shore Railway, which had been talked about for
thirty years, was yet in an embryo state, private enterprise having
failed to carry out the scheme; they resolved to build the road as a
government work, with the help of the municipalities which had
voted liberal grants towards the construction of the road, Montreal
and Quebec having given $1,000,000 each. The wonderful debating
powers of Mr. Angers, and his keen foresight in looking upon this
railway as the future link of the Canadian Pacific Railway, and
probably of a direct route to the seaboard, helped to carry the
measure. Thanks to the construction of the North Shore Railway,
Montreal, the metropolis of Canada, and Quebec became de facto
the terminal points of the Canadian Pacific Railway, and since the
completion of this gigantic national highway, Montreal has added
40,000 to her population. As a legislator, Mr. Angers ranks among
the foremost representative men of the Dominion; the Electoral Act
and the Controverted Elections Act bear testimony to his intimate
knowledge of law. The former act has been universally admitted by
the courts to be superior to the Dominion act, while the latter ranks
equally high. The enquête is made before one judge only, and the
case is pleaded before three judges, whose decision is final, whereas
in the case of the Federal law, a controverted election case that can
be carried in appeal to the Supreme Court is distasteful to the
people of the province of Quebec, and an appeal invariably entails
long delays and enormous costs. The Superannuated Fund law,
providing for the widows and orphans of civil servants, is also due to
Mr. Angers. This law is now in force, and gives satisfaction to all the
parties concerned. Not the least important of the laws introduced by
Mr. Angers, and carried through the Lower House, in 1876, was the
act framed by the government concerning education, and giving
control to both Catholics and Protestants over their respective
educational matters. It was mainly due to his efforts that the new
departmental buildings were erected in Quebec, this being a
guarantee that the historic city and the capital of letters of the
Dominion will permanently retain the seat of provincial government.
Montreal and many other municipalities having failed to meet their
obligations with respect to the grants they had voted to the North
Shore Railway, a measure was introduced during the session of
1877-8, to compel these municipalities to hand over the amounts
they owed to the provincial treasury. Great importance was attached
to this measure, inasmuch as the province would have had to pay
the large amounts subscribed by the municipalities if the latter were
allowed to evade their just liabilities. This bill, however, as well as
another government measure having for its object an increase of
revenue, created some agitation in political circles. The lieutenant-
governor, Mr. Letellier de Saint-Just, a strong Liberal partisan, who
had been a bitter enemy of the Conservative party during twenty
years, dismissed the de Boucherville administration from power on
divers pretexts, proved groundless since, alleging among other
pretences, that the premier had not obtained the consent of the
Crown before introducing the two measures above mentioned. It
was shown afterwards that Mr. de Boucherville had obtained from
the lieutenant-governor a blank form for the introduction of the
government’s financial measures. On the dismissal of Mr. de
Boucherville, the subject of our sketch took the leadership of the
Conservative opposition, and caused the legislature to adopt several
votes of want of confidence in the Liberal government, with Mr. Joly
as premier. The latter appealed to the electorate, and at the general
election held on the 28th of May, 1878, Mr. Angers was defeated in
his constituency (Montmorency) by a vote of twelve. His defeat was
due partly to the united energies of the Liberal party, and partly to
the influence of the city of Quebec, whose million Mr. Angers had
endeavoured to obtain for the province, were thrown in the balance
against him. Thoroughly convinced that the conduct of the
lieutenant-governor was contrary to the usages of responsible
government, and that such a precedent would prove dangerous to
provincial rights, Mr. Angers determined to bend his energies
towards procuring the dismissal of Mr. Letellier, and when Sir John A.
Macdonald came into power at Ottawa, at the general elections of
1878, he (Mr. Angers), together with Hon. J. A. Ouimet (now
Speaker, 1887), and the late Hon. Justice Mousseau, took steps to
attain that object, and their efforts were crowned with success. In
1880, Mr. Angers was elected a member of the House of Commons
for the county of Montmorency by an enormous majority, and after
sitting one session, was elevated to the bench, to the great regret of
his friends who had every reason to believe that a brilliant political
career was still in store for him. After the election of 1886, the
provincial premiership was offered to Mr. Angers, but as his
acceptance of the post involved a question of principle, he did not
feel inclined to accept it, and on the Hon. L. O. Taillon’s resignation,
Mr. Mercier was offered the position, which he accepted. The
parliamentary career of Mr. Angers showed that as a debater he had
no superior, and few equals in the country. A generous heart, a
manly, straightforward character, an unblemished reputation,
profound legal learning, such are the sterling qualities that will make
of Mr. Angers an honour and an ornament to the Canadian bench. It
may be added that he is a Canadian, in the sense it is understood by
the men who intend to make this Dominion a great country.

Wood, Robert Edwin, Barrister, Peterboro’, Ontario, was

born on the 31st of August, 1847, in the township of South
Monaghan, county of Northumberland. His father, Robert Wood,
emigrated from Yorkshire, England, and settled in South Monaghan,
in 1833, and died in 1857. His mother was Sarah Armstrong, of
Monaghan, Ireland. Robert was educated at the Cobourg Grammar
School and Victoria College. He graduated in arts in 1873, and
immediately afterwards entered the law office of the late John
Coyne, then M.P.P. for the county of Peel. Upon this gentleman’s
death, he entered the office of the late W. H. Scott, M.P.P.,
Peterboro’, and afterwards studied with Edward Martin, Q.C.,
Hamilton. He passed his final examination in Trinity term, 1876, but
owing to the fact that only two years and nine months had elapsed
between his primary and final examination, he could not be called to
the bar until Michaelmas term of the same year. He then
commenced the practice of law in Peterboro’, in September, 1876,
and has so continued to the present. He has a large and increasing
practice, and owes his present position mainly to his own energy and
exertions. In March, 1886, upon the elevation of C. A. Weller to the
bench, he received from the Ontario government the appointment of
county crown attorney, and clerk of the peace for the county of
Peterboro’ (on the 31st of March, 1886.) Mr. Wood takes a deep
interest in Masonry, and is master of Corinthian lodge, No. 101, A. F.
and A. M. He was master of the same lodge in 1883. Prior to his
present appointment to office, he took a leading part in all
parliamentary contests, on the Reform side, principally in advocating
the principles of this party from the platform. Mr. Wood is an
adherent of the Presbyterian church. He was married on the 17th of
February, 1881, to Henrietta Frances, daughter of the late Philip
Roblin, of Rednersville, Prince Edward county, Ontario.

Flynn, Hon. Edmund James, Q.C., LL.D., Quebec,

M.P.P. for Gaspé county, is a native of the county he so ably
represents in the Quebec legislature, having been born at Percé, on
the 16th of November, 1847. His father, the late James Flynn, who
was of Irish descent, was during his lifetime a trader and farmer in
Percé, the place of his birth. His mother, Elizabeth Tostevin, was also
a native of Percé, though her father was from the island of
Guernsey, one of the English channel islands in Europe. The Hon. Mr.
Flynn was educated at the Quebec Seminary, and at the Laval
University, Quebec, graduating with honours, having taken at Laval
the degree of master-in-law (LL.L.), in July, 1873. And Laval again, in
1878, presented him with the degree of LL.D. He adopted law as a
profession, and in September, 1873, he was called to the bar of
Quebec, and has ever since continued to practice as barrister, etc., in
the ancient capital. Previous to this time, he, from 1867 to 1869,
held the positions of deputy-registrar, deputy-prothonotary, deputy-
clerk of the Circuit Court of the Crown and of the Peace, for the
county of Gaspé, conjointly with that of secretary-treasurer of Percé
municipality. He has been a professor of Roman law in Laval
University since 1874. From the 29th of October, 1879, to the 31st of
July, 1882, he was commissioner of Crown Lands for the province of
Quebec; commissioner of Railways, from the 11th of February, 1884,
till July, 1885, and solicitor-general from 12th May, 1885, till the 20th
of January, 1887. The Hon. Mr. Flynn was made a Queen’s counsel in
1887. He has taken an active part in political affairs for the past
fourteen years, and has been a candidate at eight different elections
for Gaspé county. First in 1874, when he presented himself as a
candidate for a seat in the House of Commons at Ottawa, but
afterwards withdrew from the field when he was made a professor in
Laval University, considering it incompatible to hold both offices.
Again in 1875 and 1877, for the Quebec legislature, when he was
defeated after a very severe contest, there being only small
majorities against him, especially in 1877. This election he
contested, and unseated his opponent; and the following year, on
the 29th of April, he was elected by acclamation. On his entering the
Chapleau cabinet in the fall of 1879, as commissioner of Crown
Lands, he was again elected by acclamation. At the general election
held in 1881, Mr. Flynn was once more elected by acclamation. On
his accepting office in the Ross cabinet in 1884, which necessitated
an appeal to the electorate, he was stoutly opposed by Major John
Slous, but he beat this gentleman by a majority of 988 votes. At the
general election held in October, 1886, he once more presented
himself for election, and was returned by acclamation by his old
friends at Gaspé. The Hon. Mr. Flynn has always been in principle a
Liberal-Conservative. By his struggles in the county of Gaspé, he has
succeeded in securing for the electors complete freedom and
independence in the exercise of their franchise, which had been
affected by the interference of certain large commercial firms. In the
legislature the part played by Hon. Mr. Flynn has been most
prominent as regards constitutional questions in particular. He has
won for himself the well-deserved reputation of being a strong and
energetic upholder of constitutional liberty; in proof of this it will
suffice to refer to his noble and manly defence of the liberty of the
press in the case of the Nouvelliste, in 1885, and his most eloquent
speech on the question of Home Rule for Ireland, etc. His attention
has been given to many other subjects of importance, such as that
of colonization, which he has always and ever endeavoured to
promote. He is the author of a homestead law for the benefit of
settlers. His administration of the crown lands was marked with an
increase in the revenue, increase in the value of timber limits,
mineral lands,—and by many useful rules and regulations, calculated
to promote colonization and the welfare of the many persons in the
province who are occupiers of crown lands. Many other important
measures were framed by him and carried through the legislature
through him, namely: The Quebec General Mining Act of 1880;
several acts concerning the crown lands, railways, the protection of
forests, and encouragement of planting of trees, etc. He has also
always taken a most lively interest in the question of the
construction of a railway from Metapedia, on the Intercolonial
Railway to Paspébiac and Gaspé Basin. Grants in land were secured
in 1882, whilst he was commissioner of Crown Lands, and the same
were converted into money grants under his auspices as
commissioner of railways. He believes that in the construction of this
line rests the future welfare of the population of the Gaspé
peninsula. His travels have been always directed towards the
acquisition of a complete knowledge of Canada, and the different
parts thereof. In religion he is a Roman Catholic. He was married on
the 11th May, 1875, to Maria Mathilde Augustine, daughter of
Augustin Coté, editor of Le Journal de Québec, and niece to the late
Hon. Joseph Cauchon, heretofore lieutenant-governor of Manitoba,
etc. He has had eight children, of whom six are still living. He resides
in Quebec city.

Hanington, Hon. Daniel L., Q.C., M.P.P. for the county

of Westmoreland, residence, Dorchester, New Brunswick, was born
at Shediac, N.B., on the 27th June, 1835. His father, Colonel
Hanington, was for long years a member of the Assembly and
Legislative Council of New Brunswick; and his mother Margaret
Peters, a daughter of William Peters, a U. E. loyalist, who for years
represented Queens county in the New Brunswick legislature. Daniel,
the subject of this sketch, received a Grammar School and academic
education at Shediac and Sackville, in his native county. After leaving
school he commenced the study of law with Charles Fisher, attorney-
general, of Fredericton, and finished with Judge A. L. Palmer, of
Dorchester; was called to the bar of New Brunswick in 1861; and on
the 11th November, 1881, was appointed a Queen’s counsel. Mr.
Hanington has been very successful in his profession, and has a
large practice in the courts of his native province, and as counsel in
Nova Scotia and in the Supreme Court of the Dominion. From 1867
to 1870 he occupied the position of clerk of circuits and clerk of the
county court of Westmoreland, when he resigned those offices to
contest the election of that year, and was chosen to represent the
county of Westmoreland in the New Brunswick House of Assembly.
He sat therein until the summer of 1874, when on again appealing
for re-election, he was defeated on the “Bible and religious
instruction in the Common Schools” question, which he advocated.
However, he was again chosen at the general elections of 1878,
1882, and 1886, to represent his old constituency. In July, 1878, he
was appointed a member of the Executive Council; and on the 25th
May, of the year 1882, he became premier. In February, 1883, he
resigned office with his colleagues. Mr. Hanington has always taken a
deep interest in educational matters, and for about seventeen years
was a school trustee. In politics he is a Liberal of the old New
Brunswick school of politicians; is a supporter of the Liberal-
Conservative government at Ottawa, and took an active part in the
last Dominion election. He is an adherent of the Episcopal church,
which he represents in the Diocesan and also the Provincial Synod.
In October, 1861, Hon. Mr. Hanington was married to Emily Myers,
daughter of Thomas Robert Wetmore, barrister-at-law, and judge of
probate, Gagetown, N.B. The fruits of this marriage have been seven
children, three sons and four daughters.

Mellish, John Thomas, M.A., Halifax, Nova Scotia, was

born at Pownal, Prince Edward Island, on January 26th, 1841. He is
the eldest son of the late James Lewis Mellish, of the same place,
and Margaret Sophia, his wife, daughter of John Murray, formerly of
Tullamore, Ireland; grandson of Thomas Mellish, known in his day as
“a most loyal British subject, and a devoted adherent of the Church
of England;” and great grandson of Thomas Mellish, an officer of the
British army, and member of an old and highly respectable English
family, who settled on Prince Edward Island in 1770. Captain Mellish
was for many years provost marshal or sheriff of the island, collector
of customs, and a member of the Legislative Assembly. An
interesting trial took place at Charlottetown, in the early part of
1779, arising from his seizure of the convoy ship Duchess of Gordon,
for smuggling. He took an active interest in the defence of the
colony during the American war, and was on military duty, assisting
in raising troops at Halifax and Fort Cumberland, during the winter
of 1779-80, returning to the island in the following spring. James
Lewis Mellish, the father of John Thomas Mellish, died on the 14th
June, 1886, in the seventy-ninth year of his age. His mother, a native
of New York, was a daughter of the late James Lewis Hayden, J.P., a
loyalist, who removed from Shelburne, N.S., to the island in 1785,
having left New York in 1783. A newspaper extract says: “We have
to record the death of one of our oldest and most highly respected
citizens. James L. Mellish, Esq., late of Pownal, departed this life on
the 14th inst., at the residence of his son, Stewiacke, N.S., whither
he had gone a short time before on a visit. His remains were
brought home for interment. In his death the community loses a
most worthy and upright citizen. Energy, strength and integrity were
united in his character. From his youth up he was a devoted and
active member and office-holder of the Methodist church. He spent
his life for the most part on his farm at the place of his birth. Mr.
Mellish married, March 25th, 1840, Miss Margaret Sophia Murray, a
lady of strong mind and superior attainments, of whose
companionship he was deprived by death about ten years ago. Their
married life was blessed with ten children, each one of whom is to-
day occupying a position of usefulness and responsibility.” John
Thomas Mellish, the subject of our sketch, was educated at Prince of
Wales College, Charlottetown, and Mount Allison College, Sackville,
New Brunswick, and holds from the latter the degrees of B.A. and
M.A. On the opening of Cumberland County Academy, Amherst,
Nova Scotia, in 1865, Mr. Mellish, who had been teaching at
Guysborough, was selected to fill the position of head master, but
resigned in 1870, in order to accept a situation in Mount Allison
College and Male Academy, and was head master of this academy
from 1871 to 1874. In the latter year, he was appointed on his own
terms to the principalship of Albro Street School, Halifax, the largest
school in the province. At the close of 1880, the strain of constant
school work on Mr. Mellish’s health compelled him to place his
resignation in the hands of the Halifax school commissioners,
although that body the year before had raised his salary for the third
time, and designated him to the position in the High School, vacated
by the late Dr. H. A. Bayne, on his appointment to the Royal Military
College, Kingston. Official records and reports testify to the great
value of Mr. Mellish’s services in the cause of education. The
Superior School grant was awarded to him when at Guysborough,
his school being ranked as best in the county. While in charge of the
academy at Amherst, he prepared a large number of students to
matriculate in the different colleges, and a still larger number to pass
the examinations for teachers’ licenses, from the academy or grade
A license down. The last year he was at Mount Allison, it was found
necessary to add six additional dormitories, in order to accommodate
the increased number of boarders in the academy. Mr. Mellish has in
his possession not less than a dozen complimentary addresses and
quite a number of pieces of plate, books, &c., presented to him by
his pupils, on anniversary and other occasions. In the summer of
1874, he made the tour of Great Britain and Ireland, and has since
delivered on many occasions a lecture entitled, “My Visit to
Scotland.” He frequently lectures on different subjects, and
contributes to the newspaper press; is the author of various papers
and pamphlets on educational and kindred topics, and of several
papers on scientific subjects, published in the Transactions of the
Nova Scotia Institute of Natural Science; is a member of the
institute, and was associate secretary with the Rev. D. Honeyman,
D.C.L., in 1875-80; has been president of the Teachers’ Institutes, at
different places; is a magistrate, and a local examiner of the
University of London; was for several years a vice-president of the
Halifax Young Men’s Christian Association; and is a lay preacher of
the Methodist church. Mr. Mellish married, July 18th, 1867, Martha
Jane, only surviving daughter of the late Benjamin Chappell, of
Charlottetown. They have six children living,—Arthur, Alfred Ernest,
Mary Sophia, Anne Elizabeth, Martha Louise, and Frances, and one,
John Thomas, died in infancy. All the children old enough are going
to school. Arthur belongs to No. 3 Co., 82nd battalion militia, and
with his company was called out and ordered to proceed to the front
during the North-West rebellion. Every preparation was made to
start, but after the company had been in barracks about a fortnight,
the order was countermanded on account of the capture of Riel. Mr.
Mellish has four brothers and five sisters: Rev. I. M. Mellish,
Methodist minister, Nova Scotia conference, formerly captain in
reserve militia; H. Pope, farmer, Stewiacke; James Roland, chief
agent British American Book and Tract Society, Halifax; Humphrey,
mathematical master, Pictou Academy, B.A., of Dalhousie College,
matriculated with honours, first division in London University; Anne,
wife of J. L. Archibald, J.P., of Halifax; Catharine Douglas, wife of
Philip Large, Charlottetown; Mary (widow of the late A. N. Archibald,
of Halifax), chief preceptress Mount Allison Ladies’ College, Sackville,
New Brunswick; Martha Janet, and Margaret Elizabeth, unmarried.
 Moody, Rev. John Thomas Tidmarsh, D.D.,
Rector of Yarmouth, Nova Scotia. This deceased divine was born at
Halifax, Nova Scotia, on the 25th of March, 1804. His father, John
Moody, who was one of the earliest merchants of Halifax, was born
in New York, June 19th, 1779. His great grandfather, John Moody,
was born in London, and also his grandfather, Thomas Moody, were
loyalists. The family came to Nova Scotia about the year 1783. His
mother was Mary R. Tidmarsh, of Halifax. His parents were married
in 1800, and both lived to a great age, Mr. Moody to his 92nd and
Mrs. Moody to her 86th year. The Rev. Dr. Moody received his
education at King’s College, Windsor; took his B.A. degree in 1824;
M.A. in 1833, and had the degree of D.D. (hon.) conferred, at the
Encœnia of 1883, only a few months before his death. He was
ordained deacon by the Rt. Rev. John Inglis, D.D., bishop of Nova
Scotia, who also ordained him priest in the following year.
Immediately after his ordination as deacon, he was appointed to the
rectorship of Liverpool, N.S., where he succeeded the Rev. W.
Twining, the first rector, and had charge of this parish for nearly
twenty years. His work was largely of a missionary character
throughout the county of Queens; and he was also chairman of the
board of school commissioners during that time. Before leaving
Liverpool he had the satisfaction of seeing his parish church much
enlarged, two chapels and several school-houses erected in the rural
districts, and the communicants increased from 19 to 200. Rev. Dr.
Moody’s second appointment was that of rector of Yarmouth, in
1846. This position he held to the time of his death, which took
place, suddenly, of apoplexy, on the morning of the 18th of October,
1883. During this period he saw the number of communicants in his
parish more than trebled, and baptized, during his ministry of fifty-
one years, considerably over 2,000 persons. The present parish
church, which bears the name of Holy Trinity, was consecrated in
1872 by the late Rt. Rev. Hibbert Binney, D.D., bishop of Nova
Scotia. It is a very handsome brick structure, in the early English
period of architecture, and will seat about 700 persons. There are
also two handsome school-houses in this parish. The church
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.

More than just a book-buying platform, we strive to be a bridge

connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.

Join us on a journey of knowledge exploration, passion nurturing, and

personal growth every day!

ebookbell.com