Received January 12, 2020, accepted January 31, 2020, date of publication February 13, 2020, date of current

version February 26, 2020.

Digital Object Identifier 10.1109/ACCESS.2020.2973658

Northern Maize Leaf Blight Detection Under

Complex Field Environment Based on
Deep Learning
JUN SUN , YU YANG , XIAOFEI HE , AND XIAOHONG WU
School of Electrical and Information Engineering, Jiangsu University, Zhenjiang 212013, China
Corresponding author: Jun Sun (sun2000jun@sina.com)
This work was supported in part by the Priority Academic Program Development of Jiangsu Higher Education Institutions under
Grant PAPD, in part by the Project Establishment of Scientific Research Topics for College Students under Grant 18A099, in part by the
Project of Agricultural Equipment Department of Jiangsu University under Grant 4121680001, and in part by the Synergistic Innovation
Center of Jiangsu Modern Agricultural Equipment and Technology under Grant 4091600030.

ABSTRACT Northern maize leaf blight is one of the major diseases that endanger the health of maize. The
complex background of the field and different light intensity make the detection of diseases more difficult.
A multi-scale feature fusion instance detection method, based on convolutional neural network, is proposed
to detect maize leaf blight. The proposed technique incorporates three major steps of data set preprocessing
part, fine-tuning network and detection module. In the first step, the improved retinex is used to process
data sets, which successfully solves the problem of poor detection effects caused by high-intensity light.
In the second step, the improved RPN is utilized to adjust the anchor box of diseased leaves. The improved
RPN network identifies and deletes negative anchors, which reduces the search space of the classifier
and provides better initial information for the detection network. In this paper, a transmission module is
designed to connect the fine-tuning network with the detection module. On the one hand, the transmission
module fuses the features of the low-level and high-level to improve the detection accuracy of small target
diseases. On the other hand, the transmission module converts the feature map associated with the fine-tuning
network to the detection module, thus realizing the feature sharing between the detection module and the
fine-tuning network. In the third step, the detection module takes the optimized anchor as input, focuses
on detecting the diseased leaves. By sharing the features of the transmission module, the time-consuming
process of using candidate regions layer by layer to detect is eliminated. Therefore, the efficiency of the
whole model has reached the efficiency of the one-stage model. In order to further optimize the detection
effect of the model, we replace the loss function with generalized intersection over union (GIoU). After
60000 iterations, the highest mean average precision (mAP) reaches 91.83%. The experimental results
indicate that the improved model outperforms several existing methods in terms of greater precision and
frames per second (FPS).

INDEX TERMS Northern maize leaf blight, disease detection, transmission module, retinex, single shot
multiBox detector (SSD).

I. INTRODUCTION However, maize is usually suffered from Northern leaf blight

Maize is one of the major food crops in the world. The (NLB). In recent years, the decrease of maize yield caused
planting area and output of maize in the world are only lower by NLB has been steadily increasing [2] Therefore, it is
than that of wheat and rice [1]. In addition to be an excellent extremely important to ensure the accurate detection and
feed for animal husbandry, maize is also an important raw identification of maize leaf blight. The disease is not easy
material for the development of light industrial products. to be detected in the early period, showing water-stained
cigar-shaped spots, gradually spread to leaf sheath. At the
The associate editor coordinating the review of this manuscript and later period of the disease, the whole plant loses their vitality,
approving it for publication was Liandong Zhu. leading to the reduction of maize yield. Traditional maize

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/
VOLUME 8, 2020 33679
 J. Sun et al.: Northern Maize Leaf Blight Detection Under Complex Field Environment Based on Deep Learning

disease detection mainly relies on the experience of agricul-

tural experts and the expertise of plant pathology. Misjudg-
ment of the disease often leads to a large number of inaccurate
pesticide application, which not only leads to environmental
pollution, but also increases the pesticide content of maize.
With the widespread use of machine vision, many scholars
use the method of machine vision to study disease detec-
tion. Hyperspectral imaging combined with chemometrics [3]
was successfully proposed to identify the rice sheath blight
disease; Support vector machine (SVM) classifiers [4] were
developed for distinguishing infected and healthy seedlings.
Deep belief networks [5] in the construction of robust meth-
ods was applied to precision agriculture. All the above liter-
atures used traditional target detection methods to manually
select the characteristics of diseased leaves for segmentation
and detection. Although the detection accuracy is relatively FIGURE 1. Examples of northern leaf blight. Images are under different
high, the color and texture of manual calibration are sub- backgrounds.
jective that may still affect the objectivity of the disease
detection. to improve the detection effect of small targets. General-
Convolutional neural network (CNN), a popular method of ized Intersection over union (GIoU) [18] was adopted to
target detection, has a wide application prospect in the field redefine the original loss function for increasing detection
of crop disease detection [6]–[9]. As a kind of machine learn- accuracy. The images with maize leaf blight in the field are
ing, CNN can achieve the purpose of accurate detection by detected by the improved model, and the detection results
training a large number of images. CNN does not depend on was compared with the traditional single shot multiBox detec-
specific features, and has a good detection effect in the field of tor (SSD) model to provide reference for the accurate detec-
generalized identification, such as target detection [10], [11], tion of maize leaf blight.
target segmentation [12], and target recognition [13], [14].
Zhang et al. established a three-channel convolutional neural II. DATA SOURCE
network for the detection of vegetable leaf diseases according The NLB data set (https://osf.io/p67rz/) is produced in
to the different colors of diseased leaves, and the detection response to the terrible disease, which is the largest open data
accuracy reached 87.15% [15]. Ma et al. [16] proposed a deep set on NLB. Each image is calibrated by human plant patholo-
convolution neural network DCNN to identify and detect four gists and has high accuracy. The NLB data set includes three
cucumber diseases. In order to reduce the over-fitting of the different parts. The first part is the hand-held set, which is
model, the data enhancement method is used to expand the taken by hand. The second part is the boom set, which is
experimental data set. DCNN obtain good detection accuracy taken by mounting the camera on a 5 meters boom. The last
for anthracnose, downy mildew, powdery mildew, and target data set is unmanned drone set, which is taken by mounting
leaf spots from 14,208 images. Srdjan et al. [17] established the camera on a DJI Matrice 600. The hand-held part of the
a plant disease recognition model based on leaf image clas- data set has a higher clarity. Thus, this part is chosen as the
sification using deep convolutional neural network, and its data set in this paper, including 1019 images with different
detection accuracy reached 91%. The above literature proves angles and backgrounds, 7669 annotations. Typical images
that it is feasible to detect crop diseases by convolutional neu- are shown in Fig. 1.
ral networks. However, the above data sets are all collected The number of images in the hand-held data set is small,
from the background of the laboratory (only a single leaf or which may affect the training effect. Besides, the uneven
a single background), and the detection performance is quite classification of the disease sample labels may affect the
different from that of the images taken in the field. In addition, stability of the model. Raw data sets mainly go through
different from the detection of other diseases, the spot area of two aspects of data enhancement process. First, the pho-
maize leaf blight at the initial pathological stage is small and tometric distortions, including random brightness, random
difficult to detect, so the accuracy of detection requirements lighting noise, and random contrast, hue, saturation. Second,
for small targets is relatively high. the geometric distortions, including random crop, random
To solve the problems that high-intensity light interfered expand, and random mirror. The above two categories of
with disease detection in the field and the traditional model is data enhancement methods are carried out with a probability
insensitive to small-target disease detection, this paper added of 50% and get 8152 images in total. The augmentation
retinex model with low-pass output to preprocess the data operation does not change the original number of annotations,
set, thus the data set has a higher degree of identification. which also ensures the integrity of the data set. The data set
Meanwhile, the multi-scale feature fusion and the fine-tuning is divided into the training set, validation set, and testing set
network of anchor box was used in the detection network by 5:4:1.

33680 VOLUME 8, 2020

J. Sun et al.: Northern Maize Leaf Blight Detection Under Complex Field Environment Based on Deep Learning

III. MAIZE LEAF BLIGHT DETECTION MODEL STRUCTURE

Target detection includes one-stage and two-stage detection
methods. The main idea of the one-stage method is to use
the multi-scale method to perform intensive sampling on the
image, and then take advantage of the convolutional neural
network to extract features after the classification and regres-
sion. There is no extraction of the region proposal, so the
speed advantage is demonstrated. As a representative of the
single-stage target detection algorithm, SSD [19] improves
the detection effect of targets through the anchor of different
scales. But it uses low-level feature maps to detect small
targets, making the detection of small targets not ideal. The
uniform sampling of SSD also leads to the imbalance of
positive and negative samples, which leads to a decrease in FIGURE 2. The schematic diagram of retinex.
the accuracy of model detection.
To the best of our knowledge, most of the researches focus
on the extension of the data set, whereas ignore the specific
optimization to the data set problems. The data set studied
in this paper is taken in the field with high light intensity,
causing the appearance of ‘reflection’ phenomenon in some
images. Therefore, it is difficult to detect the diseased posi-
tion clearly. The improved retinex [20] is used to optimize
the original data set, making the images adjusted to visual FIGURE 3. Samples of original and optimized pictures. Images a-d are
acceptance range [21] for better detection results. examples with strong reflection in the data set; Images e-h are the
The aim of this study is to solve the problems of poor corresponding images processed by retinex.

detection effects caused by high-intensity light, poor detec-

tion effect for small targets [22], and inaccurate reflection
of the loss function in SSD. Thus, this paper makes three The two variables are separated by taking the logarithm
improvements based on SSD: of the reflected image R (x, y) and the luminance image
L (x, y). The formula is as follows:
• The data set is preprocessed by the improved retinex to
deal with the problem of high-intensity light; logS(x, y) = logR(x, y) + logL(x, y) (2)
• A two-stage structure is used to deal with the problem of
class imbalance while adding multi-scale feature fusion Then, by calculating the weighted average of pixel points
to improve the detection of small targets; (x, y) in the image and pixel points (x, y) in the surround-
• GIoU is adopted to optimize the original loss to improve
ing area, the change of luminance L (x, y) is estimated and
detection accuracy. removed in the original image S (x, y). Thus, the reflected
value R (x, y) in the original image S (x, y) is preserved.
A. RETINEX WITH LOW-PASS OUTPUT The specific transformation process is as follows. Firstly, the
Data set preprocessing is an important part of deep learning. original image is convolved by a high-pass filter function
The filter function of the single-scale retinex [23] is modified to obtain a high-pass filtered image H (x, y), and M (x, y)
to solve the problem of high-intensity light in this paper. The represents a high-pass filter function. It is defined as:
high-pass filter is used to instead of the original Gaussian H (x, y) = S(x, y) ⊗ M (x, y) (3)
low-pass filter to obtain a low-pass output image, which
reduces the reflection of the image. Retinex theory points out The high-pass filtered image H (x, y) is subtracted from the
that the color of an object is determined by the reflection original image S (x, y) to obtain the low-frequency weakened
ability of the object, rather than by the absolute value of image D (x, y). The definition can be normalized as follows:
the intensity of external reflected light [24]. In other words, D(x, y) = logS(x, y) − logH (x, y) (4)
the color of the object is not affected by the non-uniformity
of reflection and has universal consistency [25]. The formula Finally, an antilog is used to the low-frequency image, and
can be expressed as: the image R (x, y) with appropriate reflection is obtained. The
result is shown as follows:
S(x, y) = R(x, y) ∗ L(x, y) (1)
R(x, y) = expD(x, y) (5)
where S (x, y) represents a given image, R (x, y) represents a In this paper, the retinex is performed on the image with
reflected image, L (x, y) represents a luminance image, and strong reflection in the data set. The specific effect is shown
(x, y) represents each point in the image. in Fig. 3.

VOLUME 8, 2020 33681

 J. Sun et al.: Northern Maize Leaf Blight Detection Under Complex Field Environment Based on Deep Learning

FIGURE 4. The improved RPN network.

B. MULTI-LAYER INPUT RPN NETWORK more location and detail information. However, due to less
With the addition of region proposal network (RPN), the divi- convolution, they have lower semantics and more noise.
sion of anchor box is more detailed. However, in practical High-level features have stronger semantic information, but
application, the efficiency and precision are not enough in their resolution is very low and their perception of details is
the modified SSD with RPN network. Because a feature map poor. How to combine them efficiently is the key to improve
generated more than 45,000 anchor boxes. A large number of the accuracy of detection model.
anchor boxes are located in the background and need to be Transmission module (TM) is designed to improve the
filtered in the next step. Therefore, it is necessary to adjust detection effect on small targets and detection efficiency in
the structure of RPN to detect disease areas effectively. this paper. The feature map associated with the anchor is
The two-stage method has solved the problem of class adopted to fused feature by transmission module. As shown
imbalance well. As is shown in Fig. 4, the three-layer con- in Fig. 5, firstly, two 3 × 3 convolutions are carried to the
volution of the original RPN [26] network is replaced by one feature map and one 4×4 deconvolution is used to expand the
Kernel (size = 3×3, Channel = 1024), two Kernel (size = high-level feature map, then they are subjected to elemental
1 × 1, Channel is 1024 and 256), and a four-layer convolution summation to achieve the purpose of feature fusion. In order
of Kernel (size = 1 × 1, Channel = 512). Convolution to ensure the identifiability of the detection features, one 3×3
calculation in RPN network is adopted to slide on the feature convolution is added to the summed feature map. The module
map. Meanwhile, a series of region proposals are sent out to refines the features and sum the corresponding elements with
provide better initial information for the detection network. the deep features. The network takes the summation result
In this paper, the 320∗ 320 size of feature map is taken as as the feature of the current layer to the detection module,
an example. To deal with the problem of diseased position and it solves the problem that low-level feature used in the
with different scales, the anchor is extracted on four feature traditional SSD is insufficient. Thus, the detection accuracy
layers from the input feature map, stride sizes 5, 10, 20, of small target is improved. The fine-tuning network will
and 40 pixels are chosen as four feature layers. Each feature only send the anchors judged as target disease to the detec-
layers are combined with four different scales (20, 40, 80, tion module through the transmission module, thus realizing
160) and three kinds of aspect ratio (1:1, 1:2, 2:1). Finally, the feature sharing between the detection module and the
12 anchors with different sizes are generated. We follow the fine-tuning network.
design of anchor scales over different layers, which ensure
that anchors of different sizes can have the same density D. GENERALIZED IOU
on the image [27]. In this study, the largest IoU values Smooth-L1 is used to optimize the bounding box of the SSD.
and the samples with IoU > 0.5 are selected as positive The loss measured by distance does not fully reflect the actual
samples. Meanwhile, all anchors with negative confidence detection situation of the detection box. As shown in Fig. 6,
> 0.99 are removed, that is to say, most background anchors when the three norm values reach the same value, there is a
are removed. As a result, the complexity of the model was big difference in the actual detection effect (a big difference
reduced. The problem of class imbalance is alleviated and the in the IoU). The phenomenon indicates that the distance norm
testing time is shortened [28]. cannot accurately reflect the real detection effect. The effect
of the target detection directly affects by the accuracy of the
C. TRANSMISSION MODULE bounding box regression. Thus, the IoU-based loss can not
In many researches, fusing features [29], [30] of different only accurately reflect the detection effect of the bounding
scales is an important measure to improve detection perfor- box and the ground truth, but also has the scale invariance.
mance. Low-level features have higher resolution and contain Therefore, the accuracy of target detection can be effectively

33682 VOLUME 8, 2020

J. Sun et al.: Northern Maize Leaf Blight Detection Under Complex Field Environment Based on Deep Learning

FIGURE 5. Transmission module.

(3) When A and B do not intersect, the gradient is not 0 due

to the introduction of variable C, and the optimization can be
continued.

E. MULTI-INPUT RPN NETWORK COMBINED WITH

MULTI-SCALE FUSION DISEASE DETECTION MODEL
In this section, Fig. 7 shows the whole NLB detection model
based on multi-input RPN network and multi-scale fusion.
The model consists of the improved RPN network [32], [33]
FIGURE 6. IoU of the same norm. The black box indicates the bounding and transmission module, a total of 11 layers, and the Softmax
box and the orange box indicates the ground truth. When the distance
norm (k.k) is 6.5, the actual IoU is different (IoU is 0.62, 0.91, 0.34). is adopted to be the classifier [34]. The RPN network replaces
the original classification layer with the multi-scale fea-
improved by using the IoU as a loss function instead of the ture input network (conv1, conv2, conv3, conv4) for anchor
original smooth-L1. Using the IoU [31] as a loss function fine-tuning, and use the 4-layer convolution (conv5, conv6,
requires solving the following two problems: conv7, conv8) as the detection layer. The transmission mod-
(1) When there is no coincidence between the bounding ule includes two convolution layers and one deconvolution
box and the ground truth, in other words, IoU = 0, the gradi- layer (Conv9, Conv10, Deconv11), which is not shown in
ent is 0, it cannot be optimized. the figure due to the limits of picture size. Considering that
(2) When the bounding box coincides with the ground different parameter settings will affect the accuracy of the
truth, the detection effect is different. model, the mean Average Precision of one stage model (SSD)
Based on the excellent characteristics of IoU and its short- is compared with the mAP of the new models under images
comings as a loss function, GIoU is proposed to solve the of different sizes and different loss evaluation indicators.
problems. The loss function of original SSD is optimized 1 X
by GIoU in this paper. First, the IoU is calculated by the mAP = AP(q) (8)
|QR |
conventional method. q∈QR

|A ∩ B| where QR is the number of all categories; AP(q) is the average

IoU = (6)
|A ∪ B| precision value of the detection.
In the formula (6), A, B are the bounding box and the ground
truth, and A, B belong to the set S (S is all the boxes) IV. MODEL TRAINING
(0≤IoU≤1). A minimum closed shape C (C ⊆ S) is intro- A. EXPERIMENTAL PLATFORM
duced. The experimental platform is the Ubuntu 16.04 system,
|C\A ∩ B| which uses the Caffe as deep learning framework. The
GIoU = IoU − (7) computer memory is 16GB, and it is equipped with
|C| Intel @CoreTMi7-7 700KCPU@4.00GHzx8 processor. Two
According to the definition of GIoU, it can be seen that: NVIDIA GTX1080Ti GPUs are used in the experiment. The
(1) GIoU also has scale invariance. memory type is GDDR5, with a capacity of 11GB, and core
(2) There is a symmetric range for GIoU (−1≤GIoU≤1). frequency is 1480-1582MHz.

VOLUME 8, 2020 33683

 J. Sun et al.: Northern Maize Leaf Blight Detection Under Complex Field Environment Based on Deep Learning

FIGURE 7. Detection model of NLB. Conv1, Conv2, Conv3, and Conv4 corresponding to different input image (512∗ 512, 320∗ 320) is
64∗ 64, 32∗ 32, 16∗ 16, 8∗ 8(40∗ 40, 20∗ 20, 10∗ 10, 5∗ 5), the number of channels is 512, 512, 1024, 512 (512, 512, 1024, 512). P4 is the
highest-level input (no deconvolution) obtained from the feature map after three convolution cores (size 3∗ 3, step size 1, channel
256) and pooling. P3 is obtained from the feature map after convolution, pooling, and the sum of elements after deconvolution with
P4. P2 and P1 are the same process.

B. SETTING OF TEST PARAMETERS TABLE 1. Model parameter setting and test accuracy.

Batch training combined with momentum factor method is

used to divide the training set and test set into several batches,
and 16 images are trained in each batch. The number of
iteration is 60000. The stochastic gradient descent (SGD)
is adopted [35]. The initial learning rate is 0.001, grad-
ually reducing it to the previous 1/25 in stages, and the
weight-decay is set to 0.0005 to prevent over fitting.

V. EXPERIMENTAL RESULTS
A. COMPARISON OF DISEASED POSITION
In this section, we not merely shows the conclusions obtained
by training the images of the different sizes, but also com-
pare with the result of traditional SSD, which has been
trained from two different data set. The mean Average Pre-
cisions (mAP) and the Frames Per Second (FPS) of the
models are listed in Table 1. As a result, these improvements by high-intensity light. The mAP of SSD is improved from
have proved to be effective for improving the performance 71.8% to 75.42%. The accuracy of Data set B produces
of the new model. In the following parts, the impact of 5.31% higher than the accuracy of Data set A in model 5,
these improvements in the overall network framework will and the mAP improves by 2.26% in model 6. In general,
be analyzed. the accuracy of Data set B is higher than that of Data set A.
In Fig. 8, part of the detection accuracies (model 6) are shown
B. THE EFFECT OF IMPROVED RETINEX MODEL ON MAP in Data set A and Data set B. In view of the specific problems
Comparing the mAP of Data set A with that of Data set B of the data set in this study, the improved retinex model
in each model, it can be concluded that the retinex greatly effectively solve the problem that the disease position is not
improves the problem of poor detection accuracy caused obvious.

33684 VOLUME 8, 2020

J. Sun et al.: Northern Maize Leaf Blight Detection Under Complex Field Environment Based on Deep Learning

FIGURE 8. Detection accuracy of Data set A and Data set B under model 5. Images 1-3 and Images 4-5 show the detection effects of Data set A and Data
set B under model 6 respectively.

FIGURE 9. Comparison of detection accuracy between Model 2 and Model 4.

C. THE EFFECT OF TRANSMISSION MODULE the model 2 (512∗ 512) in Data set B, the model 4 (512∗ 512)
COMBINED WITH RPN NETWORK ON MAP achieves 13.29% mAP. The transmission module performs
It is clear that the proposed architecture for the detection of feature layer fusion and combines the high-level semantic
maize leaf blight under complex background is more effective features with the previous layer features by deconvolution,
than SSD model in Fig. 9. The multi-layer input RPN network which improves the semantic information of the bottom fea-
improves the initial information by adjusting the position ture layer. Therefore, the detection effect of model 3 and 4
of region proposal for the classification and precise adjust- on small targets is improved. A partial visualization of the
ment of the detection network. Compared with the original model 3 and model 4 under Data set B is shown in Fig. 9.
SSD model, the mAP of model 3 (320∗ 320) is improved to It can be clearly seen from Fig. 9 that the models are more
85.65%, but its FPS reduce from 48 to 45.2. Compared with effective than the original SSD model under Data set B.

VOLUME 8, 2020 33685

 J. Sun et al.: Northern Maize Leaf Blight Detection Under Complex Field Environment Based on Deep Learning

FIGURE 10. Comparison of detection accuracy between model 4 and model 6.

TABLE 2. Comparison with different models.

The images (1-3) show the detection effect of SSD. Although E. COMPARISON WITH OTHER MODELS
some small targets are detected, there is still a missing detec- Based on the preprocessed data set B, Table 2 compares
tion. The images (4-6) show the detection effect of model 4, our model with other detection methods. Our method with
more small diseased position are detected and no missed Resnet-101 produces 91.83 mAP that is better than other
detection occurred. detection models based on Resnet-101. If the input picture
(i.e., 512∗ 512) is further enlarged, a better detection effect
D. THE EFFECT OF GIOU ON MAP may be obtained. Generally speaking, the one-stage detection
From Fig. 10, it is clearly that the mAP is improved by method (e.g., RetinaNet, DSSD) still produces a relatively
optimizing the original loss function. Comparing model 3 good FPS, but the detection accuracy is still worse than the
(320∗ 320) with model 5 in Data set B, the mAP increases two-stage method (e.g., RelationNet, SNIP). This is because
from 85.65% to 88.79%, and the mAP also has improve- the anchor generated by the one-stage detection method is
ment (1.76%) in Data set A. The best performance of our only a logical structure, which only needs to be classified and
method is 91.83% (512∗ 512) in model 6. The mAP is higher regressed. The anchor generated by the two-stage detection
(1.23%) than that of model 4 (512∗ 512). As can be seen from will map to the area of feature map, and then re-enter the area
Fig. 10, the detection accuracy of diseased position has been to the full connection layer for classification and regression.
improved. The main explanation is that GIoU is adopted to Although our proposed method is slightly inferior to the
redefine the loss. The GIoU can accurately reflect the real one-stage detection method in FPS, it has greatly improved
detection situation compared with the traditional smooth-L1. its FPS due to the sharing of features of the transmission
The images (1-3) and the images (4-6) show the detection module. As far as the disease data set we use, on the premise
effects of model 4 and model 6 respectively. Adding GIoU of ensuring the detection accuracy, improving the efficiency
into the basis of original model, the detection accuracy of of the overall model will provide greater help to the whole
diseased position is improved. production process of intelligent agriculture.

33686 VOLUME 8, 2020

J. Sun et al.: Northern Maize Leaf Blight Detection Under Complex Field Environment Based on Deep Learning

VI. CONCLUSION [10] X. Bai, Z. Cao, L. Zhao, J. Zhang, C. Lv, C. Li, and J. Xie, ‘‘Rice heading
In this paper, the convolutional neural network was applied stage automatic observation by multi-classifier cascade based rice spike
detection method,’’ Agricult. Forest Meteorol., vol. 259, pp. 260–270,
to the detection of maize leaf blight. A promising detection Sep. 2018.
performance in complex field was achieved, which could be [11] K. P. Ferentinos, ‘‘Deep learning models for plant disease detection
attributed to the improvements that we had made based on and diagnosis,’’ Comput. Electron. Agricult., vol. 145, pp. 311–318,
Feb. 2018.
SSD. In the proposed method, series of steps were amal- [12] J. Ma, K. Du, L. Zhang, F. Zheng, J. Chu, and Z. Sun, ‘‘A segmentation
gamated, including data preprocessing, feature fusion, fea- method for greenhouse vegetable foliar disease spots images using color
ture sharing, disease detection. The main reason behind data information and region growing,’’ Comput. Electron. Agricult., vol. 142,
pp. 110–117, Nov. 2017.
preprocessing was to reduce the influence of high-intensity [13] M. A. Khan, T. Akram, M. Sharif, M. Awais, K. Javed, H. Ali, and T. Saba,
light on image identification and improve detection accuracy. ‘‘CCDF: Automatic system for segmentation and recognition of fruit crops
In order to further improve the detection accuracy, feature diseases based on correlation coefficient and deep CNN features,’’ Comput.
Electron. Agricult., vol. 155, pp. 220–236, Dec. 2018.
fusion was utilized to produce the best possible results. In our
[14] Z. Lin, S. Mu, F. Huang, K. A. Mateen, M. Wang, W. Gao, and
proposed method, we also took into account the improvement J. Jia, ‘‘A unified matrix-based convolutional neural network for fine-
of detection efficiency. The transmission module not only grained image classification of wheat leaf diseases,’’ IEEE Access, vol. 7,
realized the feature fusion, but also transferred the relevant pp. 11570–11590, 2019.
[15] S. Zhang, W. Huang, and C. Zhang, ‘‘Three-channel convolutional neu-
anchor information in the fine-tuning network to the detection ral networks for vegetable leaf disease recognition,’’ Cognit. Syst. Res.,
modules, realizing the feature sharing between the modules, vol. 53, pp. 31–41, Jan. 2019.
and improving the detection efficiency. Compared with the [16] J. Ma, K. Du, F. Zheng, L. Zhang, Z. Gong, and Z. Sun, ‘‘A recognition
method for cucumber diseases using leaf symptom images based on deep
original SSD model, the mAP of new models was higher convolutional neural network,’’ Comput. Electron. Agricult., vol. 154,
(from 71.80% to 91.83%) than the mAP of original SSD. pp. 18–24, Nov. 2018.
The FPS of the new model also had certain improvement [17] S. Sladojevic, M. Arsenovic, A. Anderla, D. Culibrk, and D. Stefanovic,
‘‘Deep neural networks based recognition of plant diseases by leaf
(from 24 to 28.4) and had reached the standard of real-time image classification,’’ Comput. Intell. Neurosci., vol. 2016, pp. 1–11,
detection. Jun. 2016.
The new model of this study was useful for the detection of [18] H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. Reid, and S. Savarese,
‘‘Generalized intersection over union: A metric and a loss for bounding
maize leaf blight in complex background. The disease detec- box regression,’’ in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.
tion model was efficient and accurate, which could replace (CVPR), Jun. 2019, pp. 658–666.
the on-site identification of human experts. It could reduce [19] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and
the labor force and overcome the subjectivity of selecting A. C. Berg, ‘‘SSD: Single shot multibox detector,’’ in Proc. Eur. Conf.
Comput. Vis. (ECCV), Oct. 2016, pp. 21–37.
features artificially. The model could be moved into the [20] Y. Shen, Y. Yin, C. Zhao, B. Li, J. Wang, G. Li, and Z. Zhang, ‘‘Image
embedded system, which lays a theoretical foundation for the recognition method based on an improved convolutional neural network
development of precise drug application and precise detection to detect impurities in wheat,’’ IEEE Access, vol. 7, pp. 162206–162218,
2019.
robot for maize leaf blight. [21] M. P. S. da Silva, M. S. M. Freitas, P. C. Santos, A. J. C. de Carvalho, and
T. S. Jorge, ‘‘Capsicum annuum var. Annuum under macronutrients and
boron deficiencies: Leaf content and visual symptoms,’’ J. Plant Nutrition,
REFERENCES
vol. 42, no. 5, pp. 417–427, Mar. 2019.
[1] P.-Y. Tong, ‘‘Report on corn production in China,’’ Agric. Tech. Eqpt., [22] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, ‘‘Focal loss for dense
vol. 9, pp. 12–18, Sep. 2011. object detection,’’ IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 2,
[2] C. DeChant, T. Wiesner-Hanks, S. Chen, E. L. Stewart, J. Yosinski, pp. 318–327, Feb. 2020.
M. A. Gore, R. J. Nelson, and H. Lipson, ‘‘Automated identification of [23] J. Wei, Q. Zhijie, X. Bo, and Z. Dean, ‘‘A nighttime image enhancement
northern leaf blight-infected maize plants from field imagery using deep method based on Retinex and guided filter for object recognition of apple
learning,’’ Phytopathology, vol. 107, no. 11, pp. 1426–1432, Nov. 2017. harvesting robot,’’ Int. J. Adv. Robot. Syst., vol. 15, no. 1, pp. 1–12,
[3] M.-Y. Zhu, H.-B. Yang, and Z.-W. Li, ‘‘Early detection and identification Jan. 2018.
of rice sheath blight disease based on hyperspectral image and chlorophyll [24] K. G. Lore, A. Akintayo, and S. Sarkar, ‘‘LLNet: A deep autoencoder
content,’’ Spectrosc. Spectr. Anal., vol. 39, pp. 1898–1904, Jun. 2019. approach to natural low-light image enhancement,’’ Pattern Recognit.,
[4] C.-L. Chung, K.-J. Huang, S.-Y. Chen, M.-H. Lai, Y.-C. Chen, and vol. 61, pp. 650–662, Jan. 2017.
Y.-F. Kuo, ‘‘Detecting Bakanae disease in rice seedlings by machine [25] D. J. Jobson and G. A. Woodell, ‘‘Multiscale retinex for color rendition
vision,’’ Comput. Electron. Agricult., vol. 121, pp. 404–411, Feb. 2016. and dynamic range compression,’’ Proc. SPIE, vol. 2847, pp. 183–191,
[5] D. I. Patrício and R. Rieder, ‘‘Computer vision and artificial intelligence Nov. 1996.
in precision agriculture for grain crops: A systematic review,’’ Comput. [26] S. Ren, K. He, R. Girshick, and J. Sun, ‘‘Faster R-CNN: Towards real-
Electron. Agricult., vol. 153, pp. 69–81, Oct. 2018. time object detection with region proposal networks,’’ IEEE Trans. Pattern
[6] Z. Lin, S. Mu, A. Shi, C. Pang, and X. Sun, ‘‘A novel method of maize leaf Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149, Jun. 2017.
disease image identification based on a multichannel convolutional neural [27] X. Liu, D. Zhao, W. Jia, W. Ji, C. Ruan, and Y. Sun, ‘‘Cucumber fruits
network,’’ Trans. ASABE, vol. 61, no. 5, pp. 1461–1474, Oct. 2018. detection in greenhouses based on instance segmentation,’’ IEEE Access,
[7] G. Zhou, W. Zhang, A. Chen, M. He, and X. Ma, ‘‘Rapid detection of rice vol. 7, pp. 139635–139642, 2019.
disease based on FCM-KM and faster R-CNN fusion,’’ IEEE Access, vol. 7, [28] J. Sun, W.-J. Tan, and H.-P. Mao, ‘‘Identification of plant leaf diseases
pp. 143190–143206, 2019. based on improved convolutional neural network,’’ Trans. Chin. Soc.
[8] Z. Libo, H. Tian, G. Chunyun, and M. Elhoseny, ‘‘Real-time detection of Agric., vol. 33, pp. 151–162, Oct. 2017.
cole diseases and insect pests in wireless sensor networks,’’ J. Intell. Fuzzy [29] N. Liu and J.-M. Kan, ‘‘Improved deep belief networks and multi-feature
Syst., vol. 37, no. 3, pp. 3513–3524, Oct. 2019. fusion for leaf identification,’’ Neurocomputing, vol. 216, pp. 460–467,
[9] P. Jiang, Y. Chen, B. Liu, D. He, and C. Liang, ‘‘Real-time detection Dec. 2016.
of apple leaf diseases using deep learning approach based on improved [30] S. Bertrand, R. B. Ameur, G. Cerutti, D. Coquin, L. Valet, and L. Tougne,
convolutional neural networks,’’ IEEE Access, vol. 7, pp. 59069–59080, ‘‘Bark and leaf fusion systems to improve automatic tree species recogni-
2019. tion,’’ Ecol. Informat., vol. 46, pp. 57–73, Jul. 2018.

VOLUME 8, 2020 33687

 J. Sun et al.: Northern Maize Leaf Blight Detection Under Complex Field Environment Based on Deep Learning

[31] D. G. Lowe, ‘‘Object recognition from local scale-invariant features,’’ in YU YANG is currently pursuing the master’s
Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Sep. 1999, pp. 1150–1157. degree with the School of Electrical and Infor-
[32] Y. Yu, K. Zhang, L. Yang, and D. Zhang, ‘‘Fruit detection for strawberry mation Engineering, Jiangsu University, China.
harvesting robot in non-structural environment based on mask-RCNN,’’ His research interests include applications of deep
Comput. Electron. Agricult., vol. 163, Aug. 2019, Art. no. 104846. learning in agriculture, mainly applications of con-
[33] M. Haggag, S. Abdelhay, A. Mecheter, S. Gowid, F. Musharavati, volution neural networks to the precise detection
and S. Ghani, ‘‘An intelligent hybrid experimental-based deep learn- of crop diseases.
ing algorithm for tomato-sorting controllers,’’ IEEE Access, vol. 7,
pp. 106890–106898, 2019.
[34] X. Ye and Q. Zhu, ‘‘Class-incremental learning based on feature extraction
of CNN with optimized softmax and one-class classifiers,’’ IEEE Access,
vol. 7, pp. 42024–42031, 2019.
[35] W. A. Gardner, ‘‘Learning characteristics of stochastic-gradient-descent
algorithms: A general study, analysis, and critique,’’ Signal Process., vol. 6, XIAOFEI HE is currently pursuing the mas-
no. 2, pp. 113–133, Apr. 1984. ter’s degree in electronics and communica-
tion engineering with Jiangsu University, China.
His research interests include applications of
computer vision in agriculture, using deep learn-
ing method to study weed detection in complex
background.

JUN SUN received the Ph.D. degree in mechanical XIAOHONG WU born Hefei, China, in 1971.
electronics from Jiangsu University. He is cur- He is currently a Professor with Jiangsu Univer-
rently a Professor and the Doctoral Director of sity. He is mainly engaged in machine learning,
the School of Electrical Information Engineering, pattern recognition, and spectral information pro-
Jiangsu University. He has published over 80 arti- cessing. He is serving as a member of the Sixth
cles in his research related fields. His research Council of China Electronic Education Society.
focuses on applications of computer electronics
in agriculture, including deep learning, hyper-
spectral technology, and nondestructive testing
technology.

33688 VOLUME 8, 2020