Mice 12511
Mice 12511
This is the author manuscript accepted for publication and has undergone full peer review but has not been through the copyediting,
typesetting, pagination and proofreading process, which may lead to differences between this version and the Version of Record. Please
cite this article as doi: 10.1111/mice.12511
non-parametric. The parametric approaches often rely on peratures causing a change in the frequencies are considered
characteristic parameters obtained from structural responses. as the input during the training of the network such that false
Such methods often fuse one type of learning algorithms with positives can be eliminated. The study looked at the Euclid-
a preprocessing feature extraction algorithm. For example, ian distance between measured and estimated frequencies to
system identification can be regarded as a preprocessing al- calculate a novelty index. Their proposed network is tested
gorithm capable of computing features such as natural fre- on a numerical model and in the laboratory on a small-scale
quencies, mode shapes, and damping ratios of a structure test structure. Deraemaeker and Worden [12] compared the
from raw data. A drastic change of the natural frequency is performance of Mahalanobis squared-distance, and Princi-
usually relatable to structural damage. The underlying learn- pal Component Analysis (PCA) using real experimental data
ing algorithm is expected to capture this damage. Likewise, from a wooden bridge. The features consist of eigenfrequen-
modal analysis methods, such as cross-correlation functions cies and mode shapes measured under changing environmen-
and frequency response functions can extract other strong tal conditions. Lee et al. [33] and Mehrjoo et al. [42] consid-
features of the structure that provide broader information ered a hybrid approach where a finite element model is es-
over time and space [57]. Parametric methods are advanta- tablished as a baseline and neural network is trained to detect
geous over their non-parametric counterparts since they don’t the damage based on the expected output of finite element
need to rely on a numerical model of the structure. model. These approaches also utilized natural frequencies
Author Manuscript
For example, Sohn et al. [54] developed a parametric nov- and mode shapes.
elty detection method that is capable of taking the variations The non-parametric approaches do not require a baseline
caused by ambient conditions such as a change in loading, to establish from structural parameters prior to deployment
temperature, etc. into account to minimize false positive in- and do not depend on the uncertainty of system identification
dicators. The method employs AANN to discriminate criti- or other modal analysis tools. Non-parametric techniques
cal system changes from ambient induced temperature varia- are advantageous, especially when obtaining a dense array
tions. The network is trained via supervised learning to learn of structural parameters for complex and large systems are
the correlation between the variability in the ambient condi- challenging. aAs an example to non-parametric approaches,
tions and inherent changes driven by these conditions. The Abdeljaber et al. [2] used decentralized 1D convolutional
proposed system is tested for a hard-disk model described neural networks (CNNs) to eliminate the feature extraction
as a transfer function and it is hypothesized that it could be process of typical system identification methods and perform
applied to civil structures. Worden et al. [61] used a very sim- the damage detection directly on the sensor data in real-time.
ilar approach involving an auto-associative neural networks However, sensor data from the healthy and damaged struc-
(AANNs) and novelty index and evaluated the approach us- ture is used to train the network for classification purposes
ing a more realistic structural system such as a plate sup- which makes the approach supervised learning. Addition-
ported by stringers similar to a bridge deck. In this study, ally, this study does not consider operational and environ-
frequency response functions are used as the input to the net- mental variability. The algorithm is tested on a grandstand
work. Novelty detection through machine learning is also simulator in the lab. In the study, since the trained neural
investigated for detecting damages of wind turbine blades network was not completely successful for classifying the
under fatigue loading. For example, Dervilis et al. [13] and structural condition, specifically producing false negatives,
Dervilis et al. [14] developed a noise tolerant AANN to eval- an index reflecting the likelihood of the damage is proposed
uate the condition of CX-100 wind turbine blade. The fre- by computing the ratio of true positives to the total number
quency response functions were used in this study which is of test cases. Gulgec et al. [24] and Yu et al. [67] used sim-
a similar approach to Worden et al. [61]. Zhou et al. [69] ilar approach utilizing deep CNNS to detect damage from
developed two neural networks, one Back Propagation Neu- sensor data. They also ignore ambient uncertainties. Multi-
ral Networks (BPNNs) and one AANN to detect the damage ple signal classification (MUSIC) algorithm is another non-
for Ting Kau Bridge in Hong Kong. The BPNN is used to parametric approach based on fuzzy wavelet neural networks
create a correlation model between damage-sensitive modal known to produce successful damage detection from limited
frequencies and temperature and AANN is employed to char- sensor data [28, 3, 4].
acterize the healthy state of the bridge. After the field data is This paper introduces an effective damage detection ar-
analyzed, an FE model of the bridge is created and simulated chitecture for structures under environmental uncertainty us-
to generate new monitoring data where damage was induced ing machine learning. This study utilizes well-established
in various regions of the model. In addition, the environmen- learning algorithms to extract latent features from modal pa-
tal effects are superimposed to the data. Gu et al. [23] used rameters such as natural frequencies and mode shapes under
the modal frequencies of the target structure and the mea- temperature variations and to reconstruct a new representa-
sured temperatures as the input for AANN to improve the tion of these features that is similar, if not identical, to the
generalization capability. In addition, variations in the tem- original. The difference between original and reconstructed
parameters constitutes the essential information for detect- used for training the machine learning model and obtaining
ing critical changes in the system. While modal parameters the novelty index for each case. Lastly, a large-scale three di-
are known to be a well-researched damage indicator, to au- mensional three-story structure is identified and modeled un-
thors’ best knowledge, this research is the first time that un- der temperature gradient. The detection perofrmance of the
supervised machine learning components such as PCA and proposed method is evaulated under multiple damage condi-
auto-encoder are applied to utilize mode shapes in addition tions.
to natural frequencies for effective damage detection under The overall results of the simulations and lab experiments
environmental variability. show that the proposed method has, in general, better perfor-
As stated above, the approach proposed herein uses mance in detecting damage since it utilizes mode shapes as
the natural frequencies and mode shapes resulting from a an input in addition to the natural frequencies. In essence,
well-recognized system identification tool, Natural Exci- this modal analysis based novel detection approach has the
tation Technique and Eigensystem Realization Algorithm potential to serve as a reliable and near real-time damage de-
(NExT/ERA) as the input and produces a target output which tection tool providing accurate data towards objective-driven
is the expected natural frequencies and/or mode shapes of the decisions for maintenance operations. In theory, the end-to-
system [26, 27, 7, 5, 8]. The damage detection relies on the end pipeline considered in this study is capable of streaming
concept of Novelty Index which calculates the mean squared real-time data in the time domain from sensors, extract the
Author Manuscript
error between input and outputs, e.g. actual and expected modal features from the time domain data in near real-time
natural frequencies, respectively[12]. To achieve this goal, depending on the availability of the computational resources,
two unsupervised learning approaches are investigated: (a) and compute the novelty index. This approach would indeed
principal component analysis (PCA) and (b) Auto-Encoder accelerate the decision-making process since the state of the
(AE). target structure is available immediately [2].
To evaluate and validate the approach along with the learn- In summary, the major contributions of this paper can be
ing approaches, a simply supported beam structure is mod- summarized as below:
eled and simulated in OpenSees under ambient vibration con-
• A new machine learning approach is proposed that re-
ditions. To add uncertainty to the simulation, temperature,
lies on natural frequencies and mode shapes.
which is known to affect material properties nonlinearly, is
varied over a range. The resulting response data constitutes • This paper streamlines the proposed approach into a
the reference basis for the training data of the machine algo- pipeline aggregating data collection, system identifica-
rithms. The modal properties of the structure are extracted tion, and damage detection.
from this data set, and the machine learning model (model
set A) is trained using the aforementioned approaches. In • For proof of concept, data from simulation and experi-
parallel, another set of models (model set B) is developed mental tests are employed. The performance of the pro-
using only natural frequencies as the input as it is prescribed posed method is evaluated by comparing the detection
in previous studies. To demonstrate the advantages of fusing results to those from prior machine learning methods.
frequencies with mode shapes further, a third model (model • Results demonstrate that the new approach improves the
set C) utilizing only the mode shapes as the input. Finally, the damage detection rate significantly at the presence of
proposed method is evaluated one more time using the same environmental variability.
beam exposed to gradient temperature distribution instead of
uniform temperature. 2 METHODOLOGY
Next, two three damage cases are considered where stiff-
ness loss is induced at the midspan at various levels. The In this section, the essential components of the proposed ap-
structure is again simulated under ambient vibrations, and proach, illustrated in Figure 1, are explained in detail. First, a
the resulting modal parameters are fed to the learning model. general description of the target structure and the problems to
For the two three model sets, novelty index is calculated and be solved are described. Secondly, the fundamentals of sys-
the reliability of the results are examined to demonstrate the tem identification and feature selection are introduced. The
effectiveness of the proposed approach in detecting the dam- third part of this section focuses on the general architecture
age. of the learning components, and the two machine learning
In addition to the simulations, this study considers a models used in this architecture, PCA and AE. The next part
dataset containing laboratory experiments of a scaled three- which constitutes the final component discusses the imple-
story structure created by Los Alamos National Labs for fur- mentation of the novelty detection responsible for determin-
ther validation. The structure is tested under various dam- ing if the structure is damaged. Finally, evaluation criteria to
age scenarios simulating section loss at single and multiple study the performance of the different learning components
columns. An approach identical to the analytical study is in detecting the damage are discussed.
investigated to validate the capabilty of the proposed method As mentioned above, this component combines NExT
further.. The second one is a small-scale three-story struc- with ERA. Essentially, NExT calculates the free response
ture tested by Figueiredo et al. [19]. Finally, the third one is data from ambient data, whereas ERA extracts natural fre-
a three dimensional three-story structure. Section 3 presents quencies, mode shapes, and damping rations from the free re-
analytical and experimental structures in detail. sponse data. Assuming the ambient excitation input is white
This component can be formalized as a differential noise, second order equation of motion can be written as:
equation of motion as given below: M R̈z̈,z̈i (τ ) + C Ṙz̈,z̈i (τ ) + KRz̈,z̈i (τ ) = 0 (1)
M z̈(t) + C ż(t) + Kz(t) = F (t) where M , C, K are mass, damping, and stiffness matri-
where M , C, K are mass, damping, and stiffness matrices ces of the system, Rz̈,z̈i (τ ) is the cross-correlation function
of the system, respectively, z, ż, z̈ are displacement, velocity, between the acceleration, z̈i measured at ith location and a
and acceleration vectors, respectively, F is the input force in reference acceleration z̈. Rz̈,z̈i (τ ) has the same form as the
the form of ambient vibration. Here, the stiffness matrix, free vibration response of the structure to be identified. Here,
K can be written as a function of ambient temperature, T reference signal, z̈ can be chosen as the acceleration of a
since temperature dependent material properties govern the node on the structure. By computing the cross-spectral den-
stiffness. It is assumed that the approach has access only to sity function with respect to the reference acceleration and
accelerations, z̈ and to the ambient temperature, T at the time applying inverse Fourier transformation, the free vibration
when the measurement is taken. response can be obtained in the form of cross-correlation:
Eq. ?? can be also written as: N −1
1 X 2πkn
R̈z̈,z̈i = Sz̈,z̈i (k)exp j (2)
N N
k=0
ẋ(t) = Ax(t) + Bu(t) where Sz̈,z̈i (k) is the cross-spectral density function of z̈ and
y(t) = Cx(t) + Du(t) z̈i , k is the frequency index, and n is the time index. More
details on NExT are provided by Caicedo [6] and Caicedo
where x is the state vector, y is output vector, u is the
et al. [8].
ambient vibration input, A, B, C, and D are the state-space
ERA utilizes this free vibration data to determine the
matrices.
modal parameters by first constructing the Hankel matrix:
2.2 System Identification and Feature Selection Y (k) Y (k + 1) ... Y (k + p)
System identification is the process of obtaining dynamic Y (k + 1) Y (k + 2) . . . Y (k + p + 1)
H(k−1) = ..
and static characteristics of the structure under service, ex- .
treme loads or synthetic excitation. The parameters obtained Y (k + r) Y (k + r + 1) . . . Y (k + p + r)
from identification can be used as an indicator to detect po- (3)
tential damage in the target structure [15]. In this study, it where Y (k) is the m × n response matrix at k th time step.
is assumed that reliable modal parameters can be extracted A singular value decomposition on Hankel matrix at k = 1,
from the structure under ambient vibration [18]. To mini- H(0) yields:
mize the effect of mass change due to the service load and H(0) = RΣS T (4)
where R and S are orthonormal matrices, and where Σ is the training data for the learning enabled component con-
a diagonal matrix containing the singular values. It can be tains majorly normal data (from undamaged structure) and
shown that the state-space matrices can be computed as given very few anomalies (outliers due to instantaneous abnormal
below: events, poor data processing, etc.) [9, 10]. If there is statis-
tically significant event (caused by damage but not environ-
 = Σ−1/2 RT H(1)SΣ−1/2 (5) mental effects) deviating from baseline, then unsupervised
B̂ = Σ−1/2 S T Em
T
(6) approaches are expected to capture this event; thus, the er-
Author Manuscript
Ĉ = EnT RΣ−1/2 (7) ror between actual data and the reconstructed/expected data
increases. This process is advantageous especially where hu-
where Â, B̂, Ĉ is the estimated state matrices A, B, C, man experts have difficulty detecting and observing the dam-
respectively; D̂ = 0; Em T
= [I 0] and EnT = [I 0]. Juang and age by looking at the data if there is too much variability. The
Pappa [29] discusses ERA method in details. Novelty Index presented here is not a damage classification,
By applying eigenvalue problem on Â, the modal param- but rather a signal that something has changed in the system
eters such as natural frequencies, f and mode shapes, Φ can and owner of the structure may act on this signal considering
be calculated by the risk, operation and maintenance cost.
The learning component is essentially a mapping process
(Â − λI)Φ = 0 (8) and it can be formalized as:
log λ
w= (9) X̂ = G(X) (10)
Ts
subjected to:
where w = 2πfs and Ts is the sampling time.
min X̂ − X 0 (11)
2.3 Machine Learning Model where X is the input, X 0 is the subset of X to be re-
constructed, and X̂ is the output representing the recon-
Often the system identification methods are sensitive to the
structed X 0 . G(·) is the mapping function and k·k is the
changes in the system induced by damage or environmen-
normalization operator. To achieve this objective, the map-
tal and operational effects. However, it is also a challenging
ping function G(·) should be trained with known input
task to differentiate the damage from such variations since
X. The input is defined as a set of n natural frequen-
the baseline is polluted [54]. For instance, studies conducted
cies where f = [f1 f2 . . . fn ] and n mode shape vectors,
by [43], Liu and DeWolf [38], Xu et al. [65], Xia et al. [64],
Φ = [Φ1 Φ2 . . . Φn ]. Ambient temperature, T taken during
Gonzalez [21], and Li [34] have shown that the temperature
the time of measurement can be also added, if available, to
can cause significant changes in dynamic properties of struc-
the input since it is considered as a feature in damage de-
tures. With the aid of the unsupervised learning approaches,
tection [69, 23]. A complete input from one measurement
a higher-fidelity baseline condition of the structure can be
instance can be defined as:
extracted from the polluted data set. Here, the objective of
the learning model component is to learn a representation X (i) = vec([T (i) f (i) Φ(i) ]) (12)
of the data set typically through dimension reduction and to
reconstruct a new representation that is similar, if not iden- where i is the index for ith measurement and vec(·) is the
tical, to the original data. In essence, both principal com- vectorization operator. f (i) and Φ(i) are obtained through
ponent analysis (PCA) and Auto-Encoder (AE), also known system identification and T (i) is the temperature taken during
as auto-associative neural networks (AANNs) can be used the system identification measurement. Similarly, the input
to form this behavior. This study uses those two models in- to be reconstructed and the output are defined as:
terchangeably to extract a latent features and evaluates the
X 0(i) = vec(f (i) ) (13)
performance of the architecture by how well the damage is
detected. Here, both approaches (PCA and AE) assume that X̂ (i) = vec(fˆ(i) ) (14)
where fˆ(i) is the reconstructed representation of the input, 2.3.1 Reconstruction using Principal Component Anal-
f (i) . ysis
Compared to previous research relying only on natural fre-
Principal Component Analysis is a machine learning algo-
quencies [69, 23], this study considers mode shapes also as
rithm that reduces the dimensionality of a data set leading to
a valid input. The modal parameters can be obtained from
a simpler representation of it while preserving essential in-
eigenvalue analysis of K and M as follows:
formation that defines the data set [20, 22]. This property
[K − (2πfi )2 M ]{Φi } = 0 (15) of PCA is achieved by computing a linear transformation
matrix which can project the original data containing cor-
When the Young’s modulus property of the material, E
related variables to another representation with uncorrelated
changes due to the temperature variations, the stiffness ma-
variables. One main advantage of this decorrelation is ex-
trix, K is affected linearly while M remains same. The
posing the so-called principal components that explain the
relationship between the reference stiffness, K and the
dominant patterns in the data [68, 66, 55]. By selecting the
temperature-affected stiffness, K 0 can be simply described
prevailing components, one can compress the data, in other
by:
words, reduce the dimension of the data, and expose the most
K 0 = cK (16)
important features that are still faithful to the original.
where c is a factor defining the linear relationship. When The linear transformation of the PCA can be represented
Author Manuscript
The objective of the learning component is recovering an ex- The structural responses are obtained from the finite element
pected reconstruction of the original input while eliminat- model or the experimental test setup in undamaged and dam-
aged conditions as accelerations, ÿ. The accelerations are 3.2 Analytical Verification with Simply Supported
recorded for some amount of time and saved in a file for each Beams
instance of simulation or experiment. Each of these instances A simply supported steel beam used by Gu et al. [23] with
containing accelerations is analyzed using NExT/ERA im- a span length of L = 5.0 m is discretized into equally long
plemented in MATLAB 2018b [40]. After natural frequen- 40 member having a cross-sectional area of A = 1.624 ×
cies and mode shapes are obtained from NExT/ERA, this in- 10−3 m2 and moment of inertia of I = 1.971 × 10−6 m4 .
formation is vectorized. If the temperature is recorded for The beam is modeled using finite element modeling (FEM)
an instance, it is also augmented to the vector. Before the tool, Open System for Earthquake Engineering Simulation
training, the data is standardized using StandardScaler from - OpenSees [41]. The members are assumed to be elastic-
scikit-learn toolbox 0.20.2 [46] such that each feature has beam column elements. A nonlinear relationship between
zero mean and unit variance. All the scaled data are saved in material stiffness of the elements, E and temperature, T is
their relevant files, based on the condition of the structure. described as given below:
Next, the machine learning model is trained using the data E = [206.216 − 0.4884T + 0.0044T 2 ] × 109 N/m2 (28)
from the undamaged condition. About half to two-third of where T is in the unit of Fahrenheit. The mass is adjusted
the data is used for training whereas the remaining data is such that the structure has the first natural frequency at nearly
Author Manuscript
utilized for testing and validation to make sure overfitting is 0.49 Hz when the temperature is 15°C (∼60°F).
prevented. Both PCA and AE algorithms are implemented Following the architecture discussed in the previous sec-
in Python 3.6.7 [49]. The PCA model is trained using scikit- tion, the training and validation data set for the undamaged
learn toolbox 0.20.2 [46]. By trial and error, an appropriate structure is developed by applying ambient vibration made of
number of components are selected to explain the variance white noise to the supports of the beam vertically. The input
of the data. The reconstruction is performed by first trans- white noise has a bandwidth of 1024 Hz and the peak dis-
forming the input data to reduced data and then applying placement is about 0.1 g. The vertical acceleration responses
an inverse transformation which is explained in Equation 22. to the ambient excitation at 39 nodes (excluding 2 support
AE is trained using Keras 2.2.4 running on TensorFlow 1.12 responses) are sampled at 200 Hz for 300 seconds. For each
[1, 11]. By trial and error, a neural network with 4 layers simulation, the ambient temperature governing the material
(shown in Figure 2) is developed to capture salient features stiffness (see Eq. 28) is randomly varied between −15°C and
of the data. The output of the AE model is the natural fre- 50°C bounded by a uniform distribution. The temperature
quencies to be reconstructed. The models that contain the range is selected to lay out the nonlinear relationship between
natural frequencies and mode shapes are called model set A. temperature, material stiffness and natural frequencies fully
In parallel, another set of models (model set B) is developed (see Figures 3 and 4). Additionally, the distribution allows
using only natural frequencies as the input. Additionally, a the environmental effects to contaminate data over the entire
third model (model set C) is trained which uses only mode temperature range. Figure 3 illustrates the temperature vs.
shapes. stiffness computed according to Eq. 28 for the undamaged
case. The difference between the minimum and maximum
Finally, the novelty index is obtained by comparing the in- values of stiffness corresponds to nearly 10 percent of the
put natural frequencies with the output for model set A and B minimum stiffness. Figure 4 demonstrates the temperature
or by comparing the input natural frequencies with the out- vs. identified frequency distribution for the undamaged case.
put for model set C. Effectively, there is one novelty index The difference between the minimum and maximum values
for each vector. Data from different damage conditions are of natural frequencies corresponds to nearly 4 percent of the
tested as well in this last step. This step is also implemented first natural frequency of the undamaged structure. A set
in Python. of damage conditions are defined for this structure (see Ta-
ble 1). In total, 4000 simulations are executed. NExT/ERA
It is important to note that, specifically for model set A, is performed on the resulting data to extract the first six nat-
while mode shapes could also be a part of the output vec- ural frequencies, f and mode shapes for each natural fre-
tor to be reconstructed, the scale of mode shapes is not the quency, Φ. These first six modes also constitute the features
same as frequencies; thus, their contributions to the novelty to be used for damage detection in accordance with Gu et al.
indices may not be as dramatic as natural frequencies. More- [23]. From each simulation, including the ambient temper-
over, the results presented in the following sections demon- ature, six natural frequencies, and 234 mode shape points
strate that the proposed architecture is capable of detecting (6 modes × 39 mode shape points per mode), a vector of 241
damage without reconstructing mode shapes. Reducing the data points is created which establishes the input data for the
dimension of the output not only accelerates the learning but learning enabled component. Out of 2000 vectors from un-
also reduces the risk of curse of dimensionality [25]. damaged case, randomly selected 1000 vectors are used for
training the machine learning component. The remaining ages, the utilization of mode shapes does not improve the
data is used for the validation. For the two three damage outcome of the detection further since the novelty indices are
cases considered here, the damage is emulated by reducing distinguishable enough for primitive models. The modified
the stiffness of the 20th element from the left support (cor- Euclidean distances computed using Eq. 27 for each damage
responding to the midspan) by 5% (Damage Case 1), 15% case and architecture are tabulated in Table 3. Here, for each
(Damage Case 2) and 50% (Damage Case 3). For each dam- case, the novelty indices for the No Damage Case from the
age case, 1000 simulation are executed under uniformly dis- validation data set relevant to that case are used as the refer-
tributed random ambient vibrations varying between −15°C ence, N Iud . The mean of N Iud establishes µN Iud . For No
and 50°C. It should be noted that the temperature range used Damage Case specifically, the comparison is made between
in the simulations is rather wide and is not observed for most the validation and training data. To calculate Dud,d , the com-
climate conditions. However, this range also introduces rel- plete index vector is used. When the mode shape is intro-
atively large variability to the natural frequencies. The pro- duced to the training, distances become smaller for all cases.
posed algorithm is expected to robustly detect damage under For PCA, at the absence of mode shapes, the No Damage
large temperature variations. and Damage Case 1 values are similar for both architectures.
Figure 5 presents the distribution of natural frequencies for However, AE has a higher distance suggesting that it may
the no damage and damage cases using system identification. be still possible to detect damage with AE. At the presence
Author Manuscript
One can observe that the differences in the frequencies are of mode shapes, the distances are much larger which sig-
visually not evident, especially between No Damage Case nifies improved damage detection for the given structure. In
and Damage Case 1. This can justify machine learning algo- summary, evaluation of the architecture performance demon-
rithms capable of capturing latent features of the presented strates that the introduction of mode shapes enhances dam-
data. age detection. When only mode shapes are considered for
Regarding the machine learning component, as mentioned reconstruction, it is observed that the relative distances in-
before, two architectures are considered: PCA and AE. For crease. This is due to the fact that more features are recon-
either architecture, the training input is the 1000 vectors each structed compared to Model Set A and B at the expense of
containing the following data points: (i) For model set A, in- computational complexity. For AE network, reconstructing
cluding temperature, 241 data points (1 temperature data + only mode shapes (model set C) do not improve the detec-
6 natural frequencies + 234 mode shape data) are packed as tion, whereas for PCA, the sensitivity of model set C is much
a vector from each simulation. (ii) For model set B, only higher. At this point, it is up to the designer how much sen-
six natural frequencies and temperature data is used. (iii) sitivity is desired and what are the computational resources
As for model set C, 234 mode shape points and tempera- available to reach to the desired damage detection sensitiv-
ture data are utilized. The number of components used for ity. Ideally, an ensemble combining PCA and AE ensures
PCA and network architecture for AE are tabulated in Ta- the best detection.
ble 2 for each model set. After data standardization, for
PCA, a model with 100 components are trained. For the AE,
the output is selected as the first 6 natural frequencies and a
neural network with the dimensions 241-12-12-6 is sufficient
3.3 Effect of gradient temperature distribution
to capture salient features of the data. A linear activation
function is used for all neurons. While in this paper we hypothesized that the mode shapes
The novelty index for both architectures is presented in do not change under uniform temperature distribution, mode
Figure 6. The effect of mode shapes to the performance of shapes will show slight variation if there is temperature gra-
the approach is shown by comparing the novelty index of dient. Such variations may cause some degradation in the
each architecture when mode shapes are used and omitted performance of the proposed method. This section focuses
(model set A, B and C). It is evident from the visual compar- on the effectiveness of the method under temperature gradi-
isons that including mode shapes into the learning improves ent. Here, it is assumed that the temperature difference be-
the performance of the detection. When mode shapes are tween each end of beams is 10°C and changes linearly across
not present, there is an overlap between No Damage Case the beam. The same number of inputs are used for all the
and Damage Case 1 for both architectures. This overlap may learning components. In addition, to increase the sensitivity
lead to false positives or negatives degrading the performance of the AE network for model set A, the system structure is
of detection when the damage is small. However, when the modified to 241-50-50-6. The novelty index for both archi-
damage is larger, the overlap is not observable anymore. To tectures is presented in Figure 7. In general, the proposed
summarize, the proposed approach is successful in capturing method can detect the damage under temperature gradient.
the small damage compared to the primitive model which The overall findings are consistent with the results from the
employs only natural frequencies. For large enough dam- uniform distribution.
Figure 5: Distribution of natural frequencies with varying ambient temperatures for each damage case in analytical data
Table 3: Modified Euclidean Distances for Damage Cases with and without the Inclusion of Mode Shapes for Analytical Data
(a) (b)
(c) (d)
(e) (f)
Figure 6: Comparison of novelty indices for analytical data: (a) PCA - model set B (mode shapes not included); (b) AE - model
set B (mode shapes not included); (c) PCA - model set A (mode shapes included); (d) AE - model set A (mode shapes included);
(e) PCA - model set C (only mode shapes included); (f) AE - model set C (only mode shapes included)
(a) (b)
Author Manuscript
(c) (d)
(e) (f)
Figure 7: Comparison of novelty indices for analytical data under temperature gradient: (a) PCA - model set B (mode shapes
not included); (b) AE - model set B (mode shapes not included); (c) PCA - model set A (mode shapes included); (d) AE - model
set A (mode shapes included); (e) PCA - model set C (only mode shapes included); (f) AE - model set C (only mode shapes
included)
3.4 Experimental Verification is not observed when mode shapes are introduced. One can
notice that with the use of mode shapes, the novelty index of
3.4.1 Structure 1
Damage Case 3 and 4 where third columns were damaged is
For further verification of the proposed approach, a small- higher for both architectures. The damage at the third floor
scale three-story structure tested by Figueiredo et al. [19] changes the mode shapes to the point that the novelty index is
at Los Alamos National Laboratory is studied. This struc- amplified even though the induced damage is not larger than
ture is excited with an electromagnetic shaker attached to the first floor. From this observation, it can be concluded
its base. The shaker provided a band-limited white noise that while the proposed approach is a successful damage de-
and the resulting acceleration responses of the structure is tection tool, the results may not be definite regarding the
recorded for about 25 seconds at a sampling rate of about magnitude of the damage. To understand the results quan-
320 Hz (see Figure 8). Figueiredo et al. [19] indicated that titatively, the modified Euclidean distances for each damage
the lab environment is not temperature controlled and some case computed according to Eq. 27 are provided in Table 5.
temperature variations observed. However, they also did not The validation data set from the No Damage Case is used as
record the ambient temperature during the experiments. A the reference. Since the validation data is limited, the data
set of damage conditions are defined for this structure, see is repeated to match the size of the target. Figure 10 implies
Table 4. Including no damage condition, there are five dam- that when mode shapes are introduced novelty indices may
Author Manuscript
age cases for this structure. The damage is introduced by decrease slightly. In parallel, the baseline for no damage case
reducing the stiffness of one or two columns at each floor approaches to zero resembling a flat line. Although the nov-
by 87.5 percent. NExT/ERA is applied to all the listed ex- elty indices reduce, the modified Euclidian distance between
perimental data. Since the sampling time is short, the sys- the baseline and the damage cases increases which implies
tem identification is not able to determine all the dominant that damage is quantitatively more distinguishable. Overall,
modes for all simulations. For the test data, where sys- the distances indicate that the presence of mode shapes in the
tem identification yields complete natural frequencies and data improves the reliability of the damage detection for the
mode shapes are packaged into a vector. Each vector con- given test structure. The experimental investigation of the
tains three natural frequencies, and 9 mode shape points data shows that when mode shapes are used in the machine
(3 modes × 3 mode shape points per mode), summing up to learning models, the damage detection can be more reliable.
12 data points. Since temperature was not recorded, this in-
formation is excluded in the input. No. of Data in Table 4
corresponds to the number of complete data vectors. The dis-
tribution of natural frequencies for the no damage and dam-
age cases are shown in Figure 9. Similar to the analytical in- 3.4.2 Structure 2
vestigation, both PCA and AE architectures are considered.
For the training of the machine learning model, the baseline This section utilizes a linear three-story three-dimensional
condition is used. Out of 50 data, 40 is used for training and frame located at Harbin Institute of Technology (HIT), China
10 for validation. After the data is standardized, the PCA (see Figure 11 (a)). The prototype structure has a base plan
model is trained with 6 components. The output of the AE with dimensions 1.84 m by 2.04 m and each story is 1.2 m
model is the three natural frequencies to be reconstructed. A tall. The structure is braced in one direction with inverted
neural network with the dimensions 12-8-8-3 is developed to v-brace (see Figure 11 (b)). A concrete slab weighting ap-
capture dominant features of the data. Rectified Linear Units proximately 250 kg is attached to each floor. Including the
(RELU) is used as the activation function on all the layers. mass of bare structure, total weight sums to 1066 kg. The
Figure 10 presents the novelty index for the experimental columns, beams and girders are made of structural steel with
data for both architectures with and without the introduction an elastic modulus estimated to be 220 GPa. More details
of mode shapes. In general, for all architectures, the dam- about the system identification and material properties of the
ages are distinguished from each other, given the fact that structure are discussed in Ozdagli [44] and Xi [63].
the stiffness degradation was as high as 90 percent. For all To evaluate the performance of the proposed method, tem-
figures, when two columns are damaged (Damage Case 2 perature gradient in three dimensions over the structure is
and 4) the index is higher compared to single column dam- modeled. To simulate the temperature gradient, first a finite
ages (Damage Case 1 and 3). The architectures not relying element (FE) model is established using OpenSees faithful
on mode shapes yield similar indices for Damage Case 2 and to the experimental structure in terms of boundary conditions
4, whereas the utilization of mode shapes as input return dis- and material properties. Each member of the model is crit-
tinguishable indices. Considering PCA, some instances of icized into 10 elements, resulting to 360 elements. Equa-
the novelty index for the Damage Case 1 at the absence of tion 28 is used to constitute the relationship between the am-
mode shapes leak to No Damage Case region. This behavior bient temperature and the material. The model is calibrated
Figure 9: Distribution of natural frequencies with varying ambient temperatures for each damage case in experimental data
(a) (b)
(c) (d)
Figure 10: Comparison of novelty indices for experimental data: (a) PCA - model set B (mode shapes not included); (b) PCA
- model set A (mode shapes included); (c) AE - model set B (mode shapes not included); (d) AE - model set A (mode shapes
included)
Table 5: Modified Euclidean Distances for Damage Cases with and without the Inclusion of Mode Shapes for Experimental Data
(Structure 1)
PCA PCA AE AE
without Mode Shape with Mode Shape without Mode Shape with Mode Shape
Damage Case 1 29.79 44.92 53.65 76.87
Damage Case 2 104.48 266.75 129.81 177.53
Damage Case 3 42.28 491.04 56.95 220.72
Damage Case 4 105.50 816.04 112.46 395.06
920 mm 920 mm
column
beam
1200 mm
girder
1200 mm
brace
1200 mm
y
m
m
40
20
x
1840 mm
(a) (b)
Figure 11: Three-story three-dimensional structure (Structure 2): (a) Experimental prototype; (b) idealization
Author Manuscript
to match the experimental modal properties at 15°C. The nat- more significant for PCA. For this case, reconstructing more
ural frequencies and mode shapes of the FE model are pre- features does not improve the detection of PCA when model
sented in Figure 12. set A and C are considered. As for AE, combination of
frequencies and mode shapes improves the detection for all
Four different damage conditions are established for this cases. Here reconstructing more features (model set C) in-
model with damage state varying between 5 and 10 percent. deed improve the detection compared to model set A. How-
The damage scenarios are summarized in Table 6 and illus- ever, it should be again noted that model set C requires more
trated in Figure 13. All damage cases consider a reduction of parameters to train the AE network and predict the detection;
stiffness only at the midspan element to localize the damage. thus, computationally more expensive.
ΔT = 12.5 °C ΔT = 10.0 °C
Sensor
Damage Location
ΔT = 7.5 °C
ΔT = 7.5 °C ΔT = 5.0 °C
ΔT = 5.0 °C
y
ΔT = 5.0 °C ΔT = 2.5 °C
ΔT = 2.5 °C
ΔT = 2.5 °C x ΔT = 0.0 °C
(a) (b)
(c) (d)
(e) (f)
Figure 14: Comparison of novelty indices for experimental data (Structure 2): (a) PCA - model set B (mode shapes not included);
(b) AE - model set B (mode shapes not included); (c) PCA - model set A (mode shapes included); (d) AE - model set A (mode
shapes included); (e) PCA - model set C (only mode shapes included); (f) AE - model set C (only mode shapes included)
Table 8: Modified Euclidean Distances for Damage Cases with and without the Inclusion of Mode Shapes for Experimental Data
(Structure 2)
as a viable tool in the field. Regarding the practicality of the also forced vibrations due to service loads. Finally, the pro-
method discussed herein, there are some considerations to posed architecture should be extended to locate damage by
be given. First, NExT/ERA based system identification re- relating mode shapes to the spatial data of the bridge under
quires long-time data measurements to capture higher modes environmental uncertainty.
successfully. Such data may not be always available. How-
ever, the pipeline is flexible enough to allow the use of alter- ACKNOWLEDGMENTS
native subspace system identification (SSI) methods such as
one proposed by Peeters and Roeck [47] and known to per- The financial support of this research is provided by the Na-
form well under noisy environments. Secondly, the success tional Science Foundation (CNS-1238959). The conclusions
of the method relies on the accuracy of the historical data. of this research represent solely that of the authors’.
A significant change in the system that cannot be identified
as damage, such as adding a damper or adding mass will in-
CODE AVAILABILITY STATEMENT
deed alter the features that were latent in the historical data.
In this case, a new model should be trained. Thirdly, tem- The codes to generate the results presented in this paper
perature gradients along the structure are common during the can be accessed from https://github.com/aliirmak/
field measurements. The training data should consider a wide ML-SHM.
range of measurements that capture the gradient pattern such
that the changes in the mode shapes will not cause novelty
indices to increase. Lastly, deep networks inherently require References
big data for the optimization of hyper-parameters. Extract-
[1] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen,
ing features from big data with a huge number of features
C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin,
using PCA can be computationally expensive since the data
S. Ghemawat, I. Goodfellow, A. Harp, G. Irving,
set is processed as a whole. Either, incremental PCA [48],
M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kud-
or AE should be used for a system with large sensor arrays.
lur, J. Levenberg, D. Mané, R. Monga, S. Moore,
Ideally, an ensemble of PCA and AE architecture should be
D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner,
considered to maximize the detection sensitivity if compu-
I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Va-
tational resources are allowing. Future work should include
sudevan, F. Viégas, O. Vinyals, P. Warden, M. Watten-
the investigation of the proposed approach in the presence of
berg, M. Wicke, Y. Yu, and X. Zheng. TensorFlow:
multiple environmental and operational variability in the lab-
Large-scale machine learning on heterogeneous sys-
oratory and in the field. Additionally, the data set should be
tems, 2015. URL https://www.tensorflow.org/.
diversified to include not only responses to ambient noise but
Software available from tensorflow.org.
[2] O. Abdeljaber, O. Avci, S. Kiranyaz, M. Gabbouj, using auto-associative neural networks. In Health Mon-
and D. J. Inman. Real-time vibration-based structural itoring of Structural and Biological Systems 2012, vol-
damage detection using one-dimensional convolutional ume 8348, page 834806. International Society for Op-
neural networks. Journal of Sound and Vibration, 388: tics and Photonics, 2012.
154–170, 2017.
[14] N. Dervilis, M. Choi, I. Antoniadou, K. M. Farinholt,
[3] Juan P Amezquita-Sanchez and Hojjat Adeli. A new S. G. Taylor, R. J. Barthorpe, G. Park, K. Worden, and
music-empirical wavelet transform methodology for C. R. Farrar. Novelty detection applied to vibration data
time–frequency analysis of noisy nonlinear and non- from a cx-100 wind turbine blade under fatigue loading.
stationary signals. Digital Signal Processing, 45:55– Journal of Physics: Conference Series, 382(1):012047,
68, 2015. 2012. URL http://stacks.iop.org/1742-6596/
382/i=1/a=012047.
[4] Juan P Amezquita-Sanchez, Hyo Seon Park, and Hoj-
jat Adeli. A novel methodology for modal parameters [15] S. W. Doebling, C. R. Farrar, M. B. Prime, and D. W.
identification of large smart structures using music, em- Shevitz. Damage identification and health monitoring
pirical wavelet transform, and hilbert transform. Engi- of structural and mechanical systems from changes in
neering Structures, 147:148–159, 2017. their vibration characteristics: A literature review. Los
Author Manuscript
[10] Varun Chandola, Arindam Banerjee, and Vipin Kumar. [20] K. Fukunaga and W. L. G. Koontz. Application of
Anomaly detection: A survey. ACM computing surveys the karhunen-loeve expansion to feature selection and
(CSUR), 41(3):15, 2009. ordering. IEEE Transactions on Computers, C-19(4):
311–318, April 1970. ISSN 0018-9340. doi: 10.1109/
[11] F. Chollet. Keras. https://keras.io, 2015. T-C.1970.222918.
[12] Arnaud Deraemaeker and Keith Worden. A compari- [21] I. Gonzalez. Application of monitoring to dynamic
son of linear approaches to filter out environmental ef- characterization and damage detection in bridges. PhD
fects in structural health monitoring. Mechanical sys- thesis, KTH, Structural Engineering and Bridges, 2014.
tems and signal processing, 105:1–15, 2018. QC 20140910.
[13] N. Dervilis, R. J. Barthorpe, I. Antoniadou, W. J. [22] I. Goodfellow, Y. Bengio, and A. Courville. Deep
Staszewski, and K. Worden. Damage detection in car- Learning. MIT Press, 2016. http://www.
bon composite material typical of wind turbine blades deeplearningbook.org.
[26] G. H. James, Thomas G Carne, and J. P. Lauffer. The ods. Journal of Computing in Civil Engineering, 31
natural excitation technique (next) for modal parame- (4):04017014, 2017.
ter extraction from operating wind turbines. Sandia
National Laboratories, Albuquerque, NM, Report No. [36] Yu Liang, Dalei Wu, Guirong Liu, Yaohang Li, Cuilan
SAND92-1666, 1993. Gao, Zhongguo John Ma, and Weidong Wu. Big
data-enabled multiscale serviceability analysis for ag-
[27] G. H. James, T. G. Carne, and J. P. Lauffer. The natu-
ing bridges☆. Digital Communications and Networks,
ral excitation technique (next) for modal parameter ex-
2(3):97–107, 2016.
traction from operating structures. Modal Analysis-the
International Journal of Analytical and Experimental [37] Zhibin Lin, Hong Pan, Xingyu Wang, and Mingli Li.
Modal Analysis, 10(4):260, 1995. Data-driven structural diagnosis and conditional assess-
ment: From shallow to deep learning. In Sensors
[28] Xiaomo Jiang and Hojjat Adeli. Pseudospectra, mu-
and Smart Structures Technologies for Civil, Mechani-
sic, and dynamic wavelet neural network for damage
cal, and Aerospace Systems 2018, volume 10598, page
detection of highrise buildings. International Journal
1059814. International Society for Optics and Photon-
for Numerical Methods in Engineering, 71(5):606–629,
ics, 2018.
2007.
[29] J. Juang and R. S. Pappa. An eigensystem realization [38] C. Liu and J. T. DeWolf. Effect of temperature
algorithm for modal parameter identification and model on modal variability of a curved concrete bridge un-
reduction. Journal of guidance, control, and dynamics, der ambient loads. Journal of Structural Engineer-
8(5):620–627, 1985. ing, 133(12):1742–1751, 2007. doi: 10.1061/(ASCE)
0733-9445(2007)133:12(1742).
[30] Sung Jig Kim, Richard Christenson, Brian Phillips, and
BF Spencer Jr. Geographically distributed real-time hy- [39] Thomas J Matarazzo, S Golnaz Shahidi, Minwoo
brid simulation of mr dampers for seismic hazard miti- Chang, and Shamim N Pakzad. Are todayâ C™s shm
gation. In Proceedings of the 20th Analysis and Compu- procedures suitable for tomorrowâ C™s bigdata? In
tation Specialty Conference, Chicago, IL, USA, pages Structural Health Monitoring and Damage Detection,
29–31, 2012. Volume 7, pages 59–65. Springer, 2015.
[31] M. A. Kramer. Nonlinear principal component anal- [40] MATLAB. version 9.5.0 (R2018b). The MathWorks
ysis using autoassociative neural networks. AIChE Inc., Natick, Massachusetts, 2018.
Journal, 37(2):233–243, 1991. doi: 10.1002/aic.
[41] F. McKenna, M. H. Scott, and G. L. Fenves. Non-
690370209. URL https://onlinelibrary.wiley.
linear finite-element analysis software architecture
com/doi/abs/10.1002/aic.690370209.
using object composition. Journal of Computing
[32] M. A. Kramer. Autoassociative neural net- in Civil Engineering, 24(1):95–107, 2010. doi:
works. Computers & Chemical Engineering, 16 10.1061/(ASCE)CP.1943-5487.0000002. URL
(4):313 – 328, 1992. ISSN 0098-1354. doi: https://ascelibrary.org/doi/abs/10.1061/
https://doi.org/10.1016/0098-1354(92)80051-A. URL %28ASCE%29CP.1943-5487.0000002.
[42] Mohsen Mehrjoo, Naser Khaji, H Moharrami, and [51] Hoon S. Effects of environmental and operational
A Bahreininejad. Damage detection of truss bridge variability on structural health monitoring. Philo-
joints using artificial neural networks. Expert Systems sophical Transactions of the Royal Society A: Math-
with Applications, 35(3):1122–1131, 2008. ematical, Physical and Engineering Sciences, 365
(1851):539–560, 2007. doi: 10.1098/rsta.2006.
[43] Y. Q. Ni, X. G. Hua, K. Q. Fan, and J. M. 1935. URL https://royalsocietypublishing.
Ko. Correlating modal properties with temper- org/doi/abs/10.1098/rsta.2006.1935.
ature using long-term monitoring data and sup-
port vector machine technique. Engineering Struc- [52] Ravid Shwartz-Ziv and Naftali Tishby. Opening the
tures, 27(12):1762 – 1773, 2005. ISSN 0141-0296. black box of deep neural networks via information.
doi: https://doi.org/10.1016/j.engstruct.2005.02.020. arXiv preprint arXiv:1703.00810, 2017.
URL http://www.sciencedirect.com/science/ [53] H. Sohn, M. Dzwonczyk, E. G. Straser, A. S. Kiremid-
article/pii/S0141029605002221. SEMC 2004 jian, K. H. Law, and T. Meng. An experimental study of
Structural Health Monitoring, Damage Detection and temperature effect on modal parameters of the alamosa
Long-Term Performance. canyon bridge. Earthquake engineering & structural
dynamics, 28(8):879–897, 1999.
Author Manuscript