Ocean Engineering 218 (2020) 108180

Contents lists available at ScienceDirect

Ocean Engineering
journal homepage: www.elsevier.com/locate/oceaneng

Condition-based maintenance for ship pumps subject to competing risks

under stochastic maintenance quality✩
Chaoqun Duan a , Zhongjie Li a ,∗, Fuqiang Liu b
a
School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China
b
College of Mechanical Engineering, Chongqing University, Chongqing 400044, China

ARTICLE INFO ABSTRACT

Keywords: Maintenance improvement is affected by many uncertain factors such as maintainer experience, machine
Condition-based maintenance deterioration state, maintenance tools and equipment, it is very hard to determine each restoration degree
Stochastic maintenance quality ahead of repair operation. Therefore, this paper considers a maintenance problem in the presence of competing
Maintenance optimization
risks (degradation and sudden failure) under stochastic maintenance quality for a ship pump. The degradation
Control-limit policy
process of ship pump is characterized by a non-homogeneous Gamma process and the hazard rate of sudden
failure is described by a proportional hazards model. We propose a two-level condition-based maintenance
(CBM) policy with stochastic maintenance quality. The policy combines degradation control limit and age
control limit under stochastic maintenance quality of Beta distribution, normal distribution and uniform
distribution, respectively, in maintenance decision-making. Maintenance optimization problem is formulated
and solved by a computational algorithm in a semi-Markov decision process (SMDP) framework, with the
objective of minimizing the long-run expected average cost. The proposed model is illustrated using a numerical
study and a real case study of ship pump, respectively, and comparative results show the effectiveness and
superiority of our approach.

1. Introduction The existing maintenance strategies can be classified as preven-

tive maintenance which repair is taken before failure, and corrective
Ship pump is the key component for water transportation and pol- maintenance which repair is performed after the failure occurs (Tang
lution draining for shipborne, and it is widely used in ocean industries et al., 2015). CBM combines preventive maintenance and corrective
in recent decades. However, extreme operation conditions on ocean, maintenance to perform repair according to the monitored operating
such as salt water, random shock and humid environment might cause conditions (Zou et al., 2019). The aim of CBM is to jointly determine
serious damage and fatigue on the ship pump, and consequently lead to when and what the maintenance should be taken. Some successful
unexpected breakdown, operational losses, and safety hazards (Li et al., examples have been implemented to demonstrate the effectiveness
2020). For these reasons, advanced maintenance technique (i.e., CBM) of CBM in real applications (Scheu et al., 2019; Tan et al., 2019).
is desired to perform based on the actual operating status of the Among these cases, maintenance models are categorized into two types:
pump before failure occurs (Cipollini et al., 2018). Current maintenance deterministic maintenance model and stochastic maintenance model.
practices use age-based or experience-based policy to maintain the ship Research works related to deterministic maintenance assumed that the
pump, which is not effective and may easily result in high risks of
machine can be repaired to a certain operating condition give a main-
pipe blockage and effluent backflow. Therefore, development of CBM
tenance action. The action restoration could be perfect or imperfect
for ship pump is significantly necessary and pressing in maintaining
for which brings the machine state to a brand new or to a condition
desirable reliability and satisfactory performance of shipborne (Hwang
between new and old, respectively. Maintenance with full (perfect)
et al., 2018; Garbatov et al., 2018; Dong and Frangopol, 2015).

✩ This document is the results of the research projects funded by the Shanghai Sail Plan for Talents Development, China (Grant No. 19YF1416000), the National
Natural Science Foundation of China (Grant No. 51905330; Grant No. 62001281), and the Fundamental Research Funds for the Central Universities, China (Grant
No. 2019CDJGFJX001).
∗ Corresponding author.
E-mail addresses: duancq@mie.utoronto.ca (C. Duan), lizhongjie@shu.edu.cn (Z. Li), liufq@cqu.edu.cn (F. Liu).

https://doi.org/10.1016/j.oceaneng.2020.108180
Received 28 April 2020; Received in revised form 20 August 2020; Accepted 27 September 2020
Available online 6 October 2020
0029-8018/© 2020 Elsevier Ltd. All rights reserved.
C. Duan et al. Ocean Engineering 218 (2020) 108180

restoration accounts for majority of CBM applications in various cases function to model the hazard rate of the machine. It was originally
such as boring machines, gear boxes, lasers, etc (Tang et al., 2015; Li proposed by Makis and Jardine (1992) to model a single failure mode
et al., 2019; Duan et al., 2020). While maintenance with partial (imper- (i.e., hard failure), and then extended by Tang et al. (2015) to model
fect) restoration has extensive variants including cost-related recovery the competing risks of hard failure and degradation failure in GaAs
model (Liu and Huang, 2010), failure rate reduction model (Nakagawa, lasers. Due to convenient concept and flexible covariate process, PHM
1988), age reduction model (Kijima et al., 1988), and hybrid model model was generally used in partially observable system modeling and
combining the failure rate reduction and the age reduction (Lin et al., it was extended with Wiener covariate (Zhao et al., 2016), Gamma
2001; Xia et al., 2015). Even although, models with assumption of covariate (Duan et al., 2018b), Markovian covariate (Wu and Ryan,
deterministic maintenance effects are quite convenient for maintenance 2011), normal covariate (Liu et al., 2017) to fit different degradation
modeling and optimization especially for modern complex machines, patterns in complex systems.
they usually do not truly reflect the real status of the system after Nevertheless, most PHMs in the literature have focused mainly
machine being repaired. This is due to that the maintenance restoration on the sudden/hard failure mode, the failure of degradation process
is largely affected by many uncertain or unknown factors such as has not been sufficiently discussed in the CBM community (Zhou
maintainer experience, machine deterioration state, maintenance tools et al., 2014; Duan and Deng, 2020). Furthermore, existing maintenance
and equipment. models for PHM considered the quality of maintenance on machine
As an extension to deterministic maintenance, stochastic mainte- degradation restoration as ideal, which is not true when we check the
nance considers that quality of maintenance is a random variable which degradation interactions for whole life cycle of machine (Liu et al.,
can be described by a certain distribution. This practical consider- 2017; Hu and Chen, 2020). Moreover, even some works have proposed
ation was proposed by Wu and Clementscroome (2005) to assume maintenance policy for competing risks in PHM, these maintenance
the machine condition falls into a fixed degradation interval with strategies only considered the control limit of machine hazard rate
constant probability after repair, for which the improvement follows or degradation state, an efficient two-level control-limit policy was
uniform distribution. Khatab and Aghezzaf (2016) further assumed not considered in maintenance decision-making (Tang et al., 2015;
maintenance quality follows Beta distributions to model age reduction Jafari et al., 2018). Up to now, very little work has been devoted
factors for binary state system maintenance scheduling. More gener- to analyzing the competing risks with stochastic maintenance quality.
ally, Duan et al. (2018a) assumed multiple maintenance options with To remedy this gap, in this paper we focus on the CBM problem of
different level of stochastic quality following Beta distributions in a competing risks with stochastic maintenance quality. Both degradation
selective maintenance scheduling for variety of multi-unit systems. and aging processes limits are considered as decision variables for
Another stochastic maintenance model is probabilistic repair model, developing CBM policy. Stochastic maintenance quality is also studied
which assumes that maintenance improvement is equivalent to minimal in our policy, and a detailed comparison by numerical and real case
repair (no change) with probability 𝑝, and equivalent to preventive studies is presented. The proposed approach differs from the existing
maintenance (a certain change) with probability 1 − 𝑝 (Brown and works in that: (a) It presents a competing risks model incorporating
Proschan, 1983; Shahraki et al., 2020). This considered maintenance degradation failure and hard failure. (b) It investigates the different
improvement factor is indeed a discrete random variable dependent stochastic maintenance qualities under new proposed CBM policy. (c)
on deteriorating state, maintenance cost and time. Besides these men- It develops a computational algorithm in SMDP framework to obtain
tioned works, few papers have been focused on stochastic maintenance the maintenance decision variables.
to develop and implement CBM models. The remaining parts of this paper are structured as follows. Sec-
The essential part to analyze and optimize a CBM policy is to tion 2 formulates the competing risks by modeling pump’s degradation
develop a suitable CBM model (Tang et al., 2015; Cipollini et al., 2018; as Gamma process and hard failure as PHM. Section 3 describes a new
Hwang et al., 2018). In CBM models, the evolution of the failure process CBM policy with three commonly used stochastic maintenance quality
and its underlying degradation path should be modeled according to models. Section 4 formulates and optimizes the CBM problem in the
practical deterioration of the machine (Asuquo et al., 2019). The failure SMDP framework. Section 5 presents a numerical study to illustrate
processes may interact each other internally and externally. The occur- the performance of three stochastic maintenance quality models in
rence of external shock might cause a sudden increase of deterioration the proposed CBM. Section 6 further verifies the proposed approach
level, while the deterioration might increase the hazard rate of random through a real case of ship pump. Section 7 summaries the conclusion
shock and lead to catastrophic failure. This interaction between shock and future research directions.
and gradual degradation has been studied in the literature by con-
sidering dependent competing risks process. Currently, research works 2. Competing risks modeling
focusing on system competing risks mainly apply three approaches
to model the failure interactions in CBM, namely, degradation-shock Generally, ship pump experiences a degradation failure when its
model, dependent hidden Markov model, and proportional hazards degradation state reaches a certain level, and a sudden failure which
model (PHM). Degradation-shock model was proposed earlier than occurs unexpectedly and stops pump from functioning. The ship pump
other approaches to model machine degradation and sudden failure is under periodical inspections to provide information of degradation
as two evolution processes, and defined that the sudden shock may levels at discrete time epochs 𝑡𝑛 = 𝑛ℎ (𝑛 = 1, 2, 3, …), which is the
cause ultimate random failure as well as degradation failure when the case in practices. Based on the nature of competing risks, the modeling
level of system deterioration crosses the predetermined limit. The main approach involves two distinct sub-modeling approaches: degradation
challenge for the degradation-shock model is the definition of failure signal modeling approach and sudden failure modeling approach.
interactions, which contains lots of ideal assumptions and cannot be
sufficiently verified by real cases. Dependent hidden Markov model 2.1. Model of system degradation
proposed by Akram and Makis (Khaleghei and Makis, 2016), who
assumed that the time to catastrophic failure and the sojourn time in The ship pump is subject to a continuous accumulation of wear
the healthy state follow a bivariate distribution. They have provided over time and its degradation (i.e., performance loss) path usually
detailed parameters estimation procedures and proved the effectiveness presents an non-decreasing and continuous pattern. The degradation
of the proposed approach with real applications in transmission units, path can be thought as the accumulation of an infinite number of small
gear boxes and machine tools (Li et al., 2019; Duan et al., 2020). PHM shocks, which increases with aging, and is characterized by a scalar
is a natural candidate to model the competing risks by incorporating random variable (Do et al., 2015). For such a system, Gamma process,
the degradation process and sudden failure as covariate and baseline which can be considered as an accumulation of a sequence of small

2
C. Duan et al. Ocean Engineering 218 (2020) 108180

increments, is an excellent candidate to describe such a monotonic Since we consider two failure modes, the pump’s time to failure is
degradation path. To model the degradation process, it is assumed the minimum failure time among the degradation failure time 𝜉1 and
the deterioration of the pump {𝑦(𝑡), 𝑡 ∈ 𝑅+ } evolves like a non- sudden failure time 𝜉2 and can be obtained as 𝜉 = min(𝜉1 , 𝜉2 ). With
stationary Gamma stochastic process 𝐺𝑎 (𝛼(𝑡), 𝛽), and the probability Eq. (2), the hazard rate of the pump at the 𝑛th inspection epoch is given
density function is given by by
𝛽 𝛼(𝑡) 𝛼(𝑡)−1 −𝛽𝑦 ℎ(𝑛ℎ, 𝑦(𝑛ℎ)) = ℎ0 (𝑛ℎ) · 𝜓(𝑦(𝑛ℎ); 𝛶 ) (3)
𝑓𝑦(𝑡) (𝑦) = 𝑦 𝑒 𝐼{𝑦≥0}
𝛤 (𝛼(𝑡))
The sudden failure is characterized by the PHM. Based on the
where 𝐼{𝑦≥0} is an indicator function 𝐼{𝑦≥0} = 1 if 𝑦 ≥ 0, otherwise,
competing risks model, we develop a maintenance model in the next
𝐼{𝑦≥0} = 0; 𝛼(𝑡) is non-linear shape parameter with form 𝛼(𝑡) = 𝑐𝑡𝑑 , and
∞ section.
𝛽 > 0 is the scale parameter. 𝛤 (𝛼(𝑡)) = ∫0 𝑢𝛼(𝑡)−1 𝑒−𝑢 𝑑𝑢 is the gamma
function. The degradation model has the following properties:
3. Maintenance model
• 𝑦(0) is a known initial state and is 0.
• For any given inspection times, the increments ▵ 𝑦𝑛 (𝑡) = 𝑦(𝑡𝑛 ) −
3.1. Maintenance policy
𝑦(𝑡𝑛+1 ) are independent;
• For any consecutive inspection points 0 ≤ 𝑡𝑛−1 < 𝑡𝑛 , the random
Consider a repairable pump system described as in Section 2. The
increment ▵ 𝑦𝑛 = 𝑦(𝑡𝑛 )−𝑦(𝑡𝑛+1 ) follows a Gamma distribution with shape
degradation state of the system is hidden and the degradation failure
parameter (𝛼(𝑡𝑛+1 ) − 𝛼(𝑡𝑛 )) and scale parameter 𝛽.
is non self-announcing. No indicator can exhibit the degradation state
The mean deterioration speed between two inspection points is
( ) ( ) except to do an inspection. The system starts working at time 𝑡 = 0
equal to 𝛼(𝑡𝑛+1 ) − 𝛼(𝑡𝑛 ) ∕𝛽, and the variance is equal to 𝛼(𝑡𝑛+1 ) − 𝛼(𝑡𝑛 )
2 and is inspected every ℎ time units. The inspections are assumed to be
∕𝛽 . 𝛼(𝑡) is a given, monotone increasing function. Parameter estimates
perfect and incurred a cost 𝐶𝐼 . Combining the degradation-based policy
for gamma process can be obtained by maximizing the logarithm of the
and age-based policy, a two-level control-limit policy is considered.
likelihood function of the deterioration increments with respect to 𝑐, 𝑑
Three types of maintenance actions are available on the pump, and the
and 𝛽 (Van Noortwijk, 2009; Cinlar et al., 1977).
CBM policy is as follows.
If the degradation failure threshold 𝐷𝑓 is given, the probability
of degradation failure for the next inspection interval (𝑛ℎ, (𝑛 + 1)ℎ] • Corrective maintenance is performed, which consumes 𝑇𝑆 time
conditioning on the current observation (i.e., inspected degradation units and 𝐶𝑆 cost, at inspection epochs when the degradation
level) 𝑦(𝑡) (𝑦(𝑡) < 𝐷𝑓 ) can be expressed by: state exceeds degradation failure threshold 𝐷𝑓 .
+∞ • Corrective maintenance is performed with the expected time 𝑇𝐻
𝑃 𝑟(𝑇 < (𝑛 + 1)ℎ|𝑦(𝑡), 𝑦(𝑡) < 𝐷𝑓 ) = 𝑓 (𝑧|𝛼(𝑛ℎ), 𝛽)𝑑𝑧 (1) and cost 𝐶𝐻 right after a sudden failure occurs.
∫𝐷𝑓 −𝑦(𝑡)
• Preventive maintenance is triggered at the expected time 𝑇𝑃 and
2.2. Model of sudden failure cost 𝐶𝑃 when the degradation level of the pump system reaches
degradation control limit 𝑊 at inspection epoch.
When a ship pump is functioning, the sudden failure is more likely • A preventive replacement is triggered at the expected time 𝑇𝑅
to occur with service time. The sudden failure can interrupt functioning and cost 𝐶𝑅 when the age of the pump system exceeds the age
state and cause unexpected catastrophic loss. In this paper, we assume control limit 𝑇 , where 𝑇 = 𝑘ℎ and 𝑇 ≤ 𝑇̄ , 𝑇̄ is the maximum
that the occurrence rate of sudden failures is described by the pro- predetermined age.
portional hazards model (see e.g., Kumar and Klefsjo (1994)), which A corrective maintenance or replacement action can be either a
explicitly includes both the effect of the age and the degradation state. physical replacement or an overhaul or repair such that the pump
It can be expressed by the following relation: system is as-good-as-new after the replacement. Even though both the
ℎ(𝑡, 𝑦(𝑡)) = ℎ0 (𝑡) · 𝜓(𝑦(𝑡); 𝛶 ) (2) corrective maintenance actions for degradation failure and sudden fail-
ure put the pump system back in the as-good-as-new state, they are not
where ℎ0 (𝑡) denotes baseline failure rate at time 𝑡. 𝜓(𝑦(𝑡); 𝛶 ) is positive necessarily identical in practices because the corrective maintenance
function with a regression parameter 𝛶 depending on the degradation for sudden failure is unplanned and is considered to be more complex
state of 𝑦(𝑡). The survival function is given by and more expensive. The preventive maintenance is assumed to be
( 𝑡 ) imperfect with stochastic restoration impact on the system degradation
𝑃 (𝜉2 > 𝑡|𝑦(𝑠), 0 ≤ 𝑠 ≤ 𝑡) = exp − ℎ0 (𝑠) · 𝜓(𝑦(𝑠); 𝛶 )𝑑𝑠 state, which is practical in real situations. The decisions variables in
∫0
this maintenance policy are age control limit 𝑇 and degradation con-
where 𝜉2 is the time of sudden failure. For the pump which has not trol limit 𝑊 for initiating preventive maintenance/replacement. The
failed by time 𝑛ℎ, if the latest observation is 𝑦(𝑛ℎ) = 𝑌𝑛 , the conditional objective is to jointly optimize the age control limit and degradation
reliability function of surviving beyond 𝑛ℎ + 𝑡 for any 0 < 𝑡 < ℎ can be control limit to minimize the long-run expected average cost per unit
approximated as: time.
𝑅(𝑌𝑛 , 𝑛, 𝑡) = 𝑃 (𝜉2 > 𝑛ℎ + 𝑡|𝜉2 > 𝑛ℎ, 𝑌𝑛 )
( 𝑛ℎ+𝑡 ) 3.2. Stochastic quality of maintenance
= exp − ℎ0 (𝑠) · 𝜓(𝑌𝑛 ; 𝛶 )𝑑𝑠
∫𝑛ℎ
( ) In the literature related to maintenance strategy (Keizer et al., 2017;
𝑛ℎ+𝑡
= exp −𝜓(𝑌𝑛 ; 𝛶 ) · ℎ0 (𝑠)𝑑𝑠 Alaswad and Xiang, 2017), a system is typically assumed to be restored
∫𝑛ℎ to a certain condition by a deterministic quality of a maintenance
Thus, the expected sojourn time for pump to keep functioning action. However, due to the expertise differences and maintenance
during interval [𝑛ℎ, 𝑛ℎ + 𝑎) is given by environment factors, the restoration of a maintenance action is not
𝑎 fixed. Throughout this paper, the quality of maintenance is assumed
𝜏(𝑌𝑛 , 𝑛, 𝑎) = 𝑅(𝑌𝑛 , 𝑛, 𝑡)𝑑𝑡 to be stochastic and characterized by a random variable 𝜂. After a
∫0
( ) maintenance operation, the degradation level 𝑦(𝑡) of the pump system is
𝑎 𝑛ℎ+𝑡
= exp −𝜓(𝑌𝑛 ; 𝛶 ) · ℎ0 (𝑠)𝑑𝑠 𝑑𝑡 reduced to 𝑦(𝑡)×(1−𝜂) where 𝜂 for 0 ≤ 𝜂 ≤1 is the degradation reduction
∫0 ∫𝑛ℎ coefficient. Accordingly, the degradation of system becomes to initial

3
C. Duan et al. Ocean Engineering 218 (2020) 108180

through the degradation measurement after repair. After repair oper-

ation, the system degradation will restore system to state 𝑦(𝑡)(1 − 𝜂).
Besides Beta distribution, two models have appeared in literature to
model the quality of maintenance, which are normal distribution and
uniform distribution. For the maintenance quality follows normal dis-
tribution, the stochastic maintenance quality is defined on the support
[0, 1], the corresponding probability density function is given by
⎧ √ 1 exp( −(𝜂−𝜇𝜂 )2 ) for 0 < 𝜂 < 1
⎪ 2𝜋𝜎 2 2𝜎 2
⎪ 0 1 −(𝜂−𝜇 )2
𝑓 (𝜂) = ⎨∫−∞ √ exp( 2𝜎 2𝜂 )𝑑𝜂 for 𝜂 = 0
2
⎪ 2𝜋𝜎
2
⎪∫ +∞ √ 1 exp( −(𝜂−𝜇2𝜂 ) )𝑑𝜂 for 𝜂 = 1
⎩ 1 2𝜋𝜎 2 2𝜎

where 𝜇𝜂 is the expected value of random variable 𝜂, 𝜎𝜂 is the stan-

dard deviation. Since normal distribution is defined on the support
(−∞, +∞), we assume that the possible values of the reduction coef-
ficient 𝜂 fall within [0, 1]. For the values falling into intervals smaller
than 0, they are considered to be 0, while for values greater than 1 are
Fig. 1. Stochastic maintenance quality modeled as Beta distribution under different
considered to be 1. Thus, we re-define the probability density function,
expected improvements.
and the new 𝑓 (𝜂) is a hybrid function. The expected average value 𝜇𝜂
can be obtained as
+∞ 1
1 −(𝜂 − 𝜇)2 1 −(𝜂 − 𝜇)2
𝜇𝜂 = √ exp( )𝑑𝜂 + √ exp( ) ⋅ 𝜂𝑑𝜂
∫1 2 2𝜎 2 ∫0 2 2𝜎 2
2𝜋𝜎 2𝜋𝜎
and the standard deviation 𝜎𝜂 can be calculated by
1 0
1 −(𝜂 − 𝜇)2
𝜎𝜂2 = (𝜂 − 𝜇𝜂 )2 ⋅ 𝑓 (𝜂)𝑑𝜂 + 𝜇𝜂2 √ exp( )𝑑𝜂
∫0 ∫−∞ 2𝜎 2
2𝜋𝜎 2
+∞
1 −(𝜂 − 𝜇)2
+ (1 − 𝜇𝜂 )2 √ exp( )𝑑𝜂
∫1 2𝜎 2
2𝜋𝜎 2
Fig. 2 shows the stochastic maintenance model under normal dis-
tribution with cases of (𝜇𝜂 , 𝜎𝜂 ) = (0.25, 0.05), (𝜇𝜂 , 𝜎𝜂 ) = (0.5, 0.05), and
(𝜇𝜂 , 𝜎𝜂 ) = (0.75, 0.05), respectively.
The third stochastic model used in preventive maintenance qual-
ity is uniform distribution, which has equal probability for different
value of 𝜂. The probability density function is defined on the support
(−∞, +∞) as well, and can be expressed by
{
1
Fig. 2. Stochastic maintenance quality modeled as normal distribution under different
𝑚
for 0 ≤ 𝜂 ≤ 1
expected improvements.
𝑓 (𝜂) =
0 otherwise
where 𝑚 is the number of intervals divided in the support [0, 1]. The
support is defined by the two parameters, 0 and 1, which are the
new status if its deterioration is reset to zero with reduction coeffi-
cient 𝜂=1, while it becomes as bad as old if the reduction coefficient minimum and maximum values of maintenance improvement. The
𝜂=0 (Duan et al., 2018a; Zhao et al., 2017). quality of maintenance is assumed to have probability of 1∕𝑚 in each
Therefore, a random variable 𝜂 is used to represent the new degra- small interval. There are some limitations for applying uniform dis-
dation reduction factor, which characterizes the stochastic maintenance tribution to model the quality of imperfect maintenance. On the one
quality. Since the probability density function 𝑓 (𝜂) and cumulative hand, the improvement of maintenance is fixed and it does not support
distribution function 𝐹 (𝜂) of the stochastic maintenance quality are multiple levels maintenance policy. On the other hand, the restoration
defined on the support [0, 1], we use a Beta distribution to model the of maintenance is equal for each deteriorating state, which is violated
random variable 𝜂, which is defined as: with maintenance practices in real world situations.
1
𝑓 (𝜂) = 𝜂 𝑎−1 (1 − 𝜂)𝑏−1 4. Maintenance optimization in SMDP framework
𝐵(𝑎, 𝑏)
where 𝑎 and 𝑏, for 𝑎 > 0, 𝑏 > 0 are shape parameters, 𝐵(𝑎, 𝑏) =
1
∫0 𝑡𝑎−1 (1 − 𝑡)𝑏−1 𝑑𝑡 is the Beta function. The expected value 𝜇𝜂 and the The objective of our maintenance policy is to find the optimal
standard deviation 𝜎𝜂 of random variable 𝜂 are values of the control limits 𝑊 ∗ , 𝑇 ∗ to initiate a preventive mainte-
nance/replacement that minimizes the long-run expected average cost
𝑎 𝑎⋅𝑏
𝜇𝜂 = 𝑎+𝑏
, 𝜎𝜂 = (𝑎+𝑏)2 (𝑎+𝑏+1) per unit time. By renewal theory, the cost minimization problem is
equivalent to finding optimal control limits 𝑊 ∗ , 𝑇 ∗ such that
The stochastic quality of maintenance under different expected
improvements is depicted in Fig. 1. In the figure, the curves corre- 𝐸𝑊 ∗ ,𝑇 ∗ (𝐶𝐶) 𝐸𝑊 ,𝑇 (𝐶𝐶)
𝑔(𝑊 ∗ , 𝑇 ∗ ) = = inf (4)
sponds to the probability density functions of the stochastic mainte- 𝐸𝑊 ∗ ,𝑇 ∗ (𝐶𝐿) 𝐸𝑊 ,𝑇 (𝐶𝐿)
nance improvements with expected values and standard deviations on
subject to:
(0.25, 0.01), (0.5, 0.01), (0.75, 0.01), respectively
In this work, the improvement of maintenance is set on degradation 0 < 𝑊 < 𝐷𝑓 (5)
process. This is due to that the degradation is observable and mea-
surable, the maintainer can evaluate the maintenance improvement 0 < 𝑇 < 𝑇̄ (6)

4
C. Duan et al. Ocean Engineering 218 (2020) 108180

where 𝐶𝐶 and 𝐶𝐿 denote the cycle cost and cycle length, respectively.
(𝐿−𝑙+1)𝛥
Constraints (5) and (6) confine the ranges of the control limits 𝑊 , 𝑇 . 𝛽 𝛼((𝑛+1)ℎ)−𝛼(𝑛ℎ)
𝑝𝑙𝐷𝑓 (ℎ) = 1 − 𝑦𝛼((𝑛+1)ℎ)−𝛼(𝑛ℎ)−1 𝑒−𝛽𝑦 𝑑𝑦
To optimize the maintenance policy, we now develop a compu- ∫0 𝛤 (𝛼((𝑛 + 1)ℎ) − 𝛼(𝑛ℎ))
tational algorithm in the SMDP framework to determine the optimal The first term indicates the probability of system degradation tran-
control limits 𝑊 ∗ , 𝑇 ∗ . The degradation process at each inspection siting to state 𝑙′ under control limit 𝑊 . The second term means the
epoch is monitored. Suppose that the pump is operational at the 𝑛th probability of system degradation going to state in [𝑊 , 𝐷𝑓 ). The last
inspection epoch 𝑛ℎ, then we compute the hazard rate using Eq. (3). term shows the probability that system degradation exceeds 𝐷𝑓 at the
Generally, the computation of the SMDP requires discretization of next inspection. Next, we present the derivation of the required SMDP
the possible range of 𝑦(𝑛) ∈ [0, 𝐷𝑓 ). We partition the interval of transition probabilities.
the degradation level [0, 𝐷𝑓 ) into a fixed large 𝐿 subintervals with
constant length 𝛥 = 𝐷𝑓 ∕𝐿 and define [𝐷𝑓 , ∞) as the failure state 𝐹 . 4.1. Developing the transition probabilities
The number of subintervals 𝐿 should be selected properly. The more
accurate results will be obtained by choosing the larger number of The SMDP transition probabilities for the states defined in the
subintervals 𝐿, however the computational time increases. Therefore, previous paragraphs are calculated as follows.
the number of subintervals should be selected big enough to get the 1. Assume that the system is in the state (𝑙, 𝑛), where 𝑙𝛥 < 𝐷𝑓 ,
precise results in a reasonable time. If at inspection point, the observed 𝑛ℎ < 𝑇 . Then the transition probability to the next decision epoch at
system degradation 𝑦(𝑡) is in [𝑖𝐷𝑓 ∕𝐿, (𝑖 + 1)𝐷𝑓 ∕𝐿), then the degradation state (𝑙′ , 𝑛 + 1) where 𝑙′ 𝛥 < 𝐷𝑓 , 𝑛ℎ < 𝑇 , is given by
process is defined to be in state 𝑖. If the observed system degradation (
𝑃(𝑙,𝑛),(𝑙′ ,𝑛+1) = 𝑃 𝑦(𝑛 + 1) ∈ [𝑙′ 𝛥, (𝑙′ + 1)𝛥), 𝜉2
exceeds the degradation level 𝑊 , then the process is defined to be )
in 𝑃 𝑀 state. Before calculating the transition probabilities, the exact > (𝑛 + 1)ℎ|𝑦(𝑛) ∈ [𝑙𝛥, (𝑙 + 1)𝛥), 𝜉2 > 𝑛ℎ
definitions of the state space for the SMDP are required: The right side of formula is the probability that the system degra-
State (0,0): the system is in as-good-as-new state. dation will not exceed the degradation control limit 𝑊 , the age of the
State (𝑙, 𝑛): the first component denotes the state of the observed system will not exceed age control limit 𝑇 , and the system will not fail
degradation at time 𝑛ℎ and the second component is the operating age in the next inspection interval. Then, this probability can be calculated
of the system at time 𝑛ℎ. as
State 𝑃 𝑀: the value of observable degradation crosses the degrada- ( )
tion level 𝑊 , a preventive maintenance action is performed. 𝑃 𝑦(𝑛 + 1) ∈ [𝑙′ 𝛥, (𝑙′ + 1)𝛥), 𝜉2 > (𝑛 + 1)ℎ|𝑦(𝑛) ∈ [𝑙𝛥, (𝑙 + 1)𝛥), 𝜉2 > 𝑛ℎ
(
State 𝐹 : the value of observable degradation reaches the degrada- = 𝑃 𝑦(𝑛 + 1) ∈ [𝑙′ 𝛥, (𝑙′ + 1)𝛥)|𝜉2 > (𝑛 + 1)ℎ, 𝑦(𝑛) ∈
tion failure threshold 𝐷𝑓 , a corrective maintenance action is performed. )
[𝑙𝛥, (𝑙 + 1)𝛥), 𝜉2 > 𝑛ℎ
State 𝑅: the age of the system reaches the age level 𝑇 , a replacement ( )
is initiated. · 𝑃 𝜉2 > (𝑛 + 1)ℎ|𝑦(𝑛) ∈ [𝑙𝛥, (𝑙 + 1)𝛥), 𝜉2 > 𝑛ℎ
Therefore, the state space of SMDP can be represented as 𝑆 = = 𝑃𝑙𝑙′ (ℎ) · 𝑅(𝑙, 𝑛, ℎ)
𝑆1 ∪ 𝑆2 ∪ 𝑆3 ∪ 𝑆4 ∪ 𝑆5 , where 𝑆1 = {(𝑦, 𝑛)|𝑦 ∈ [0, 𝑊 ), 𝑛ℎ < 𝑇 , 𝑛 ∈ 𝑁},
2. If the system degradation level crosses the degradation control
𝑆2 = {𝑃 𝑀}, 𝑆3 = {𝐹 }, 𝑆4 = {𝑅}, 𝑆5 = {(0, 0)}. With the definition of
limit, i.e, 𝑙′ 𝛥 ≥ 𝑊 , at the next decision epoch given the current state
the state space of the SMDP, for cost minimization problem, the SMDP
(𝑙, 𝑛), where 𝑙𝛥 < 𝐷𝑓 , 𝑛ℎ < 𝑇 . Then the system goes to the 𝑃 𝑀 state
is determined by the following quantities (Tijms, 1994):
and the transition probability is given by
𝑃𝑖𝑗 =the probability that at the next epoch the system will be in
(
state 𝑗 ∈ 𝑆 given the present state is 𝑖 ∈ 𝑆. 𝑃(𝑙,𝑛),(𝑃 𝑀) =𝑃 𝑦(𝑛 + 1) ∈ [𝑊 , 𝐷𝑓 )|𝜉2 > (𝑛 + 1)ℎ, 𝑦(𝑛) ∈
𝜏𝑖 =the expected sojourn until the next decision epoch given the )
[𝑙𝛥, (𝑙 + 1)𝛥), 𝜉2 > 𝑛ℎ
current state 𝑖 ∈ 𝑆. ( )
𝐶𝑖 =the expected cost incurred until the next decision epoch given · 𝑃 𝜉2 > (𝑛 + 1)ℎ|𝑦(𝑛) ∈ [𝑙𝛥, (𝑙 + 1)𝛥), 𝜉2 > 𝑛ℎ
the current state 𝑖 ∈ 𝑆. =𝑃𝑙𝑊 (ℎ) · 𝑅(𝑙, 𝑛, ℎ)
With the quantities defined above, for fixed control limits 𝑊 ∗ ,
3. If the degradation state of the system reaches the failure threshold
𝑇 ∗ , the long-run expected average cost 𝑔(𝑊 ∗ , 𝑇 ∗ ) can be obtained by
𝐷𝑓 at the next decision epoch given the current state (𝑙, 𝑛), where
solving the following linear equations:
∑ 𝑙𝛥 < 𝐷𝑓 and 𝑛ℎ < 𝑇 . Then the system goes to the 𝐹 state and the
𝑣𝑘 = 𝐶𝑘 − 𝑔(𝑊 ∗ , 𝑇 ∗ ) + 𝑃𝑘𝑙 𝑣𝑙 , for each 𝑘 ∈ 𝑆 transition probability is given by
𝑙∈𝑆
𝑃(𝑙,𝑛),(𝐹 ) = 𝑃𝑙𝐷𝑓 (ℎ) · 𝑅(𝑙, 𝑛, ℎ)
𝑣𝑠 = 0, for some 𝑠∈𝑆 (7)
where 𝑃𝑦𝐷𝑓 (ℎ) is the probability that system will exceed the failure
where 𝑣𝑘 is the relative value. The optimal control limits 𝑊 ∗ , 𝑇 ∗ and
threshold 𝐷𝑓 at inspection interval ℎ.
corresponding minimal average cost will be obtained by iteratively
4. If the age of the system exceeds the age level 𝑇 at the next
solving (7). The remainder of the mathematical analysis in this section
decision epoch given the current state (𝑙, 𝑛), where 𝑙𝛥 < 𝐷𝑓 and 𝑛ℎ < 𝑇 .
is devoted to deriving closed form expressions for the SMDP quantities
Then the system goes to the 𝑅 state and the transition probability is
𝑃𝑖𝑗 , 𝜏𝑖 , 𝐶𝑖 .
given by
To proceed with the SMDP approach, the transition probability for
Gamma degradation process of pump is required. Since the state of 𝑃(𝑙,𝑛),(𝑅) = 𝑅(𝑙, 𝑛, ℎ)
deterioration process is only observable at inspection points, it may
5. If the sudden failure occurs, the corrective maintenance will be
transit to any state at any time between two inspections. For the
set up which brings the system back to state (0, 0), then the transition
degradation process, the transition probability from state 𝑙 at current
probability is given by
inspection 𝑛ℎ to 𝑙′ at the next inspection (𝑛 + 1)ℎ can be formulated as:
(𝑙′ −𝑙+1)𝛥 𝑃(𝑙,𝑛),(0,0) = 1 − 𝑅(𝑙, 𝑛, ℎ)
𝛽 𝛼((𝑛+1)ℎ)−𝛼(𝑛ℎ)
𝑝𝑙𝑙′ (ℎ) = 𝑦𝛼((𝑛+1)ℎ)−𝛼(𝑛ℎ)−1 𝑒−𝛽𝑦 𝑑𝑦
∫(𝑙′ −𝑙)𝛥 𝛤 (𝛼((𝑛 + 1)ℎ) − 𝛼(𝑛ℎ)) 6. New cycle begins when corrective maintenance action or age
replacement is taken, which is given by
(𝐿−𝑙+1)𝛥
𝛽 𝛼((𝑛+1)ℎ)−𝛼(𝑛ℎ)
𝑝𝑙𝑊 (ℎ) = 𝑦𝛼((𝑛+1)ℎ)−𝛼(𝑛ℎ)−1 𝑒−𝛽𝑦 𝑑𝑦 𝑃(𝐹 ),(0,0) = 1
∫(𝑙′ −𝑙)𝛥 𝛤 (𝛼((𝑛 + 1)ℎ) − 𝛼(𝑛ℎ))

5
C. Duan et al. Ocean Engineering 218 (2020) 108180

𝑃(𝑅),(0,0) = 1

Since the quality of preventive maintenance is stochastic, instead of

returning back to (0, 0), the system will go to state (𝑙′ , 𝑛 + 1) with proba-
bility 𝑃(𝑃 𝑀),(𝑙′ ,𝑛+1) after preventive maintenance action. The 𝑃(𝑃 𝑀),(𝑙′ ,𝑛+1)
is given by
1−(𝑙′ −1)∕𝑙
𝑃(𝑃 𝑀),(𝑙′ ,𝑛+1) = 𝑙 ⋅ (1 − 𝜂)𝑑𝐹 (𝜂)
∫1−𝑙′ ∕𝑙
where 𝑙 is the degradation state before the preventive maintenance,
with impact between 1 − 𝑙′ ∕𝑙 and 1 − (𝑙′ − 1)∕𝑙, the degradation state
returns back to the state 𝑙′ with probability 𝑃(𝑃 𝑀),(𝑙′ ,𝑛+1) .
In the next subsection, the expected sojourn times and the expected
cost in each state are determined.

4.2. Computing the mean sojourn times and expected costs

We consider different scenarios of mean sojourn times and expected

costs for SMDP states, which are presented as follows. Fig. 3. Optimal control limits 𝑊 ∗ under different stochastic maintenance quality
The mean sojourn time and expected cost incurred until the next models.
inspection epoch at state (𝑙′ , 𝑛 + 1), where 𝑙′ 𝛥 < 𝐷𝑓 , 𝑛ℎ < 𝑇 given the
current state (𝑙, 𝑛), where 𝑙𝛥 < 𝐷𝑓 , 𝑛ℎ < 𝑇 , are given by
ℎ hazard rate is assumed to follow a Weibull distribution ℎ0 (𝑡) = 𝜌𝑡𝜌−1 ∕𝜃 𝜌 ,
𝜏(𝑙,𝑛) = 𝑅(𝑙, 𝑛, 𝑡)𝑑𝑡 + (1 − 𝑅(𝑙, 𝑛, ℎ)) ⋅ 𝑇𝐻 where 𝜃 = 1.84, 𝜌 = 5.5, and 𝜓(𝑦(𝑡)) = exp(0.5𝑦(𝑡)). The system is
∫0
periodically inspected with ℎ = 1, and maximum predetermined age
𝐶(𝑙,𝑛) =𝐶𝐼 + (1 − 𝑅(𝑙, 𝑛, ℎ)) ⋅ 𝐶𝐻 is 𝑇̄ = 50. We consider the following cost and time parameters in our
case to illustrate our maintenance policy: 𝐶𝐼 = 5, 𝐶𝑆 = 400, 𝐶𝐻 = 800,
When the degradation process crosses the failure threshold 𝐷𝑓 , the
𝐶𝑅 = 600, 𝐶𝑝 = 150; 𝑇𝑆 = 2.5, 𝑇𝐻 = 4, 𝑇𝑅 = 2.5, 𝑇𝑃 = 1.
mean sojourn time and expected cost in the failure state 𝐹 are given
To investigate how the maintenance decision will change with the
by
quality of the maintenance, we compare three stochastic models with
𝜏(𝐹 ) =𝑇𝑆 the deterministic case under different values of 𝜇𝜂 from 0.1 to 0.9 under
𝐶(𝐹 ) =𝐶𝐼 + 𝐶𝑆 𝜎𝜂 = 0.01. The maintenance quality in deterministic maintenance is
set to be same as the expected impact of stochastic cases. For notation
When the system reaches the age control limit 𝑇 , the mean sojourn convenience, we denote the case of deterministic maintenance as DM.
time and expected cost in the replacement state 𝑅 are given by Since the expected value of uniform distribution is 0.5 regardless
of parameters changing, the comparison is only made for case of
𝜏(𝑅) =𝑇𝑅
𝜇𝜂 = 0.5 for uniform distribution. The optimal control limits 𝑊 ∗ , 𝑇 ∗
𝐶(𝑅) =𝐶𝐼 + 𝐶𝑅 and corresponding expected average cost are obtained using SMDP
When the system degradation level reaches the control limit 𝑊 , the algorithm. Table 1 indicates the results of optimal control limits 𝑊 ∗ ,
mean sojourn time and expected cost in the preventive maintenance 𝑇 ∗ considering different expected maintenance improvements for four
state 𝑃 𝑀 are given by compared cases. It can be seen that the control limits increase with the
value of 𝜇𝜂 , which means the higher expected restoration, the looser
𝜏(𝑃 𝑀) =𝑇𝑃 control policy. Compare with deterministic maintenance quality, the
𝐶(𝑃 𝑀) =𝐶𝐼 + 𝐶𝑃 Beta distribution has a relative lower control limits of 𝑊 ∗ , 𝑇 ∗ , while
normal distribution has a stricter control at the beginning and a looser
By substituting the above quantities in (7), the optimal values of control after 𝜇𝜂 = 0.5. The uniform distribution, which is not so efficient
𝑊 , 𝑇 and corresponding minimal long-run expected average cost per in practical maintenance quality modeling, has the lowest 𝑊 ∗ , 𝑇 ∗ with
unit time can be obtained by solving the linear equations in the SMDP the highest average cost 32.5 among four cases under 𝜇𝜂 = 0.5.
algorithm. In Fig. 3, it is clear that the Beta distribution has a very similar
An appropriate 𝐿 to partition the continuous state space 𝑦(𝑡), 𝑡 ≥ 0 control limit 𝑊 ∗ with case of deterministic maintenance quality, al-
is needed in this algorithm. A basic approach rule is developed to though some small changes in [0.5,0.9]. The control limit 𝑊 ∗ of normal
determine 𝐿, which is given by the following expression: distribution shows a high fluctuation and control policy is conservative
as expected maintenance improvement is below 0.5.
𝑔(𝑊 ∗ , 𝑇 ∗ |𝐿 = 2𝑈 +1 ) − 𝑔(𝑊 ∗ , 𝑇 ∗ |𝐿 = 2𝑈 ) ≤ 𝜀 (8)
Fig. 4 shows the minimum expected average cost under differ-
where 𝐿 is positive integers, and 𝜀 is the selected small number. ent expected maintenance improvement impact. We can see that the
maintenance quality following Beta distribution has smaller expected
5. Comparison of maintenance quality through a numerical study average cost than deterministic case when 𝜇𝜂 ≤ 0.3, and the expected
average cost becomes almost equal with the deterministic maintenance
The purpose of our work is twofold. One is to investigate the case as 𝜇𝜂 > 0.3. In general, the expected average cost of Beta distri-
stochastic maintenance quality, the other is to validate the effects of the bution is either higher or lower than deterministic maintenance case,
proposed CBM model. In this section, we compare the proposed stochas- but their costs are very close with increase of expected maintenance
tic maintenance models with deterministic maintenance model through impact.
a numerical study. We assume that the pump system deterioration In contrast, the expected average cost of normal distribution shows
follows a PHM with dependent degradation failure and sudden failure. a highly undulated trend, which is much greater than other two cases
The degradation process is described by a non-homogeneous gamma as 𝜇𝜂 ≤ 0.4, and much lower as 𝜇𝜂 > 0.5. The reason of this result is that
process with shape parameter 𝛼(𝑡) = 0.63𝑡1.09 , and the scale parameter the normal distribution is a hybrid distribution which is defined on the
𝛽 = 1.18. The failure threshold 𝐷𝑓 for the system is 18. The baseline internal [0, 1]. When the mean of distribution approaches to 0.1 or to

6
C. Duan et al. Ocean Engineering 218 (2020) 108180

Table 1
Comparison of maintenance quality models under different expected improvements.
Quality Control limit 𝑊 ∗ Control limit 𝑇 ∗ Average cost
DM Beta Normal Uniform DM Beta Normal Uniform DM Beta Normal Uniform
0.1 5.72 5.72 5.44 – 21 21 19 – 39.2 38.5 41.5 –
0.2 7.95 7.95 7.39 – 25 25 24 – 35.7 35.1 36.8 –
0.3 10.46 10.46 10.16 – 29 29 27 – 31.8 31.68 32.9 –
0.4 11.02 11.02 10.74 – 32 32 33 – 31.2 31.15 33.5 –
0.5 12.14 11.86 12.42 11.58 34 33 36 32 30.1 30.35 28.7 32.5
0.6 12.14 11.86 12.42 – 34 33 36 – 30 30.3 28.6 –
0.7 12.14 12.14 12.42 – 34 33 36 – 29.8 30.1 28.4 –
0.8 12.98 12.70 13.26 – 35 34 36 – 28.0 28.1 27.4 –
0.9 12.98 12.70 13.26 – 35 34 36 – 27.9 28.1 27.3 –

Table 2
Estimates for model parameters.
Parameters 𝜌 𝜃 𝛶 𝑐 𝑑 𝛽
Estimates 2.75e+004 1.72 0.45 0.82 1.23 1.29

Table 3
The minimum expected average cost and the control limits under different number of
intervals 𝐿.
Discretization level (L) 32 64 128 256
𝑊∗ 17.31 17.64 17.61 17.60
𝑇∗ 18 18 18 18
Average cost 258.89 255.78 255.12 255.12

rate ℎ0 (𝑡) = 𝜌𝑡𝜌−1 ∕𝜃 𝜌 , and covariate process is Gamma process with

non-linear shape parameter 𝛼(𝑡) = 𝑐𝑡𝑑 . Using maximum log-likelihood
method, the parameter estimates for both degradation process and
Fig. 4. Minimal expected average cost under different stochastic maintenance quality
models. baseline function are given in Table 2.
The stochastic maintenance quality for Beta distribution and normal
distribution are estimated using the procedure in Zhang et al. (2015).
0.9, the 𝜇𝜂 has a large probability to fall into the value of 0 or 1. When Through the goodness-of-fit tests, we find that Anderson–Darling statis-
𝜇𝜂 equals 0, it has no any restoration effect but incurs maintenance tic (AD) value of Beta distribution is 0.218, and 𝑝-value is above 0.500,
cost, which is relative costly in all cases. Whereas if 𝜇𝜂 equals 1, while AD value of normal distribution is 18.33 and 𝑝-value is less
the restoration is perfect with imperfect preventive maintenance cost, than 0.05. For the uniform distribution, the test indicates that the
which is very cost effective. maintenance data does not follow this specified distribution. There-
Through the comparison, the maintenance decision is influenced by fore, we pick the Beta distribution as the better fit for modeling the
the maintenance quality. Different maintenance quality models lead to stochastic maintenance quality. The expected impact of maintenance
different control-limit policies and operating costs. The normal distri- improvement 𝜇𝜂 and corresponding standard derivation 𝜎𝜂 are 0.78
bution model has a great impact on the maintenance decision-making, and 0.0093, respectively. The pump preventive and corrective time
whereas the Beta distribution model has not changed too much with parameters are given by 𝑇𝑆 = 24, 𝑇𝐻 = 36, 𝑇𝑅 = 24 and 𝑇𝑃 = 12 hours,
deterministic case in the proposed maintenance policy. respectively. The maintenance costs considered in the experiment are
given by: 𝐶𝐼 = 50, 𝐶𝑆 = 3000, 𝐶𝐻 = 7200, 𝐶𝑅 = 4000, 𝐶𝑃 = 1500.
6. Case study We partition the continuous degradation process [0, 25] into 𝐿 =
64 intervals for maintenance optimization in SMDP framework. The
6.1. Experimental results minimum expected average cost per inspection cycle is 255.78 with
optimal 𝑊 ∗ = 17.64, 𝑇 ∗ = 12960. It means the replacement is taken
To further investigate the performance of the proposed model in real at 18th inspection epoch if no sudden failure and degradation failure
practical applications, we study a real diagnostic data set coming from occur. The computational algorithm in SMDP framework for acquiring
the ship diaphragm pump. The major function of this pump is to realize optimal decision variables (𝑊 ∗ , 𝑇 ∗ ) requires an appropriate discretiza-
the control of liquid draining and dewatering in the ship. The pump tion 𝐿 to get the sufficiently precise results with a reasonable time.
is subject to gradual degradation failure and random sudden failure. Using the proposed model in Section 4, the expected average cost with
The degradation failure does not halt the pump operating condition optimal control limits under different discretization 𝐿 are demonstrated
but leads to inferior performance. The sudden failure will abruptly in Table 3. When 𝐿 ≥ 32, the expected average cost has not changed
stop the pump from operating condition. The degradation of the ship much. Using the stopping rule defined in (8), we can conclude that
pump leads to performance loss of diaphragm operating. The pump 𝐿 ≥ 32 the partition leads to a sufficient degree of precision.
diaphragm operating frequency is periodically inspected every 720 h It should be noted that the AI-based techniques can be used here
for collecting degradation information and the maximum lifetime of to find the optimal decision variables instead of SMDP algorithm, but
the pump is 18 000 h. The pump starts operating with frequency of 35 the quality of the solution is not stable. For this reason, we choose
per second and is considered to be failed if the degradation (namely, SMDP algorithm to optimize the maintenance problem since it has com-
operating frequency loss) increases to 25 per second. The baseline plete mathematical derivations and can obtain the stable and optimal
function of PHM is assumed to be Weibull distribution with the hazard decision variables.

7
C. Duan et al. Ocean Engineering 218 (2020) 108180

Fig. 5. The illustration of the proposed approach to monitor the degradation process Fig. 6. Changes of maintenance quality on minimal expected average cost.
at each inspection epoch.

Table 4 variable has a lower average cost for each maintenance quality parame-
The optimal parameters and the long-run expected average cost per inspection cycle. ter 𝜇𝜂 . This is due to that the real maintenance improvement in the case
Maintenance policy Degradation Age limit Hazard rate Average study is not constant, modeling as a constant may either overestimate
limit limit cost
or underestimate the pump actual condition, and therefore lead to high
Proposed approach 17.64 12 960 – 255.78 risk and some operational losses.
Deterministic case 18.02 13 680 – 264.58
From above comparison, the main gains of our work are as fol-
Hazard rate-based – – 5.63 × 10−4 279.83 lows. (a) Through investigating the maintenance data, the stochastic
policy
maintenance model fits better to maintenance quality than determin-
Age-based policy – 14 400 – 348.87
istic model. (b) Through comparison with the deterministic case, the
Run to failure – – – 580.96
proposed CBM model with Beta distribution leads to much stricter
maintenance control policy with lower control limits. (c) Through
comparison with other commonly used policies, the proposed approach
6.2. Case comparison with two-level control limit is cost effective and policy efficient.

We first compare the proposed model with the maintenance policy 7. Conclusions
which assumes that maintenance improvement is constant. Same pa-
rameters as in the previous case study are considered. The maintenance In this paper, we have developed a stochastic CBM model for ship
impact is set to be the expected value of Beta distribution 𝜇𝜂 = 0.78. The pump whose sensor information can be modeled by stochastic process,
discretization 𝐿 is set to 64 as well. The minimum long-run expected and sudden failure information can be modeled by PHM, which is
average cost using the maintenance policy with constant maintenance influenced by both the age and the degradation state of the pump.
impact has increased to 264.58 with 𝑊 ∗ = 18.02, 𝑇 ∗ = 13680, which The quality of maintenance in CBM is considered as stochastic and
are higher than the optimal average cost and control limits obtained three stochastic models are presented. Under these conditions, a two-
using the proposed approach with stochastic maintenance model. level CBM policy is proposed, which combines the degradation control
limit and age control limit of competing risks. The decision variables in
Fig. 5 shows the degradation path plot for the one of data histo-
CBM optimization is determined by a computational algorithm in SMDP
ries with the optimal decision variables 𝑊 ∗ , 𝑇 ∗ . The stars show the
framework. The effectiveness of the stochastic CBM is demonstrated by
degradation state at each inspection epoch. Once the degradation state
a numerical study and a real case of ship pump.
exceeds the degradation control limit 𝑊 ∗ , preventive maintenance will
The aim of this proposed model is to develop a general and prac-
be performed. For the proposed policy considering stochastic mainte-
tical model for maintaining operating machines. It has presented new
nance effect, this occurred at the 16th inspection epoch, which is one
techniques and broadened the applications of stochastic CBM. In this
inspection interval earlier than its deterministic counterpart. Similar
paper, there are four main contributions of the proposed model. (I)
result can be seen with age control limit.
The first contribution is the modeling of stochastic maintenance quality
We also have verified the superiority of the proposed approach with three random distributions, respectively, and the investigations of
by comparing the results with the hazard rate-based policy, age-based advantages and disadvantage of these models. (II) The second contribu-
policy, run-to-failure policy. Considering the same parameters as in the tion is the modeling of the competing risks using PHM and illustration
previous example, the minimum long-run expected average cost using of the applicability of the model. (III) The third contribution is the de-
the conventional maintenance policies increased to 279.83, 348.87, velopment of a two-level CBM model, and illustration of the superiority
580.96, respectively, which are higher than the optimal average cost of the new approach. (IV) The fourth contribution is the development
obtained using the proposed approach with degradation and age con- of computational algorithms in SMDP framework to solve optimization
trol limits. Table 4 shows the long-run expected average cost per unit problem of the stochastic CBM under competing risks.
time and the optimal decision variables for the proposed approach and The proposed approach can be used in both single-unit and multi-
conventional maintenance policies. unit machines. It provides new insights into the optimization modeling
When the maintenance impact varies, the optimal maintenance of stochastic maintenance quality. Further extension of the model to
policy will be changed. The relationship of minimal expected average develop the stochastic CBM for a multi-unit machine with non-identical
cost for varying maintenance quality parameter 𝜇𝜂 is depicted in Fig. 6. units under competing risks is a suitable topic for future research. More-
In this real case study, the maintenance impact modeled as a random over, another interesting research direction could be to consider the

8
C. Duan et al. Ocean Engineering 218 (2020) 108180

dynamic environments in competing risks modeling, which is valuable Khaleghei, A., Makis, V., 2016. Reliability estimation of a system subject to condition
for applications of stochastic CBM under ocean environment. monitoring with two dependent failure modes. IIE Trans. 48 (11), 1058–1071.
Khatab, A., Aghezzaf, E., 2016. Selective maintenance optimization when quality of
imperfect maintenance actions are stochastic. Reliab. Eng. Syst. Saf. 150, 182–189.
CRediT authorship contribution statement Kijima, M., Morimura, H., Suzuki, Y., 1988. Periodical replacement problem without
assuming minimal repair. European J. Oper. Res. 37 (2), 194–203.
Chaoqun Duan: Conceptualization of this study, Methodology, Al- Kumar, D., Klefsjo, B., 1994. Proportional hazards model: a review. Reliab. Eng. Syst.
gorithm, Writing. Zhongjie Li: Project administration, Investigation. Saf. 44 (2), 177–188.
Li, X., Makis, V., Zhao, Z., Zuo, H., Duan, C., Zhang, Y., 2019. Optimal bayesian
Fuqiang Liu: Data curation, Investigation. maintenance policy for a gearbox subject to two dependent failure modes. Qual.
Reliab. Eng. Int. 35 (2), 659–676.
Declaration of competing interest Li, Z., Zi, Y., Chen, J., Pan, J., Zhou, Z., 2020. Performance-guided maintenance
policy and optimization for transmission system of shipborne antenna with multiple
components. Ocean Eng. 199, 106903.
The authors declare that they have no known competing finan-
Lin, D., Zuo, M.J., Yam, R.C.M., 2001. Sequential imperfect preventive maintenance
cial interests or personal relationships that could have appeared to models with two categories of failure modes. Nav. Res. Logist. 48 (2), 172–183.
influence the work reported in this paper. Liu, Y., Huang, H., 2010. Optimal selective maintenance strategy for multi-state systems
under imperfect maintenance. IEEE Trans. Reliab. 59 (2), 356–367.
References Liu, B., Liang, Z., Parlikad, A.K., Xie, M., Kuo, W., 2017. Condition-based maintenance
for systems with aging and cumulative damage based on proportional hazards
model. Reliab. Eng. Syst. Saf. 168, 200–209.
Alaswad, S., Xiang, Y., 2017. A review on condition-based maintenance optimization
Makis, V., Jardine, A.K.S., 1992. Optimal replacement in the proportional hazards
models for stochastically deteriorating system. Reliab. Eng. Syst. Saf. 157, 54–63.
model. INFOR: Inf. Syst. Oper. Res. 30 (1), 172–183.
Asuquo, M.P., Wang, J., Zhang, L., Phylipjones, G., 2019. Application of a multiple
Nakagawa, T., 1988. Sequential imperfect preventive maintenance policies. IEEE Trans.
attribute group decision making (magdm) model for selecting appropriate main-
Reliab. 37 (3), 295–298.
tenance strategy for marine and offshore machinery operations. Ocean Eng. 179,
Scheu, M.N., Tremps, L., Smolka, U., Kolios, A., Brennan, F.P., 2019. A systematic fail-
246–260.
ure mode effects and criticality analysis for offshore wind turbine systems towards
Brown, M., Proschan, F., 1983. Imperfect repair. J. Appl. Probab. 20 (4), 851–859.
integrated condition based maintenance strategies. Ocean Eng. 176, 118–133.
http://dx.doi.org/10.2307/3213596.
Shahraki, A.F., Yadav, O.P., Vogiatzis, C., 2020. Selective maintenance optimization for
Cinlar, E., Bazant, Z.P., Osman, E.M., 1977. Stochastic process for extrapolating concrete
multi-state systems considering stochastically dependent components and stochastic
creep. J. Eng. Mech. 103 (ASCE 13447 Proceeding).
imperfect maintenance actions. Reliab. Eng. Syst. Saf. 196, 106738.
Cipollini, F., Oneto, L., Coraddu, A., Murphy, A.J., Anguita, D., 2018. Condition-based
Tan, Y., Niu, C., Tian, H., Hou, L., Zhang, J., 2019. A one-class svm based approach
maintenance of naval propulsion systems with supervised data analysis. Ocean Eng.
for condition-based maintenance of a naval propulsion plant with limited labeled
149, 268–278.
data. Ocean Eng. 193, 106592.
Do, P., Voisin, A., Levrat, E., Iung, B., 2015. A proactive condition-based maintenance
Tang, D., Yu, J., Chen, X., Makis, V., 2015. An optimal condition-based maintenance
strategy with both perfect and imperfect maintenance actions. Reliab. Eng. Syst.
policy for a degrading system subject to the competing risks of soft and hard failure.
Saf. 133, 22–32.
Comput. Ind. Eng. 83, 100–110.
Dong, Y., Frangopol, D.M., 2015. Risk-informed life-cycle optimum inspection and
Tijms, H.C., 1994. Stochastic Models: An Algorithmic Approach, Vol. 303. John Wiley
maintenance of ship structures considering corrosion and fatigue. Ocean Eng. 101,
& Sons Inc.
161–171.
Van Noortwijk, J., 2009. A survey of the application of gamma processes in
Duan, C., Deng, C., 2020. Prognostics of health measures for machines with aging
maintenance. Reliab. Eng. Syst. Saf. 94 (1), 2–21.
and dynamic cumulative damage. IEEE/ASME Trans. Mechatronics 1–12. http:
Wu, S., Clementscroome, D., 2005. Preventive maintenance models with random
//dx.doi.org/10.1109/TMECH.2020.2995757.
maintenance quality. Reliab. Eng. Syst. Saf. 90 (1), 99–105.
Duan, C., Deng, C., Gharaei, A., Wu, J., Wang, B., 2018a. Selective maintenance
Wu, X., Ryan, S.M., 2011. Optimal replacement in the proportional hazards model with
scheduling under stochastic maintenance quality with multiple maintenance
semi-markovian covariate process and continuous monitoring. IEEE Trans. Reliab.
actions. Int. J. Prod. Res. 56 (23), 7160–7178.
60 (3), 580–589.
Duan, C., Makis, V., Deng, C., 2018b. An integrated framework for health mea-
Xia, T., Jin, X., Xi, L., Ni, J., 2015. Production-driven opportunistic maintenance for
sures prediction and optimal maintenance policy for mechanical systems using a
batch production based on mam–apb scheduling. European J. Oper. Res. 240 (3),
proportional hazards model. Mech. Syst. Signal Process. 111, 285–302.
781–790.
Duan, C., Makis, V., Deng, C., 2020. A two-level bayesian early fault detection for
Zhang, M., Gaudoin, O., Xie, M., 2015. Degradation-based maintenance decision using
mechanical equipment subject to dependent failure modes. Reliab. Eng. Syst. Saf.
stochastic filtering for systems under imperfect maintenance. European J. Oper.
193, 106676.
Res. 245 (2), 531–541.
Garbatov, Y., Sisci, F., Ventura, M., 2018. Risk-based framework for ship and structural
Zhao, X., Alkhalifa, K.N., Hamouda, A.M.S., Nakagawa, T., 2017. Age replacement
design accounting for maintenance planning. Ocean Eng. 166, 12–25.
models: A summary with new perspectives and methods. Reliab. Eng. Syst. Saf.
Hu, J., Chen, P., 2020. Predictive maintenance of systems subject to hard failure based
161, 95–105.
on proportional hazards model. Reliab. Eng. Syst. Saf. 196, 106707.
Zhao, S., Makis, V., Chen, S., Li, Y., 2016. On-line reliability assessment for an electronic
Hwang, H.J., Lee, J.H., Hwang, J.S., Jun, H., 2018. A study of the development of a
system subject to condition monitoring. In: 2016 IEEE International Conference on
condition-based maintenance system for an lng fpso. Ocean Eng. 164, 604–615.
Prognostics and Health Management (ICPHM). pp. 1–6.
Jafari, L., Naderkhani, F., Makis, V., et al., 2018. Joint optimization of maintenance
Zhou, Q., Son, J., Zhou, S., Mao, X., Salman, M.A., 2014. Remaining useful life
policy and inspection interval for a multi-unit series system using proportional
prediction of individual units subject to hard failure. IIE Trans. 46 (10),
hazards model. J. Oper. Res. Soc. 69 (1), 36–48.
1017–1030.
Keizer, M.C.A.O., Flapper, S.D.P., Teunter, R.H., 2017. Condition-based maintenance
Zou, G., Banisoleiman, K., Gonzalez, A., Faber, M.H., 2019. Probabilistic investigations
policies for systems with multiple dependent components: A review. European J.
into the value of information: A comparison of condition-based and time-based
Oper. Res. 261 (2), 405–420.
maintenance strategies. Ocean Eng. 188, 106181.